Equipment oil leakage detection method, device, equipment and storage medium in dark light environment

Abstract

The present invention relates to the field of automatic detection technology, and discloses a method, device, equipment and storage medium for detecting oil leakage of equipment in a dark light environment. The method for detecting oil leakage of equipment in a dark light environment comprises: acquiring an image of the equipment to be tested, performing image enhancement processing by a multi-scale retinal enhancement algorithm with color restoration; performing oil pan detection on the enhanced image and taking a screenshot to obtain an oil pan image; performing histogram equalization processing by a contrast-limited adaptive histogram equalization algorithm to obtain an oil pan balanced image; converting the oil pan balanced image into an HSV color space, and locating the oil pan area image according to a preset color threshold; converting the oil pan area image into an oil pan grayscale image, and calculating the oil pan area variance based on the oil pan grayscale image and judging whether the equipment to be tested is leaking oil. The present invention realizes automatic detection of equipment oil leakage in a low-light environment, reduces production costs, and significantly improves the efficiency and accuracy of equipment oil leakage detection.

Description

Technical Field

[0001] The present invention relates to the field of automatic detection, and in particular to a method, device, equipment and storage medium for detecting oil leakage of equipment in a dark light environment. Background Art

[0002] In order to reduce labor costs and realize the construction of smart factories, cigarette factories use artificial intelligence technologies such as machine vision to further improve the intelligent automation level of production in the tobacco workshop and improve production and management efficiency. Cigarette factories need to strictly control the production environment without open flames. Oil leakage may occur in the motor bearings on the top of the tobacco conveying trolley, posing a safety hazard.

[0003] In the existing technology, it is usually completely dependent on manual on-the-job inspection to see if the equipment is leaking oil. The inspection cost is high, the efficiency is low, and there is a problem of missed detection. Especially in low-light environments, the lighting is uneven, the oil pan area is not visible, and the inspection accuracy is low. Summary of the invention

[0004] The main purpose of the present invention is to provide a method, device, equipment and storage medium for detecting oil leakage of equipment in a dark light environment, aiming to solve the technical problems of high cost and low efficiency of oil leakage detection of equipment in the prior art.

[0005] A first aspect of the present invention provides a method for detecting oil leakage of equipment in a dark light environment, comprising:

[0006] Acquire an image of the device under test, and enhance the image of the device under test by a multi-scale retinal enhancement algorithm with color restoration to obtain an enhanced image;

[0007] Performing oil pan detection on the enhanced image, and taking screenshots according to the detection results to obtain an oil pan image;

[0008] Performing histogram equalization processing on the oil pan image by using a contrast-limited adaptive histogram equalization algorithm to obtain an oil pan equalized image;

[0009] Converting the RGB color space of the oil pan balanced image into the HSV color space, and locating the oil pan area image of the oil pan balanced image according to a preset color threshold;

[0010] The oil pan area image is converted into an oil pan grayscale image, and the oil pan area variance is calculated based on the oil pan grayscale image, and whether the device to be tested is leaking oil is determined according to the calculation result.

[0011] Optionally, in a first implementation of the first aspect of the present invention, the acquiring the image of the device under test, and performing enhancement processing on the image of the device under test by a multi-scale retinal enhancement algorithm with color restoration to obtain the enhanced image includes:

[0012] Acquire an image of the device under test, and input the image of the device under test into a multi-scale retinal enhancement algorithm model with color restoration;

[0013] The multi-scale retinal enhancement algorithm model with color restoration performs enhancement processing on the image of the device under test based on the Retinex algorithm to obtain an enhanced image;

[0014] The mathematical expression of the Retinex algorithm is as follows:

[0015] I=L*R

[0016] Among them, I represents the image signal observed or received by the camera; L represents the illumination component of the ambient light; and R represents the reflection component of the target object carrying the image detail information.

[0017] Optionally, in a second implementation of the first aspect of the present invention, the performing oil pan detection on the enhanced image and taking a screenshot according to the detection result to obtain the oil pan image includes:

[0018] Input the enhanced image into the backbone network ShuffleNetV2 for feature extraction to obtain image features of the enhanced image;

[0019] Inputting the image features into a path aggregation network for multi-scale feature fusion processing to obtain a feature fusion result;

[0020] Based on the feature fusion result, a preset detection head is used to perform detection to obtain a coordinate frame of the oil receiving pan;

[0021] Screenshot processing is performed based on the coordinate frame of the oil receiving pan to obtain an oil receiving pan image.

[0022] Optionally, in a third implementation of the first aspect of the present invention, performing histogram equalization processing on the oil receiving pan image by using a contrast-limited adaptive histogram equalization algorithm to obtain an oil receiving pan equalized image includes:

[0023] Performing grayscale calculation on each pixel of the oil receiving pan image and pixels in a preset rectangular range around it to obtain a grayscale histogram within each rectangular range;

[0024] Clipping the grayscale histogram based on a preset threshold value, and evenly distributing the clipping value in the grayscale interval to obtain a balanced grayscale histogram;

[0025] Normalizing each of the balanced grayscale histograms, and calculating a cumulative distribution function of the normalized balanced grayscale histogram;

[0026] Mapping calculations are performed on each of the balanced grayscale histograms based on each of the cumulative distribution functions, and the calculation results are combined to obtain a balanced image of the oil receiving pan.

[0027] Optionally, in a fourth implementation of the first aspect of the present invention, converting the RGB color space of the oil receiving pan balanced image into the HSV color space, and locating the oil receiving pan area image of the oil receiving pan balanced image according to a preset color threshold comprises:

[0028] Converting the RGB color space of the oil pan balanced image into the HSV color space;

[0029] The converted oil pan balanced image is subjected to color extraction based on the preset color threshold to obtain an oil pan area image, wherein the portion of the oil pan area image that is not within the preset color threshold is represented by black.

[0030] Optionally, in a fifth implementation of the first aspect of the present invention, converting the oil pan area image into an oil pan grayscale image, calculating the oil pan area variance based on the oil pan grayscale image, and judging whether the equipment to be tested is leaking oil according to the calculation result includes:

[0031] Converting the oil pan area image into an oil pan grayscale image;

[0032] Calculating the grayscale average of pixels with non-zero grayscale values, and calculating the oil pan area variance based on the grayscale average;

[0033] If the oil receiving pan area variance is greater than a preset threshold, it is confirmed that the device under test is leaking oil; otherwise, it is confirmed that the device under test is not leaking oil.

[0034] Optionally, in a sixth implementation of the first aspect of the present invention, after acquiring the image of the device under test and performing enhancement processing on the image of the device under test by a multi-scale retinal enhancement algorithm with color restoration to obtain an enhanced image, the method further includes:

[0035] Using a real-time scene text detection algorithm based on differential binarization to perform text detection on the enhanced image to obtain a text area image;

[0036] Using a convolutional neural network to extract features from the text region image to obtain a feature sequence, and inputting the feature sequence into a recurrent neural network for prediction to obtain a prediction sequence;

[0037] The predicted sequence is corrected by using a CTC algorithm to obtain a text recognition result, and the device information of the device under test is determined based on the text recognition result.

[0038] A second aspect of the present invention provides an oil leakage detection device for equipment in a dark light environment, comprising:

[0039] An enhancement module is used to acquire an image of the device under test, and enhance the image of the device under test by a multi-scale retinal enhancement algorithm with color restoration to obtain an enhanced image;

[0040] A detection module, used to perform oil pan detection on the enhanced image, and perform screenshot processing according to the detection result to obtain an oil pan image;

[0041] an equalization module, configured to perform histogram equalization processing on the oil pan image by using a contrast-limited adaptive histogram equalization algorithm to obtain an oil pan equalized image;

[0042] A positioning module, used for converting the RGB color space of the oil receiving pan balanced image into the HSV color space, and locating the oil receiving pan area image of the oil receiving pan balanced image according to a preset color threshold;

[0043] The calculation module is used to convert the oil pan area image into an oil pan grayscale image, calculate the oil pan area variance based on the oil pan grayscale image, and determine whether the device to be tested is leaking oil according to the calculation result.

[0044] Optionally, in a first implementation manner of the second aspect of the present invention, the enhancement module is specifically used to:

[0045] Acquire an image of the device under test, and input the image of the device under test into a multi-scale retinal enhancement algorithm model with color restoration;

[0046] The multi-scale retinal enhancement algorithm model with color restoration performs enhancement processing on the image of the device under test based on the Retinex algorithm to obtain an enhanced image;

[0047] The mathematical expression of the Retinex algorithm is as follows:

[0048] I=L*R

[0049] Among them, I represents the image signal observed or received by the camera; L represents the illumination component of the ambient light; and R represents the reflection component of the target object carrying the image detail information.

[0050] Optionally, in a second implementation of the second aspect of the present invention, the detection module is specifically used to:

[0051] Input the enhanced image into the backbone network ShuffleNetV2 for feature extraction to obtain image features of the enhanced image;

[0052] Inputting the image features into a path aggregation network for multi-scale feature fusion processing to obtain a feature fusion result;

[0053] Based on the feature fusion result, a preset detection head is used to perform detection to obtain a coordinate frame of the oil receiving pan;

[0054] Screenshot processing is performed based on the coordinate frame of the oil receiving pan to obtain an oil receiving pan image.

[0055] Optionally, in a third implementation of the second aspect of the present invention, the balancing module is specifically configured to:

[0056] Performing grayscale calculation on each pixel of the oil receiving pan image and pixels in a preset rectangular range around it to obtain a grayscale histogram within each rectangular range;

[0057] Clipping the grayscale histogram based on a preset threshold value, and evenly distributing the clipping value in the grayscale interval to obtain a balanced grayscale histogram;

[0058] Normalizing each of the balanced grayscale histograms, and calculating a cumulative distribution function of the normalized balanced grayscale histogram;

[0059] Mapping calculations are performed on each of the balanced grayscale histograms based on each of the cumulative distribution functions, and the calculation results are combined to obtain a balanced image of the oil receiving pan.

[0060] Optionally, in a fourth implementation of the second aspect of the present invention, the positioning module is specifically used to:

[0061] Converting the RGB color space of the oil pan balanced image into the HSV color space;

[0062] The converted oil pan balanced image is subjected to color extraction based on the preset color threshold to obtain an oil pan area image, wherein the portion of the oil pan area image that is not within the preset color threshold is represented by black.

[0063] Optionally, in a fifth implementation of the second aspect of the present invention, the calculation module is specifically used to:

[0064] Converting the oil pan area image into an oil pan grayscale image;

[0065] Calculating the grayscale average of pixels with non-zero grayscale values, and calculating the oil pan area variance based on the grayscale average;

[0066] If the oil receiving pan area variance is greater than a preset threshold, it is confirmed that the device under test is leaking oil; otherwise, it is confirmed that the device under test is not leaking oil.

[0067] Optionally, in a sixth implementation of the second aspect of the present invention, the device for detecting oil leakage of equipment in a dark light environment further includes:

[0068] The recognition module is used to perform text detection on the enhanced image by using a real-time scene text detection algorithm based on differential binarization to obtain a text area image; perform feature extraction on the text area image by using a convolutional neural network to obtain a feature sequence, and input the feature sequence into a recurrent neural network for prediction to obtain a prediction sequence; correct the prediction sequence by using a CTC algorithm to obtain a text recognition result, and determine the device information of the device to be tested based on the text recognition result.

[0069] A third aspect of the present invention provides an electronic device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the electronic device executes the above-mentioned device oil leakage detection method in a dark light environment.

[0070] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned method for detecting oil leakage of equipment in a dark light environment.

[0071] In the technical solution provided by the present invention, an image of the device to be tested is obtained, and a multi-scale retinal enhancement algorithm with color restoration is used for enhancement processing to obtain an enhanced image; the enhanced image is subjected to oil pan detection and screenshot processing to obtain an oil pan image; the oil pan image is subjected to histogram equalization processing by a contrast-limited adaptive histogram equalization algorithm to obtain an oil pan balanced image; the RGB color space of the oil pan balanced image is converted to the HSV color space, and the oil pan area image of the oil pan balanced image is located according to a preset color threshold; the oil pan area image is converted to an oil pan grayscale image, and the oil pan area variance is calculated based on the oil pan grayscale image, and whether the device to be tested is leaking oil is determined according to the calculation result. The present invention realizes automatic detection of equipment oil leakage, reduces production costs, significantly improves the efficiency and accuracy of equipment oil leakage detection, and can accurately detect equipment oil leakage in low-light environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 A schematic diagram of an embodiment of a method for detecting oil leakage of equipment in a dark light environment according to an embodiment of the present invention;

[0073] Figure 2 It is a schematic diagram of another embodiment of the method for detecting oil leakage of equipment in a dark light environment according to an embodiment of the present invention;

[0074] Figure 3 It is a schematic diagram of an embodiment of a device for detecting oil leakage of equipment in a dark light environment according to an embodiment of the present invention;

[0075] Figure 4It is a schematic diagram of another embodiment of the device for detecting oil leakage of equipment in a dark light environment according to an embodiment of the present invention;

[0076] Figure 5 FIG. 1 is a schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The embodiments of the present invention provide a method, device, equipment and storage medium for detecting oil leakage of equipment in a dark light environment, which realizes automatic detection of oil leakage of equipment, reduces production costs, significantly improves the efficiency and accuracy of oil leakage detection of equipment, and can accurately detect oil leakage of equipment in a low-light environment.

[0078] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0079] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the method for detecting oil leakage of equipment in a dark light environment in an embodiment of the present invention includes:

[0080] 101. Acquire an image of a device under test, and enhance the image of the device under test by a multi-scale retinal enhancement algorithm with color restoration to obtain an enhanced image;

[0081] It is understandable that the execution subject of the present invention may be an oil leakage detection device for equipment in a dark light environment, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0082] In this embodiment, the image of the device under test is an image in which the oil receiving pan or other oil leakage collection device of the device under test can be seen, and the method for obtaining the image of the device under test is not limited.

[0083] In this embodiment, the Multi-Scale Retinex with Color Restoration (MSRCR) is an image enhancement algorithm used to enhance the image. It can better complete the compression of the dynamic range, enhance the details of the darker areas, maintain the consistency of color, and eliminate color cast.

[0084] Optionally, in one embodiment, the above step 101 includes:

[0085] Acquire an image of the device under test, and input the image of the device under test into a multi-scale retinal enhancement algorithm model with color restoration;

[0086] The multi-scale retinal enhancement algorithm model with color restoration performs enhancement processing on the image of the device under test based on the Retinex algorithm to obtain an enhanced image;

[0087] The mathematical expression of the Retinex algorithm is as follows:

[0088] I=L*R

[0089] Among them, I represents the image signal observed or received by the camera; L represents the illumination component of the ambient light; and R represents the reflection component of the target object carrying the image detail information.

[0090] In this embodiment, the brightness of an object observed and perceived in a Retinex (Retina-Cortex) algorithm depends on the lighting of the environment and the reflection of the irradiated light by the surface of the object.

[0091] 102. Perform oil pan detection on the enhanced image, and perform screenshot processing according to the detection result to obtain an oil pan image;

[0092] In this embodiment, the oil pan or other oil leakage collection device in the enhanced image is detected by a target detection method, and the target detection method is not limited.

[0093] In this embodiment, the detection result is an adjacent coordinate frame of the oil pan or other oil leakage collection device, and a screenshot is taken based on the coordinate frame to obtain an image of the oil pan or other oil leakage collection device.

[0094] Optionally, in one embodiment, NanoDet which can perform real-time online detection is used for target detection.

[0095] Optionally, in one embodiment, the step 102 includes:

[0096] Input the enhanced image into the backbone network ShuffleNetV2 for feature extraction to obtain image features of the enhanced image;

[0097] Inputting the image features into a path aggregation network for multi-scale feature fusion processing to obtain a feature fusion result;

[0098] Based on the feature fusion result, a preset detection head is used to perform detection to obtain a coordinate frame of the oil receiving pan;

[0099] Screenshot processing is performed based on the coordinate frame of the oil receiving pan to obtain an oil receiving pan image.

[0100] In this embodiment, ShuffleNetV2 is a lightweight deep convolutional neural network with low latency and high accuracy.

[0101] In this embodiment, the path aggregation network (PANet) is used to fuse the multi-scale features in the image to obtain the predicted target box. PANet uses a bottom-up expansion path method to accurately locate information and strengthen the entire feature structure, shorten the information path between the low-level network and the top-level network features, and use an adaptive feature pool to connect the feature grid and all feature layers, so that the useful information in each feature layer can be directly propagated to the subsequent candidate sub-network. The supplementary branch captures different aspects of each proposal and further improves the mask prediction. The network has only two pathways, top-down and bottom-up, which is simple and easy to implement.

[0102] In this embodiment, NanoDet removes all convolutions in PANet and only retains the 1x1 convolution after feature extraction from the backbone network to align the feature channel dimension. Both upsampling and downsampling are completed using interpolation.

[0103] In this embodiment, multi-scale feature maps are directly added in PANet, which greatly reduces the computational complexity of feature fusion.

[0104] In this embodiment, the detection head adopts the FCOS (Fully Convolutional One-Stage Object Detection) detection head. The FCOS series uses a shared weight detection head, that is, the same set of convolution prediction detection frames are used for the multi-scale feature maps output by PANet. Each layer uses a learnable Scale value as a coefficient to scale the predicted target frame, thereby accurately locating the target.

[0105] 103. Performing histogram equalization processing on the oil receiving pan image by using a contrast-limited adaptive histogram equalization algorithm to obtain an oil receiving pan equalized image;

[0106] In this embodiment, the contrast limited adaptive histogram equalization algorithm (CLAHE) calculates the local histogram of the image and redistributes the brightness to change the image contrast, limit the contrast amplitude, suppress the noise in the same area, achieve histogram equalization, enhance the image contrast and image edges, and retain important details.

[0107] Optionally, in one embodiment, the above step 103 includes:

[0108] Performing grayscale calculation on each pixel of the oil receiving pan image and pixels in a preset rectangular range around it to obtain a grayscale histogram within each rectangular range;

[0109] Clipping the grayscale histogram based on a preset threshold value, and evenly distributing the clipping value in the grayscale interval to obtain a balanced grayscale histogram;

[0110] Normalizing each of the balanced grayscale histograms, and calculating a cumulative distribution function of the normalized balanced grayscale histogram;

[0111] Mapping calculations are performed on each of the balanced grayscale histograms based on each of the cumulative distribution functions, and the calculation results are combined to obtain a balanced image of the oil receiving pan.

[0112] In this embodiment, each pixel is equalized by a grayscale histogram in a rectangular range around it, so as to redistribute the contrast of the image and limit the contrast amplitude.

[0113] In this embodiment, when the grayscale value is greater than a preset threshold, the portion exceeding the preset threshold is clipped, and the area of ​​the clipped portion is evenly distributed in the entire grayscale interval, and balancing is performed while ensuring that the total area of ​​the histogram remains unchanged.

[0114] In this embodiment, the cumulative distribution function is the corresponding brightness transformation function, and the mapping calculation is to multiply the original gray value by the cumulative distribution function to obtain the oil receiving pan image after contrast equalization, that is, the oil receiving pan equalized image.

[0115] 104. Convert the RGB color space of the oil receiving pan balanced image into the HSV color space, and locate the oil receiving pan area image of the oil receiving pan balanced image according to a preset color threshold;

[0116] In this embodiment, the RGB color space is composed of red (Red, R), green (Green, G) and blue (Blue, B), and the HSV color space is composed of hue (Hue, H), saturation (Saturation, S) and value (Value, V). HSV is closer to people's perception of color, and can intuitively express the hue, vividness and brightness of the color, which is convenient for color comparison.

[0117] In this embodiment, the preset color thresholds include the upper and lower limits of hue, the upper and lower limits of saturation, and the upper and lower limits of brightness. The area that simultaneously meets the upper and lower limits of hue, saturation, and brightness is regarded as the oil pan area.

[0118] Optionally, in one embodiment, the step 104 includes:

[0119] Converting the RGB color space of the oil pan balanced image into the HSV color space;

[0120] The converted oil pan balanced image is subjected to color extraction based on the preset color threshold to obtain an oil pan area image, wherein the portion of the oil pan area image that is not within the preset color threshold is represented by black.

[0121] In this embodiment, color extraction is to retain the colors within the preset color threshold range, and replace the parts that do not meet the preset color threshold condition with black.

[0122] 105. Convert the oil pan area image into an oil pan grayscale image, calculate the oil pan area variance based on the oil pan grayscale image, and determine whether the device to be tested is leaking oil according to the calculation result.

[0123] In this embodiment, the method for calculating the variance is not limited.

[0124] Optionally, in one embodiment, the step 105 includes:

[0125] Converting the oil pan area image into an oil pan grayscale image;

[0126] Calculating the grayscale average of pixels with non-zero grayscale values, and calculating the oil pan area variance based on the grayscale average;

[0127] If the oil receiving pan area variance is greater than a preset threshold, it is confirmed that the device under test is leaking oil; otherwise, it is confirmed that the device under test is not leaking oil.

[0128] In the embodiment of the present invention, an image of the device to be tested is obtained, and a multi-scale retinal enhancement algorithm with color restoration is used for enhancement processing to obtain an enhanced image; an oil pan is detected on the enhanced image and a screenshot is taken to obtain an oil pan image; a contrast-limited adaptive histogram equalization algorithm is used to perform histogram equalization on the oil pan image to obtain an oil pan balanced image; the RGB color space of the oil pan balanced image is converted into the HSV color space, and the oil pan area image of the oil pan balanced image is located according to a preset color threshold; the oil pan area image is converted into an oil pan grayscale image, and the oil pan area variance is calculated based on the oil pan grayscale image, and whether the device to be tested is leaking oil is determined according to the calculation result. The present invention realizes automatic detection of equipment oil leakage, reduces production costs, significantly improves the efficiency and accuracy of equipment oil leakage detection, and can accurately detect equipment oil leakage in low illumination environments.

[0129] See also Figure 2 Another embodiment of the method for detecting oil leakage of equipment in a dark light environment in the embodiment of the present invention includes:

[0130] 201. Acquire an image of a device under test, and enhance the image of the device under test by a multi-scale retinal enhancement algorithm with color restoration to obtain an enhanced image;

[0131] 202. Perform text detection on the enhanced image using a real-time scene text detection algorithm based on differential binarization to obtain a text region image;

[0132] In this embodiment, a real-time scene text detection algorithm based on differential binarization performs segmentation and binarization processing on an image, wherein the binarization processing is inserted into the segmentation processing for joint optimization, and after obtaining a binary image, the pixels are grouped and clustered to obtain a text box. This algorithm is simpler to operate, faster, and more accurate.

[0133] 203. Extract features from the text region image using a convolutional neural network to obtain a feature sequence, and input the feature sequence into a recurrent neural network for prediction to obtain a prediction sequence;

[0134] 204. Correct the predicted sequence using a CTC algorithm to obtain a text recognition result, and determine device information of the device under test based on the text recognition result;

[0135] In this embodiment, the CTC algorithm adjusts the parameters in the recurrent neural network through gradients for correction.

[0136] 205. Perform oil pan detection on the enhanced image, and perform screenshot processing according to the detection result to obtain an oil pan image;

[0137] 206. Performing histogram equalization processing on the oil receiving pan image by using a contrast-limited adaptive histogram equalization algorithm to obtain an oil receiving pan equalized image;

[0138] 207. Convert the RGB color space of the oil receiving pan balanced image into the HSV color space, and locate the oil receiving pan area image of the oil receiving pan balanced image according to a preset color threshold;

[0139] 208. Convert the oil pan area image into an oil pan grayscale image, calculate the oil pan area variance based on the oil pan grayscale image, and determine whether the device to be tested is leaking oil according to the calculation result.

[0140] In an embodiment of the present invention, an image of a device to be tested is acquired, and enhanced by a multi-scale retinal enhancement algorithm with color restoration to obtain an enhanced image; a real-time scene text detection algorithm based on differential binarization is used to perform text detection, and a CRNN+CTC architecture model is used to perform text recognition to obtain device information; an oil pan is detected on the enhanced image and a screenshot is taken to obtain an oil pan image; a histogram equalization is performed on the oil pan image by a contrast-limited adaptive histogram equalization algorithm to obtain an oil pan balanced image; the RGB color space of the oil pan balanced image is converted into an HSV color space, and the oil pan area image of the oil pan balanced image is located according to a preset color threshold; the oil pan area image is converted into an oil pan grayscale image, and the oil pan area variance is calculated based on the oil pan grayscale image, and whether the device to be tested is leaking oil is determined according to the calculation result. The present invention realizes automatic detection of equipment oil leakage, reduces production costs, significantly improves the efficiency and accuracy of equipment oil leakage detection, and can accurately detect equipment oil leakage in low-light environments. The real-time scene text detection algorithm based on differential binarization is suitable for horizontal, inclined, and distorted text detection scenes, and can realize real-time online detection. The CRNN+CTC architecture is easy to operate, fast, and has high accuracy.

[0141] The above describes the oil leakage detection method for equipment under dark light environment in the embodiment of the present invention. The following describes the oil leakage detection device for equipment under dark light environment in the embodiment of the present invention. Figure 3 In one embodiment of the present invention, an oil leakage detection device for equipment in a dark light environment includes:

[0142] The enhancement module 301 is used to acquire an image of the device under test, and enhance the image of the device under test by a multi-scale retinal enhancement algorithm with color restoration to obtain an enhanced image;

[0143] A detection module 302 is used to perform oil pan detection on the enhanced image and perform screenshot processing according to the detection result to obtain an oil pan image;

[0144] An equalization module 303, configured to perform a histogram equalization process on the oil receiving pan image by using a contrast-limited adaptive histogram equalization algorithm to obtain an oil receiving pan equalized image;

[0145] A positioning module 304, configured to convert the RGB color space of the oil receiving pan balanced image into the HSV color space, and to locate the oil receiving pan region image of the oil receiving pan balanced image according to a preset color threshold;

[0146] The calculation module 305 is used to convert the oil pan area image into an oil pan grayscale image, calculate the oil pan area variance based on the oil pan grayscale image, and determine whether the device under test is leaking oil according to the calculation result.

[0147] Optionally, the enhancement module 301 may also be specifically used for:

[0148] Acquire an image of the device under test, and input the image of the device under test into a multi-scale retinal enhancement algorithm model with color restoration;

[0149] The multi-scale retinal enhancement algorithm model with color restoration performs enhancement processing on the image of the device under test based on the Retinex algorithm to obtain an enhanced image;

[0150] The mathematical expression of the Retinex algorithm is as follows:

[0151] I=L*R

[0152] Among them, I represents the image signal observed or received by the camera; L represents the illumination component of the ambient light; and R represents the reflection component of the target object carrying the image detail information.

[0153] Optionally, the detection module 302 may also be specifically configured to:

[0154] Input the enhanced image into the backbone network ShuffleNetV2 for feature extraction to obtain image features of the enhanced image;

[0155] Inputting the image features into a path aggregation network for multi-scale feature fusion processing to obtain a feature fusion result;

[0156] Based on the feature fusion result, a preset detection head is used to perform detection to obtain a coordinate frame of the oil receiving pan;

[0157] Screenshot processing is performed based on the coordinate frame of the oil receiving pan to obtain an oil receiving pan image.

[0158] Optionally, the balancing module 303 may also be specifically used for:

[0159] Performing grayscale calculation on each pixel of the oil receiving pan image and pixels in a preset rectangular range around it to obtain a grayscale histogram within each rectangular range;

[0160] Clipping the grayscale histogram based on a preset threshold value, and evenly distributing the clipping value in the grayscale interval to obtain a balanced grayscale histogram;

[0161] Normalizing each of the balanced grayscale histograms, and calculating a cumulative distribution function of the normalized balanced grayscale histogram;

[0162] Mapping calculations are performed on each of the balanced grayscale histograms based on each of the cumulative distribution functions, and the calculation results are combined to obtain a balanced image of the oil receiving pan.

[0163] Optionally, the positioning module 304 may also be specifically used for:

[0164] Converting the RGB color space of the oil pan balanced image into the HSV color space;

[0165] The converted oil pan balanced image is subjected to color extraction based on the preset color threshold to obtain an oil pan area image, wherein the portion of the oil pan area image that is not within the preset color threshold is represented by black.

[0166] Optionally, the calculation module 305 may also be specifically used for:

[0167] Converting the oil pan area image into an oil pan grayscale image;

[0168] Calculating the grayscale average of pixels with non-zero grayscale values, and calculating the oil pan area variance based on the grayscale average;

[0169] If the oil receiving pan area variance is greater than a preset threshold, it is confirmed that the device under test is leaking oil; otherwise, it is confirmed that the device under test is not leaking oil.

[0170] In the embodiment of the present invention, an image of the device to be tested is obtained, and a multi-scale retinal enhancement algorithm with color restoration is used for enhancement processing to obtain an enhanced image; an oil pan is detected on the enhanced image and a screenshot is taken to obtain an oil pan image; a contrast-limited adaptive histogram equalization algorithm is used to perform histogram equalization on the oil pan image to obtain an oil pan balanced image; the RGB color space of the oil pan balanced image is converted into the HSV color space, and the oil pan area image of the oil pan balanced image is located according to a preset color threshold; the oil pan area image is converted into an oil pan grayscale image, and the oil pan area variance is calculated based on the oil pan grayscale image, and whether the device to be tested is leaking oil is determined according to the calculation result. The present invention realizes automatic detection of equipment oil leakage, reduces production costs, significantly improves the efficiency and accuracy of equipment oil leakage detection, and can accurately detect equipment oil leakage in low illumination environments.

[0171] See also Figure 4 Another embodiment of the oil leakage detection device for equipment in a dark light environment in the embodiment of the present invention includes:

[0172] The enhancement module 301 is used to acquire an image of the device under test, and enhance the image of the device under test by a multi-scale retinal enhancement algorithm with color restoration to obtain an enhanced image;

[0173] A detection module 302 is used to perform oil pan detection on the enhanced image and perform screenshot processing according to the detection result to obtain an oil pan image;

[0174] An equalization module 303, configured to perform a histogram equalization process on the oil receiving pan image by using a contrast-limited adaptive histogram equalization algorithm to obtain an oil receiving pan equalized image;

[0175] A positioning module 304, configured to convert the RGB color space of the oil receiving pan balanced image into the HSV color space, and to locate the oil receiving pan region image of the oil receiving pan balanced image according to a preset color threshold;

[0176] A calculation module 305 is used to convert the oil pan area image into an oil pan grayscale image, calculate the oil pan area variance based on the oil pan grayscale image, and determine whether the device under test is leaking oil according to the calculation result;

[0177] The identification module 306 is used for text detection, character recognition and obtaining device information of the device under test.

[0178] Optionally, the identification module 306 may also be specifically configured to:

[0179] Using a real-time scene text detection algorithm based on differential binarization to perform text detection on the enhanced image to obtain a text area image;

[0180] Using a convolutional neural network to extract features from the text region image to obtain a feature sequence, and inputting the feature sequence into a recurrent neural network for prediction to obtain a prediction sequence;

[0181] The predicted sequence is corrected by using a CTC algorithm to obtain a text recognition result, and the device information of the device under test is determined based on the text recognition result.

[0182] In an embodiment of the present invention, an image of a device to be tested is acquired, and enhanced by a multi-scale retinal enhancement algorithm with color restoration to obtain an enhanced image; a real-time scene text detection algorithm based on differential binarization is used to perform text detection, and a CRNN+CTC architecture model is used to perform text recognition to obtain device information; an oil pan is detected on the enhanced image and a screenshot is taken to obtain an oil pan image; a histogram equalization is performed on the oil pan image by a contrast-limited adaptive histogram equalization algorithm to obtain an oil pan balanced image; the RGB color space of the oil pan balanced image is converted into an HSV color space, and the oil pan area image of the oil pan balanced image is located according to a preset color threshold; the oil pan area image is converted into an oil pan grayscale image, and the oil pan area variance is calculated based on the oil pan grayscale image, and whether the device to be tested is leaking oil is determined according to the calculation result. The present invention realizes automatic detection of equipment oil leakage, reduces production costs, significantly improves the efficiency and accuracy of equipment oil leakage detection, and can accurately detect equipment oil leakage in low-light environments. The real-time scene text detection algorithm based on differential binarization is suitable for horizontal, inclined, and distorted text detection scenes, and can realize real-time online detection. The CRNN+CTC architecture is easy to operate, fast, and has high accuracy.

[0183] above Figure 3 and Figure 4 The oil leakage detection device for equipment under a dark light environment in an embodiment of the present invention is described in detail from the perspective of modular functional entities. The electronic device in an embodiment of the present invention is described in detail from the perspective of hardware processing.

[0184] Figure 5is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 510 (for example, one or more processors) and a memory 520, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 533 or data 532. Among them, the memory 520 and the storage medium 530 can be short-term storage or permanent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the electronic device 500. Furthermore, the processor 510 can be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the electronic device 500.

[0185] The electronic device 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 5 The structure of the electronic device shown does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0186] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the equipment oil leakage detection method in a dark light environment in the above-mentioned embodiments.

[0187] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the method for detecting oil leakage of equipment in a dark light environment.

[0188] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0189] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0190] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting oil leakage of equipment in a dark light environment, characterized in that: The equipment oil leakage detection method comprises: Acquire an image of the device under test, and enhance the image of the device under test by a multi-scale retinal enhancement algorithm with color restoration to obtain an enhanced image; Input the enhanced image into the backbone network ShuffleNetV2 for feature extraction to obtain image features of the enhanced image; Inputting the image features into a path aggregation network for multi-scale feature fusion processing to obtain a feature fusion result; Based on the feature fusion result, a preset detection head is used to perform detection to obtain a coordinate frame of the oil receiving pan; Performing screenshot processing based on the coordinate frame of the oil receiving pan to obtain an image of the oil receiving pan; Performing histogram equalization processing on the oil pan image by using a contrast-limited adaptive histogram equalization algorithm to obtain an oil pan equalized image; Converting the RGB color space of the oil pan balanced image into the HSV color space, and locating the oil pan area image of the oil pan balanced image according to a preset color threshold; The oil pan area image is converted into an oil pan grayscale image, and the oil pan area variance is calculated based on the oil pan grayscale image, and whether the device to be tested is leaking oil is determined according to the calculation result.

2. The method for detecting oil leakage of equipment in a dark light environment according to claim 1, characterized in that: The step of acquiring an image of the device under test and performing enhancement processing on the image of the device under test by a multi-scale retinal enhancement algorithm with color restoration to obtain an enhanced image comprises: Acquire an image of the device under test, and input the image of the device under test into a multi-scale retinal enhancement algorithm model with color restoration; The multi-scale retinal enhancement algorithm model with color restoration performs enhancement processing on the image of the device under test based on the Retinex algorithm to obtain an enhanced image; The mathematical expression of the Retinex algorithm is as follows: I=L*R Among them, I represents the image signal observed or received by the camera; L represents the illumination component of the ambient light; and R represents the reflection component of the target object carrying the image detail information.

3. The method for detecting oil leakage of equipment in a dark light environment according to claim 1, characterized in that: The step of performing histogram equalization processing on the oil pan image by using a contrast-limited adaptive histogram equalization algorithm to obtain an oil pan equalized image includes: Performing grayscale calculation on each pixel of the oil receiving pan image and pixels in a preset rectangular range around it to obtain a grayscale histogram within each rectangular range; Clipping the grayscale histogram based on a preset threshold value, and evenly distributing the clipping value in the grayscale interval to obtain a balanced grayscale histogram; Normalizing each of the balanced grayscale histograms, and calculating a cumulative distribution function of the normalized balanced grayscale histogram; Mapping calculations are performed on each of the balanced grayscale histograms based on each of the cumulative distribution functions, and the calculation results are combined to obtain a balanced image of the oil receiving pan.

4. The method for detecting oil leakage of equipment in a dark light environment according to claim 1, characterized in that: The converting the RGB color space of the oil pan balanced image into the HSV color space and locating the oil pan area image of the oil pan balanced image according to a preset color threshold comprises: Converting the RGB color space of the oil pan balanced image into the HSV color space; The converted oil pan balanced image is subjected to color extraction based on the preset color threshold to obtain an oil pan area image, wherein the portion of the oil pan area image that is not within the preset color threshold is represented by black.

5. The method for detecting oil leakage of equipment in a dark light environment according to claim 1, characterized in that: The converting the oil pan area image into an oil pan grayscale image, calculating the oil pan area variance based on the oil pan grayscale image, and judging whether the device to be tested is leaking oil according to the calculation result comprises: Converting the oil pan area image into an oil pan grayscale image; Calculating the grayscale average of pixels with non-zero grayscale values, and calculating the oil pan area variance based on the grayscale average; If the oil receiving pan area variance is greater than a preset threshold, it is confirmed that the device under test is leaking oil; otherwise, it is confirmed that the device under test is not leaking oil.

6. The method for detecting oil leakage of equipment in a dark light environment according to claim 1, characterized in that: After acquiring the image of the device under test and performing enhancement processing on the image of the device under test by a multi-scale retinal enhancement algorithm with color restoration to obtain an enhanced image, the method further includes: Using a real-time scene text detection algorithm based on differential binarization to perform text detection on the enhanced image to obtain a text area image; Using a convolutional neural network to extract features from the text region image to obtain a feature sequence, and inputting the feature sequence into a recurrent neural network for prediction to obtain a prediction sequence; The predicted sequence is corrected by using a CTC algorithm to obtain a text recognition result, and the device information of the device under test is determined based on the text recognition result.

7. An oil leakage detection device for equipment in a dark light environment, characterized in that: The equipment oil leakage detection device in the dark light environment comprises: An enhancement module is used to acquire an image of the device under test, and enhance the image of the device under test by a multi-scale retinal enhancement algorithm with color restoration to obtain an enhanced image; The detection module is used to input the enhanced image into the backbone network ShuffleNetV2 for feature extraction to obtain image features of the enhanced image; input the image features into the path aggregation network for multi-scale feature fusion processing to obtain feature fusion results; based on the feature fusion results, use the preset detection head to perform detection to obtain the coordinate frame of the oil receiving pan; perform screenshot processing based on the coordinate frame of the oil receiving pan to obtain an image of the oil receiving pan; an equalization module, configured to perform histogram equalization processing on the oil pan image by using a contrast-limited adaptive histogram equalization algorithm to obtain an oil pan equalized image; a positioning module, used for converting the RGB color space of the oil receiving pan balanced image into the HSV color space, and locating the oil receiving pan area image of the oil receiving pan balanced image according to a preset color threshold; The calculation module is used to convert the oil pan area image into an oil pan grayscale image, calculate the oil pan area variance based on the oil pan grayscale image, and determine whether the device to be tested is leaking oil according to the calculation result.

8. An electronic device, characterized in that: The electronic device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the device for detecting oil leakage of equipment in a dark-light environment to execute the method for detecting oil leakage of equipment in a dark-light environment according to any one of claims 1 to 6.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the method for detecting oil leakage of equipment in a dark light environment according to any one of claims 1 to 6 is implemented.

Images