Image recognition-based oil dripping monitoring method and device, and imaging equipment

Abstract

The application provides an oil dripping monitoring method based on image recognition, comprising the following steps: acquiring thermal imaging image data and visible light image data; dividing the thermal imaging image data into corresponding thermal imaging image data according to a plurality of preset monitoring areas; identifying the thermal imaging image data meeting a preset condition in each thermal imaging image data as to-be-labeled image data, so as to identify the preset monitoring area corresponding to the to-be-labeled image data as a to-be-labeled monitoring area; generating a thermal imaging image according to the thermal imaging image data and the to-be-labeled monitoring area; feeding the visible light image data into a preset training model to generate a visible light image; and identifying whether the oil pipeline exists oil dripping according to the thermal imaging image and the visible light image. In addition, the application also provides an oil dripping monitoring device based on image recognition and an imaging equipment. The application combines the temperature distribution of thermal imaging and the specific image of visible light, and realizes accurate analysis of the existence and distribution of oil dripping of the oil pipeline.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pipeline detection, and in particular to an oil dripping monitoring method and device based on image recognition and imaging equipment. BACKGROUND

[0002] With the development of the oil industry, pipelines for oil transportation have also developed rapidly. At the same time, the monitoring method for pipeline leakage has also developed to a certain extent. In the prior art, the monitoring method based on infrared light or the monitoring method based on visible light is usually used. However, the monitoring method based on infrared light is difficult to correspond to the specific detection position of the pipeline when detecting the pipeline. At the same time, the monitoring method based on visible light is too dependent on visible light and is sensitive to the change of the sun's reflection and the angle of view. When the ambient light is very strong or relatively dim, the video image will be too bright or too dark, and it is difficult to distinguish the image content. SUMMARY

[0003] The present application provides an oil dripping monitoring method and device based on image recognition and imaging equipment to intuitively and accurately analyze whether the oil pipeline exists oil dripping.

[0004] In a first aspect, the present application provides an oil dripping monitoring method based on image recognition, which comprises: acquiring thermal imaging image data and visible light image data, wherein the thermal imaging image data and the visible light image data are image data of an oil pipeline in a monitoring area and a ground area corresponding to the oil pipeline; dividing the thermal imaging image data into a plurality of corresponding thermal imaging image data according to a plurality of preset monitoring areas; identifying thermal imaging image data meeting a preset condition in each thermal imaging image data as to-be-labeled image data, so as to identify a preset monitoring area corresponding to the to-be-labeled image data as a to-be-labeled monitoring area; generating a thermal imaging image according to the thermal imaging image data and the to-be-labeled monitoring area, wherein the thermal imaging image comprises a target monitoring area corresponding to the to-be-labeled monitoring area; feeding the visible light image data into a preset training model to generate a visible light image; and identifying whether the oil pipeline exists oil dripping according to the thermal imaging image and the visible light image.

[0005] In a second aspect, the embodiments of the present application provide an oil dripping monitoring device based on image recognition, which comprises a data acquisition module, a first data processing module, a second data processing module, and an image data analysis module. The data acquisition module is configured to acquire thermal imaging image data and visible light image data, which are image data of oil pipeline and corresponding ground area in a monitoring area. The first data processing module is communicatively connected to the data acquisition module and comprises a data division module, a data analysis module, and a first image generation module. The data division module is configured to divide the thermal imaging image data into corresponding thermal imaging image data according to a plurality of preset monitoring areas. The data analysis module is configured to identify thermal imaging image data meeting a preset condition in each thermal imaging image data as to-be-labeled image data, so as to identify a preset monitoring area corresponding to the to-be-labeled image data as a to-be-labeled monitoring area. The first image generation module is configured to generate a thermal imaging image according to the thermal imaging image data and the to-be-labeled monitoring area, wherein the thermal imaging image comprises a target monitoring area corresponding to the to-be-labeled monitoring area. The second data processing module is communicatively connected to the data acquisition module and comprises a second image generation module. The second image generation module is configured to feed the visible light image data into a preset training model to generate a visible light image. The image data analysis module is communicatively connected to the first data processing module and the second data processing module, and is configured to identify whether the oil pipeline has oil dripping according to the thermal imaging image and the visible light image.

[0006] In a third aspect, the embodiments of the present application provide an imaging device, which comprises a thermal imaging image data module, a visible light image data module, and the oil dripping monitoring device based on image recognition. The thermal imaging image data module is configured to acquire thermal imaging image data. The visible light image data module is configured to acquire visible light image data.

[0007] The oil dripping monitoring method, device, and imaging device based on image recognition described above can acquire thermal imaging image data and visible light image data of oil pipeline and corresponding ground area, identify thermal imaging image data meeting a preset condition, identify a to-be-labeled monitoring area and generate a thermal imaging image, feed visible light image data into a preset training model to generate a visible light image, and identify whether the oil pipeline has oil dripping according to the thermal imaging image and the visible light image. The temperature distribution of thermal imaging and the specific image of visible light are combined to accurately analyze the existence and distribution of oil dripping of the oil pipeline. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only represent some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from the structures shown in these drawings without creative labor.

[0009] Figure 1 The first flow chart of the oil dripping monitoring method based on image recognition provided by the embodiments of the present application.

[0010] Figure 2 The second flow chart of the oil dripping monitoring method based on image recognition provided by the embodiments of the present application.

[0011] Figure 3 The third flow chart of the oil dripping monitoring method based on image recognition provided by the embodiments of the present application.

[0012] Figure 4 The flow chart of the sub-steps of step S105 provided by the embodiments of the present application.

[0013] Figure 5 The flow chart of the sub-steps of step S106 provided by the embodiments of the present application.

[0014] Figure 6 The fourth flow chart of the oil dripping monitoring method based on image recognition provided by the embodiments of the present application.

[0015] Figure 7 The structural schematic diagram of the oil dripping monitoring device based on image recognition provided by the embodiments of the present application.

[0016] Figure 8 The structural schematic diagram of the imaging device provided by the embodiments of the present application.

[0017] Figure 9 The image schematic diagram of the oil dripping of the pipeline provided by the embodiments of the present application.

[0018] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the drawings. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present application more clear, the following will further describe the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor also belong to the scope of protection of the present application.

[0020] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application and above-mentioned drawings, if any, are used for distinguishing between similar objects, and do not necessarily have a particular chronological, sequential or hierarchical order. It is to be understood that the data used in the description and in the claims of the present application are interchangeable, unless otherwise explicitly stated. In other words, the embodiments described according to any one of the examples can be implemented in any order, unless otherwise explicitly stated. Furthermore, the terms "comprise" and "have" and any variations thereof are used synonymously with "include" and "detect" and are intended to be open-ended, allowing for the possibility that other, non-enumerated, elements can also be present. Processes, methods, systems, products, or apparatuses including a series of steps or units are not necessarily limited to only those steps or units that are explicitly listed, unless expressly stated otherwise.

[0021] It should be noted that the terms "first", "second", "third", "fourth" and the like in the description are merely used for description purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined as "first", "second" can explicitly or implicitly include one or more of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the realization of a person skilled in the art, when the combination of technical solutions contradicts each other or cannot be realized, it should be considered that the combination of technical solutions does not exist, nor within the protection scope required by the present application.

[0022] Please refer to Figure 7 and Figure 8 , Figure 7 the structure diagram of the oil dripping monitoring device based on image recognition provided by the embodiments of the present application, Figure 8 the structure diagram of the imaging device provided by the embodiments of the present application.

[0023] As Figure 8As shown, the embodiment of the present application provides an imaging device 100. The imaging device 100 is used to monitor whether there is oil leakage under the oil pipeline covered by the oil pipeline, that is, whether there is oil leakage under the ground area corresponding to the oil pipeline. Specifically, the imaging device 100 includes an oil leakage monitoring device 11 based on image recognition, a thermal imaging image data module 12, and a visible light image data module 13. The thermal imaging image data module 12 is used to obtain thermal imaging image data. The thermal imaging image data is used to reflect the heat distribution, temperature difference, and current temperature of each position of the object corresponding to the collected data. The thermal imaging image data can be obtained by a device with thermal imaging function, such as an infrared detector, an infrared thermal imaging monitoring device, etc. The visible light image data module 13 is used to obtain visible light image data. The visible light image data is the data corresponding to the image in the visible light range, which is used to reflect the current image of the object corresponding to the collected data under visible light conditions. The visible light image data can be obtained by a device with visible light imaging function, such as a digital camera, a mobile phone, a component with shooting function, etc.

[0024] As shown, Figure 7 The oil leakage monitoring device 11 based on image recognition includes a data acquisition module 110, a first data processing module 111, a second data processing module 112, an image data analysis module 113, and a communication module 114.

[0025] The data acquisition module 110 is used to obtain thermal imaging image data and visible light image data, which are image data of the oil pipeline and the ground area corresponding to the oil pipeline in the monitoring area.

[0026] The first data processing module 111 is in communication connection with the data acquisition module 110. The first data processing module 111 includes a data division module 1110, a data analysis module 1111, a first image generation module 1112, and a data labeling module 1113.

[0027] The data division module 1110 is used to divide the thermal imaging image data into corresponding thermal imaging image data according to a plurality of preset monitoring areas.

[0028] The data analysis module 1111 is used to identify the thermal imaging image data meeting the preset condition in each thermal imaging image data as the to-be-labeled image data, so as to identify the preset monitoring area corresponding to the to-be-labeled image data as the to-be-labeled monitoring area.

[0029] The first image generation module 1112 is used to generate a thermal imaging image according to the thermal imaging image data and the to-be-labeled monitoring area. The thermal imaging image includes a target monitoring area, and the target monitoring area corresponds to the to-be-labeled monitoring area.

[0030] The data labeling module 1113 is configured to label the maximum temperature value and the minimum temperature value in the temperature values corresponding to the image data to be labeled to the target monitoring area in the thermographic image after the thermographic image is generated.

[0031] The second data processing module 112 is in communication connection with the data acquisition module 110. The second data processing module 112 includes a second image generation module 1120 and an image data labeling module 1121.

[0032] The second image generation module 1120 is configured to feed the visible light image data into the preset training model to generate the visible light image.

[0033] The image data labeling module 1121 is configured to perform the following steps before feeding the visible light image data into the preset training model to generate the visible light image: judging whether the visible light image data contains crack image data; and labeling the crack image data and monitoring the oil pipeline and the ground area in the monitoring area corresponding to the crack image data when the visible light image data contains the crack image data.

[0034] The image data analysis module 113 is in communication connection with the first data processing module 111 and the second data processing module 112, respectively, and is configured to identify whether the oil pipeline exists oil dripping according to the thermographic image and the visible light image.

[0035] The communication module 114 is in communication connection with the image data analysis module 113, and the communication module 114 is in communication connection with the monitoring device 200. The communication module 114 is configured to send the thermographic image and the visible light image to the monitoring device 200 to perform the following steps: generating a prompt information when it is identified that the oil pipeline corresponding to the target monitoring area exists oil dripping, the prompt information being used to indicate the corresponding position of the oil pipeline existing oil dripping and the target monitoring area corresponding to the oil pipeline; and displaying the thermographic image, the visible light image and the prompt information.

[0036] In the embodiment, the recognition result obtained by the image recognition-based oil dripping monitoring device 11 in the imaging device 100 is transmitted to the monitoring device 200 through the communication module 114. The monitoring device 200 controls the image recognition-based oil dripping monitoring device 11 to perform the image recognition-based oil dripping monitoring method through the communication module 114. The monitoring device 200 can be a device assembly with monitoring function, such as a monitor in a monitoring room, or an alarm device assembly in a monitoring room. The steps of the image recognition-based oil dripping monitoring method will be described below to illustrate how the image recognition-based oil dripping monitoring device 11 identifies the pipeline oil dripping.

[0037] Please refer to Figure 1Fig. 1 is a first flowchart of an oil dripping monitoring method based on image recognition provided by an embodiment of the present application. The oil dripping monitoring method based on image recognition comprises steps S101-S106.

[0038] In step S101, thermal imaging image data and visible light image data are acquired.

[0039] In step S101, the thermal imaging image data and the visible light image data are image data of oil pipeline in a monitoring area and a ground area corresponding to the oil pipeline. The oil pipeline is used for transporting fluid such as oil and gasoline, including but not limited to internal pipeline of a supply device for supplying fluid, transportation pipeline connecting the supply device and a storage device, etc. The ground area corresponding to the oil pipeline is an area covered by the oil pipeline. The thermal imaging image data is a temperature value of a corresponding position in the ground area. The visible light image data can be in the form of image data or video data. Specifically, when a video in visible light format is acquired, the visible light image data can also be acquired by acquiring image data of a specified frame in the video frame data. That is, the visible light image data is one or more of image data and video frame data.

[0040] It can be understood that when the imaging device 100 is arranged directly above the oil pipeline, the ground area can be a ground surface directly below the oil pipeline. The temperature of the ground area corresponding to different positions of the pipeline in the pipeline is also different when the pipeline is working. For example, in the process of transporting oil from a supply device to a storage device, the oil in the pipeline needs to be heated multiple times to ensure smooth transportation of the oil. At this time, the temperature of the ground area corresponding to the pipeline in which oil dripping occurs during the transportation process is different from that of the normal pipeline, and the temperature difference of the ground area directly reflects the difference in heat corresponding to each ground area. Therefore, the temperature of each ground area can be obtained through the thermal imaging image data.

[0041] In some possible embodiments, the thermal imaging image data can form a temperature distribution image corresponding to each ground area. The temperature distribution image corresponds to the temperature value of a corresponding position in the ground area, and is used to acquire information such as heat distribution, temperature difference and current temperature of each corresponding position in the ground area. After the temperature distribution image is acquired, image processing needs to be performed on the temperature distribution image to improve the accuracy of subsequent extraction of data features of thermal imaging detection, thereby improving the accuracy of oil dripping detection of the ground area. The image processing of the temperature distribution image includes but is not limited to image denoising, image enhancement and other image processing operations.

[0042] In step S102, the thermal imaging image data is divided into a plurality of corresponding thermal imaging image data according to a plurality of preset monitoring areas.

[0043] In step S102, the preset monitoring area can be divided according to the position, quantity and other factors of the thermal imaging image data module 12 and the visible light image data module 13 in the imaging device 100. Each thermal imaging image data contains a plurality of temperature values.

[0044] In step S103, the thermal imaging image data meeting the preset condition in each thermal imaging image data is identified as the to-be-labeled image data, so as to identify the preset monitoring area corresponding to the to-be-labeled image data as the to-be-labeled monitoring area.

[0045] In step S103, after obtaining a plurality of corresponding thermal imaging image data, temperature data features for representing the oil dripping phenomenon need to be extracted from the plurality of corresponding thermal imaging image data, so as to judge the thermal imaging image data meeting the oil dripping feature in each corresponding thermal imaging image data, so as to obtain the ground area corresponding to the to-be-labeled monitoring area to identify the oil pipeline corresponding to the potential oil dripping area. The temperature data features include but are not limited to temperature gradient, temperature change rate, temperature peak and the like. These temperature data features will be used to judge whether there is a sign of oil dripping. For example, when the pipeline has a leakage problem, the surface temperature of the ground area under the pipeline leakage changes to have a thermal difference with the surrounding environment. At this time, the temperature gradient, temperature change rate, temperature peak and the like in the thermal imaging image data will also change to a certain extent. When the above changes are sufficient, the preset condition can be obtained according to the above changes. Preferably, the preset condition is that the temperature value is greater than a preset temperature value.

[0046] In step S104, a thermal imaging image is generated according to the thermal imaging image data and the to-be-labeled monitoring area.

[0047] In step S104, the thermal imaging image includes a target monitoring area, and the target monitoring area corresponds to the to-be-labeled monitoring area.

[0048] In step S105, the visible light image data is fed into a preset training model to generate a visible light image.

[0049] In step S105, the preset training model is obtained by feeding an initial training model with a first preset image data set containing oil dripping image data and a second preset image data set not containing oil dripping image data. The training process of the initial training model and the preset training model will be described below.

[0050] Firstly, image data of a plurality of visible light images is acquired as a preset image data set, a first preset image data set containing oil droplet leakage image data and a second preset image data set not containing oil droplet leakage image data are labeled, and position information of oil droplet leakage is labeled in the first preset image data set. After the preset image data set is acquired, each image in the preset image data set is pre-processed to improve the quality of each image and reduce interference factors, so that the initial training model can better learn and judge. The pre-image processing includes but is not limited to image enhancement, size adjustment, color correction and other image processing. After the image-processed preset image data set is acquired, image data features containing oil droplet leakage image data and not containing oil droplet leakage image data in the preset image data set are extracted. That is, the oil droplet leakage feature is the image data feature of the oil droplet leakage image data. The image data feature can be extracted by computer vision methods such as edge detection and texture feature extraction, or deep learning techniques such as convolutional neural networks are used to automatically extract image data features, thereby completing the training of the initial training model to obtain a preset training model.

[0051] It can be understood that the preset training model also needs to be continuously trained to optimize the parameters of the preset training model, so that the preset training model can accurately judge whether there is oil droplet leakage. Specifically, the trained preset training model is evaluated using a validation set, that is, the accuracy, recall rate, F1 value and other indicators are calculated to evaluate the performance of the preset training model. According to the evaluation result, the preset training model is optimized, such as adjusting the parameters of the preset training model, increasing the amount of training data, using data enhancement technology, etc., to improve the generalization ability and stability of the preset training model.

[0052] Please refer to Figure 4 which is a flowchart of the sub-step of step S105 provided by the embodiment of the application. The visible light image data is fed into the preset training model to generate a visible light image, which includes steps S1051-S1052.

[0053] Step S1051, when the visible light image data is image data, the image data is fed into the preset training model to generate a visible light image.

[0054] Step S1052, when the visible light image data is video frame data, a plurality of continuous video frame data is acquired to obtain continuous image data, and the continuous image data is fed into the preset training model to generate a visible light image.

[0055] In step S1052, the video frame data is used to extract the ground area in the moving or changed ground area containing the oil droplet leakage trace. The data corresponding to the moving or changed ground area is continuous, and is essentially a plurality of continuous images. Therefore, when the visible light image data is video frame data, it can also be determined according to the preset training model whether the continuous video frame data contains the preset image data feature, so as to generate the corresponding visible light image.

[0056] In step S106, whether the oil liquid pipeline exists oil droplet leakage is identified according to the thermal imaging image and the visible light image.

[0057] Please refer to Figure 5 which is a flowchart of the sub-step of step S106 provided by the embodiment of the present application. Identifying whether the oil liquid pipeline exists oil droplet leakage according to the thermal imaging image and the visible light image includes steps S1061-S1063.

[0058] In step S1061, the ground area at the corresponding position of the visible light image is compared with the target monitoring area.

[0059] In step S1061, since the data in the thermal imaging image and the visible light image are all obtained based on the same ground area, each ground area in the thermal imaging image can obtain the corresponding ground area in the visible light image.

[0060] In step S1062, when the ground area at the corresponding position of the visible light image exists oil droplet leakage, it is identified that the oil liquid pipeline corresponding to the target monitoring area exists oil droplet leakage.

[0061] In step S1062, when the ground area at the corresponding position of the visible light image exists oil droplet leakage, the temperature of the to-be-labeled monitoring area is abnormal in the thermal imaging image, and the image data feature corresponding to the to-be-labeled monitoring area is similar to the oil droplet leakage data feature. Therefore, the to-be-labeled monitoring area can be considered as oil droplet leakage.

[0062] In step S1063, when the ground area at the corresponding position of the visible light image does not exist oil droplet leakage, the target monitoring area is monitored.

[0063] In step S1063, when the ground area at the corresponding position of the visible light image does not have oil dripping, i.e., a single monitoring mode abnormality occurs, the cause of the single monitoring mode abnormality needs to be considered. For the thermal imaging image, on the basis that the pipeline has oil dripping, the cause of the occurrence of the preset condition can also be an external factor such as a sudden increase in the conveying speed of the fluid in a short time or a sharp change in the ambient temperature. For the visible light image, on the basis that the visible light image contains oil dripping image data, the cause of the occurrence of the oil dripping image data can also be an external factor such as light or angle of the environment where the pipeline is located. Therefore, for the case of a single monitoring mode abnormality, the staff should perform real-time monitoring on the to-be-labeled monitoring area, i.e., identify whether each to-be-labeled monitoring area is indeed an oil dripping area, so as to achieve the effect of accurately analyzing the oil dripping area of the pipeline.

[0064] Please refer to Figure 9 , which is an image schematic diagram for identifying pipeline oil dripping provided by the embodiment of the present application.

[0065] As Figure 9 indicated, the staff directly compares the target monitoring area of the thermal imaging image 1 and the ground area at the corresponding position of the visible light image 2 through the acquired thermal imaging image 1 and visible light image 2. Specifically, the target monitoring area is determined by observing whether the temperature of the to-be-labeled monitoring area in the thermal imaging image 1 is abnormal, and whether the oil pipeline corresponding to the target monitoring area has oil dripping is determined by whether the corresponding ground area in the visible light image 2 has oil dripping. Among them, the maximum temperature value Max, the minimum temperature value Min of the currently selected to-be-labeled monitoring area, and the preset temperature value Avg of the currently selected target monitoring area have been displayed in the thermal imaging image 1. When the currently selected to-be-labeled monitoring area has a temperature value higher than the preset temperature value Avg, the temperature value higher than the preset temperature value Avg is marked and displayed in the thermal imaging image 1 as all target monitoring areas, so as to be compared with the ground area at the corresponding position of the visible light image. When the ground area at the corresponding position of the visible light image 2 does not have oil dripping, the staff directly goes to the position of the oil pipeline according to the target monitoring area of the thermal imaging image 1 to determine whether the oil pipeline corresponding to the target monitoring area indeed has oil dripping, so as to achieve the effect of accurately analyzing the oil dripping area of the pipeline.

[0066] Please refer to Figure 2 , which is a second flowchart of the oil dripping monitoring method based on image recognition provided by the embodiment of the present application. After generating the thermal imaging image, the oil dripping monitoring method based on image recognition further includes step S201.

[0067] Step S201, label the maximum temperature value and the minimum temperature value in the several temperature values corresponding to the image data to be labeled to the target monitoring area in the thermal imaging image.

[0068] In step S201, the temperature difference of the ground area corresponding to the maximum temperature value and the minimum temperature value and the temperature of the ground area in the current environment is large, that is, the position of the ground area is the position of the ground area with the highest probability of oil droplet leakage hidden danger. The labeling methods include but are not limited to highlighting labeling, color labeling and other labeling methods. How to generate a thermal imaging image will be described in detail below.

[0069] After obtaining each corresponding thermal imaging image data, first, according to the surface temperature of the ground area corresponding to different pipelines in the ground area, a preset condition is set to determine whether there is oil droplet leakage. The setting of the preset condition can be obtained based on experience or through statistical analysis and other methods. According to the preset condition, the temperature comparison of each corresponding thermal imaging image data is carried out. The temperature comparison can be carried out through machine learning, image processing technology and other methods to determine whether there is a to-be-labeled monitoring area greater than the preset temperature value in the corresponding thermal imaging image data and output the corresponding judgment result, such as whether there is oil droplet leakage, the corresponding position of the oil droplet leakage in the ground area, etc., and label the to-be-labeled monitoring area and the corresponding maximum temperature value and minimum temperature value in the to-be-labeled monitoring area.

[0070] Please refer to Figure 3 , which is a third flowchart of the oil droplet leakage monitoring method based on image recognition provided by the embodiment of the application. Before feeding the visible light image data into the preset training model to generate a visible light image, the oil droplet leakage monitoring method based on image recognition further includes steps S301-S302.

[0071] Step S301, determine whether the visible light image data contains crack image data.

[0072] Step S302, when the visible light image data contains crack image data, label the crack image data and monitor the oil pipeline and the ground area in the monitoring area corresponding to the crack image data.

[0073] Please refer to Figure 6 , which is a fourth flowchart of the oil droplet leakage monitoring method based on image recognition provided by the embodiment of the application. The oil droplet leakage monitoring method based on image recognition further includes steps S401-S402.

[0074] Step S401, when it is identified that the oil pipeline corresponding to the target monitoring area has oil droplet leakage, generate a prompt information.

[0075] In step S401, the prompt information is used to indicate the corresponding position of the oil pipeline with oil dripping and the target monitoring area. In this embodiment, the prompt information is generated according to the thermal imaging image and the visible light image, that is, the prompt information should at least contain the monitoring area to be labeled in the thermal imaging image and the visible light image.

[0076] In step S402, the thermal imaging image, the visible light image and the prompt information are displayed.

[0077] In the above embodiment, by obtaining the thermal imaging image data and the visible light image data of the oil pipeline and the corresponding ground area image data, the thermal imaging image data meeting the preset condition is determined to identify the monitoring area to be labeled and generate the thermal imaging image, and then the visible light image data is fed into the preset training model to generate the visible light image, and whether the oil pipeline has oil dripping is identified according to the thermal imaging image and the visible light image. Combining the temperature distribution of the thermal imaging and the specific image of the visible light, the existence and distribution of the oil dripping of the oil pipeline are accurately analyzed.

[0078] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the claims of the present application and their equivalents, the present application also intends to include these modifications and variations.

[0079] It should be understood that although each step in the flowchart of the accompanying drawings is displayed in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and they can be executed in other orders. Moreover, at least part of the steps in the flowchart of the accompanying drawings can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or sub-steps or stages of other steps.

[0080] The above-mentioned only for the preferred embodiments of the present application, of course, can not be limited by the scope of the claims of the present application, therefore, the equivalent changes made in accordance with the claims of the present application, still belong to the scope covered by the present application.

Claims

1. An image recognition-based oil dripping monitoring method, characterized by, The oil dripping monitoring method based on image recognition comprises: acquiring thermal imaging image data and visible light image data, the thermal imaging image data and the visible light image data being image data of oil pipeline in a monitoring area and a ground area corresponding to the oil pipeline, the ground area being a ground surface directly below the oil pipeline; dividing the thermal imaging image data into corresponding thermal imaging image data according to a plurality of preset monitoring areas; identifying thermal imaging image data meeting a preset condition in each thermal imaging image data as to-be-labeled image data, so as to identify a preset monitoring area corresponding to the to-be-labeled image data as a to-be-labeled monitoring area, each thermal imaging image data containing a plurality of temperature values, and the preset condition being that the temperature values are greater than a preset temperature value; generating a thermal imaging image according to the thermal imaging image data and the to-be-labeled monitoring area, the thermal imaging image comprising a target monitoring area corresponding to the to-be-labeled monitoring area; feeding the visible light image data into a preset training model to generate a visible light image; and identifying whether the oil pipeline has oil dripping according to the thermal imaging image and the visible light image. The identification of whether the oil pipeline has oil dripping according to the thermal imaging image and the visible light image specifically comprises: comparing the target monitoring area with a ground area at a corresponding position of the visible light image; when the ground area at the corresponding position of the visible light image has oil dripping, identifying that the oil pipeline corresponding to the target monitoring area has oil dripping; and when the ground area at the corresponding position of the visible light image has no oil dripping, monitoring the target monitoring area.

2. The image recognition based oil spill monitoring method of claim 1, wherein, After the thermal imaging image is generated, the oil dripping monitoring method based on image recognition further comprises: labeling a maximum temperature value and a minimum temperature value in a plurality of temperature values corresponding to the to-be-labeled image data to the target monitoring area in the thermal imaging image.

3. The image recognition based oil spill monitoring method of claim 1, wherein, Before the visible light image data is fed into the preset training model to generate a visible light image, the oil dripping monitoring method based on image recognition further comprises: determining whether the visible light image data contains crack image data; and when the visible light image data contains crack image data, labeling the crack image data and monitoring an oil pipeline and a ground area in a monitoring area corresponding to the crack image data.

4. The image recognition based oil weeping monitoring method of claim 1, wherein, The visible light image data is one or more of image data and video frame data; feeding the visible light image data into the preset training model to generate a visible light image comprises: when the visible light image data is image data, feeding the image data into the preset training model to generate the visible light image; and when the visible light image data is video frame data, acquiring a plurality of continuous video frame data to obtain continuous image data and feeding the continuous image data into the preset training model to generate the visible light image. The oil dripping monitoring method based on image recognition further comprises:

5. The image recognition based oil spill monitoring method of claim 1, wherein, ​ generate prompt information when it is identified that the oil liquid pipeline corresponding to the target monitoring area exists oil liquid dripping, the prompt information is used to indicate the oil liquid pipeline existing oil liquid dripping and the corresponding position of the target monitoring area; and display the thermal imaging image, the visible light image and the prompt information.

6. The image recognition based oil weeping monitoring method of claim 1, wherein, The preset training model is trained by feeding an initial training model with a first preset image data set containing oil liquid dripping image data and a second preset image data set not containing oil liquid dripping image data.

7. An image recognition-based oil dripping monitoring device, characterized by, The oil liquid dripping monitoring device based on image recognition comprises: a data acquisition module configured to acquire thermal imaging image data and visible light image data, the thermal imaging image data and the visible light image data being image data of oil liquid pipelines and ground areas corresponding to the oil liquid pipelines in a monitoring area, the ground areas being ground surfaces directly below the oil liquid pipelines; a first data processing module communicatively connected to the data acquisition module and comprising: a data division module configured to divide the thermal imaging image data into corresponding thermal imaging image data according to a plurality of preset monitoring areas; a data analysis module configured to identify thermal imaging image data meeting a preset condition in each thermal imaging image data as to-be-labeled image data, so as to identify a preset monitoring area corresponding to the to-be-labeled image data as a to-be-labeled monitoring area, each thermal imaging image data containing a plurality of temperature values, and the preset condition being that the temperature values are greater than a preset temperature value; and a first image generation module configured to generate a thermal imaging image according to the thermal imaging image data and the to-be-labeled monitoring area, the thermal imaging image comprising a target monitoring area corresponding to the to-be-labeled monitoring area; a second data processing module communicatively connected to the data acquisition module, the second data processing module comprising a second image generation module configured to feed the visible light image data into a preset training model to generate a visible light image; and an image data analysis module communicatively connected to the first data processing module and the second data processing module respectively and configured to identify whether the oil liquid pipelines exist oil liquid dripping according to the thermal imaging image and the visible light image; wherein identifying whether the oil liquid pipelines exist oil liquid dripping according to the thermal imaging image and the visible light image specifically comprises: comparing the target monitoring area with a ground area at a corresponding position of the visible light image; identifying that the oil liquid pipeline corresponding to the target monitoring area exists oil liquid dripping when the ground area at the corresponding position of the visible light image exists oil liquid dripping; and monitoring the target monitoring area when the ground area at the corresponding position of the visible light image does not exist oil liquid dripping.

8. The image recognition based oil spill monitoring device as claimed in claim 7, wherein, The first data processing module further comprises: a data labeling module configured to label a maximum temperature value and a minimum temperature value in a plurality of temperature values corresponding to the to-be-labeled image data to the target monitoring area in the thermal imaging image after the thermal imaging image is generated; and the second data processing module further comprises: An image data labeling module, before feeding the visible light image data into a preset training model to generate a visible light image, the image data labeling module is configured to perform the following steps: determining whether the visible light image data contains crack image data; and when the visible light image data contains crack image data, labeling the crack image data and monitoring the oil pipeline and ground area in the monitoring area corresponding to the crack image data; the oil leakage monitoring device based on image recognition further comprises: a communication module, which is communicatively connected to the image data analysis module and has a monitoring device, and is configured to send the thermal imaging image and the visible light image to the monitoring device to perform the following steps: when it is identified that the oil pipeline corresponding to the target monitoring area exists oil leakage, generating a prompt information, the prompt information is used to indicate the oil pipeline which exists oil leakage and the corresponding position of the target monitoring area; and displaying the thermal imaging image, the visible light image and the prompt information.

9. An image forming apparatus characterized by comprising: The imaging device comprises: a thermal imaging image data module, configured to collect thermal imaging image data; a visible light image data module, configured to collect visible light image data; and the oil leakage monitoring device based on image recognition according to any one of claims 7-8.

Images