An image detection method based on YOLOv5 target detection algorithm

Abstract

This invention provides an image detection method based on the YOLOv5 object detection algorithm, including acquiring an original captured RGB image A, and performing object detection on the RGB image A using the YOLOv5 object detection algorithm; liquid leak detection is performed using the YOLOv5 object detection algorithm: an attention mechanism is added after the first convolutional layer of the CSPDarknet53 network in the Backbone layer of the YOLOv5 object detection algorithm, which not only preserves feature map information but also reduces the computational load and parameter count of subsequent networks. Simultaneously, the attention module with the added attention mechanism can perform attention weighting on the feature map to improve the attention of subsequent networks to important features; furthermore, a feature pyramid layer is added to each BiFPN module of the BiFPN structure in the YOLOv5 object detection algorithm to extract more features of different scales from the image, improving the accuracy of the YOLOv5 object detection algorithm in detecting small objects; if the above techniques are used simultaneously, more accurate detection results will be produced.

Description

Technical Field

[0001] This invention belongs to the field of image detection technology, and relates to an image detection method, particularly an image detection method based on the YOLOv5 target detection algorithm. Background Technology

[0002] In recent years, with my country's increasing emphasis on clean energy, natural gas has gradually entered households and is now used in vast rural areas. As the demand for natural gas continues to increase, the length of natural gas pipelines has also grown. By 2021, the total length of main natural gas pipelines nationwide reached 116,000 kilometers. Over the years, natural gas pipelines are inevitably affected by natural aging, corrosion, human damage, and mechanical failures, leading to leaks. Especially at pipeline joints, leaks can occur due to rusted and loosened bolts and nuts. Natural gas transmission stations are locations with densely packed natural gas pipelines. Leaks in these stations can easily lead to fires and explosions, causing environmental damage and threatening the lives and property of personnel. Therefore, detecting leaks in pipelines within these stations is essential to minimize the damage caused by liquefied natural gas leaks.

[0003] Although existing pipeline liquid leak detection technologies mainly employ traditional image processing methods such as image segmentation, edge detection, and infrared imaging, these methods are difficult to adapt to complex scenarios. In particular, natural gas stations have complex equipment structures with numerous pipelines at varying heights. The shadow areas of these pipelines on the ground under sunlight and the leaking liquid natural gas areas on the ground are quite similar in color in the images. Thus, when using existing technologies to detect liquid natural gas leaks on the ground under sunlight, it is easy to confuse them with the shadow areas of various pipeline equipment on the ground, making it impossible to accurately distinguish between the shadow areas of objects on the ground and the liquid natural gas leak areas, and even leading to erroneous detection results. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide an image detection method based on the YOLOv5 object detection algorithm, comprising the steps of: acquiring an original captured RGB image A, and performing object detection on the RGB image A using the YOLOv5 object detection algorithm;

[0005] In the YOLOv5 object detection algorithm, an attention mechanism is added after the first convolutional layer of the Backbone layer CSPDarknet53 network. This attention mechanism consists of two sub-modules: a channel rearrangement module and an attention module; or / and,

[0006] In the YOLOv5 target detection algorithm, a feature pyramid layer is added to each BiFPN module of the BiFPN structure.

[0007] Preferably, the RGB image A is an RGB image that includes areas without object shadows.

[0008] Preferably, the RGB image A is an RGB image B obtained through filtering and sharpening.

[0009] Preferably, the RGB image A is an image of the leaking area of ​​the natural gas pipeline captured by the inspection robot.

[0010] Preferably, the RGB image A is an RGB image including areas without object shadows and areas with object shadows. Before performing object detection on the RGB image A using the YOLOv5 object detection algorithm, the method further includes a step of removing the object shadow areas from the RGB image, that is, a step of converting the RGB image A into an RGB image J, as follows:

[0011] First, the RGB image A is converted into a YCbCr image C, and the position I at (i,j) in the YCbCr image C is calculated. ij The pixel value is calculated using the following formula:

[0012] I ij =(t ij cosθ ij L d +L e )R ij

[0013] Where L e L d These represent the brightness of the shadowed area and the area without shadow, respectively, with units of nits and R. ij It is the surface reflectance of the pixel, θ ij It is the angle between the direct lighting direction and the surface normal, t ij The attenuation coefficient of direct light; when t ij When t = 1, the target point is in a well-lit area; when t = 1, the target point is in a well-lit area. ij When = 0, the target point is within the object's shadow area;

[0014] Next, calculate the average pixel values ​​of the shadowed area and the area without shadow in the YCbCr image C, respectively. The calculation formula is as follows:

[0015]

[0016] Where w and h represent the height and width of the image, respectively;

[0017] Then, add the average pixel value of the object's shadow area and the average pixel value of the area without object shadows to the pixel value of the Y channel image, respectively, to obtain:

[0018] The ratio of the average number of pixels in the shadow region of an object to the average number of pixels in the region without shadow is called r;

[0019] Next, calculate the YCbCr image after removing the object's shadow region from the YCbCr image C. The calculation formula is as follows:

[0020]

[0021] Where k ij =t ij *cosθ ij ;

[0022] Based on the calculated YCbCr image C with the object shadow area removed, the YCbCr image C is converted into an RGB image J with the object shadow area removed.

[0023] Preferably, the RGB image A is an image of the leaking area of ​​the natural gas pipeline captured by the inspection robot.

[0024] Preferably, the step of removing the object shadow region from the RGB image further includes the following steps:

[0025] Step 01: Obtain the Y luminance component image D in the YCbCr image C, and calculate the mean value of the Y luminance component image D. The calculation formula is as follows:

[0026]

[0027] Where w and h are the width and height of the Y luminance component image D, respectively, and p ij Let be the brightness at position (i, j) in the Y-luminance component image D, with units of nits;

[0028] Step 02: Determine the pixel composition of the Y luminance component image D and its relation to the pixel composition of the Y luminance component image D. The size relationship; where P is the ratio adjustment coefficient; if If the area is a shadow area, then it is a shadow area of ​​an object; otherwise, it is a shadow area without an object.

[0029] Step 03: Perform binarization on the Y luminance component image D: set the pixels in the object shadow area to white and the pixels in the non-object shadow area to black; thus, obtain the Y luminance component binarized image E that distinguishes between the object shadow area and the non-object shadow area. Then, based on the binarized image E, distinguish the object shadow area and the non-object shadow area in the YCbCr image C.

[0030] Preferably, the RGB image A is an RGB image B obtained through filtering and sharpening.

[0031] Preferably, the step of distinguishing the object shadow region and the non-object shadow region in the YCbCr image C based on the binarized image E further includes a step: performing an opening operation on the binarized image E of the Y luminance component to obtain the final binarized image H, which distinguishes the object shadow region and the non-object shadow region in the YCbCr image.

[0032] This invention uses RGB images of the liquid leakage area and applies a YOLOv5-based object detection algorithm to these images for liquid leakage detection. Specifically, an attention mechanism is added after the first convolutional layer of the CSPDarknet53 network in the Backbone layer of the YOLOv5 object detection algorithm. This attention mechanism consists of two sub-modules: a channel rearrangement module and an attention module. This not only preserves feature map information but also reduces the computational cost and parameter count of subsequent networks. Furthermore, the attention module adds attention weights to the feature map, increasing the attention given to important features by subsequent networks. Additionally, a feature pyramid layer is added to each BiFPN module of the BiFPN structure in the YOLOv5 object detection algorithm to extract more features at different scales from the image, improving the accuracy of the YOLOv5 object detection algorithm in detecting small objects. Using these techniques simultaneously yields more accurate detection results. Attached Figure Description

[0033] Figure 1 A schematic flowchart of an image detection method for liquid leakage in natural gas pipelines based on the vision of an inspection robot provided by the present invention;

[0034] Figure 2 This invention provides a schematic flowchart of an image processing method for removing shadow regions of objects in a YCbCr image.

[0035] Figure 3 The image detection results are shown based on the basic YOLOv5 object detection algorithm.

[0036] Figure 4 The image detection results of the YOLOv5 object detection algorithm based on the added attention mechanism are shown.

[0037] Figure 5 The image detection results are shown in the YOLOv5 object detection algorithm based on the improved BiFPN structure.

[0038] Figure 6The image detection results are shown in the YOLOv5 object detection algorithm, which is improved based on the addition of an attention mechanism and the BiFPN structure. Detailed Implementation

[0039] The technical solution of the present invention will be clearly and completely described below using ground images of liquid leaks in natural gas pipelines based on the vision of inspection robots as an example, in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] To address the issue of reduced accuracy in natural gas pipeline liquid leak detection due to the influence of shadowed areas on the final detection results, it is necessary to remove shadowed areas from the images before detecting liquid leaks in natural gas pipelines.

[0041] like Figure 1 The image shows an image detection method for liquid leaks in natural gas pipelines based on the vision of an inspection robot. The steps are as follows:

[0042] Step 1: Obtain the original RGB image A of the natural gas pipeline leak area captured by the inspection robot. Filter and sharpen the original RGB image A to obtain the RGB image B. The purpose is to reduce noise interference in the original image and highlight the contour edges and details in the image.

[0043] Step 2: Convert the filtered and sharpened RGB image B into a YCbCr image C to prepare for removing the shadow areas of the object;

[0044] Step 3: Obtain the Y luminance component image D from the YCbCr image C in Step 2, and calculate the mean of the luminance component image D. The calculation formula is as follows:

[0045]

[0046] Where w and h are the width and height of the Y luminance component image D, respectively, and p ij Let be the brightness at position (i, j) in the Y-luminance component image D, with units of nits;

[0047] Step 4: Determine the pixel composition of the Y luminance component image D and its relation to the pixel composition of the Y luminance component image D. The relationship (where P is the proportion adjustment coefficient): If If the region is a shadow region of an object, then it is a shadowless region of an object; otherwise, it is a shadowless region of an object. The Y-luminance component image D is binarized: the pixels in the shadow region of an object are set to white, and the pixels in the shadowless region of an object are set to black; thus, the Y-luminance component binarized image E, which distinguishes between the shadow region of an object and the shadowless region of an object, is obtained.

[0048] Step 5: Perform an opening operation on the above binarized Y luminance component image E to remove noise pixels and fill in missing edge contours, thus obtaining the final binarized image H. Then, based on the binarized image H, distinguish the object shadow area and the non-object shadow area in the YCbCr image C.

[0049] Step 6: Remove the shadow areas of objects in image C of YCbCr:

[0050] First, calculate the position I at (i,j) in the YCbCr image. ij The pixel value is calculated using the following formula:

[0051] I ij =(t ij cosθ ij L d +L e )R ij

[0052] L e L d These represent the brightness of the shadowed area and the area without shadow, respectively, with units of nits and R. ij It is the surface reflectance of the pixel, θ ij It is the angle between the direct lighting direction and the surface normal, t ij Let be the attenuation coefficient of direct light; where when t ij When t = 1, the target point is in a well-lit area; when t = 1, the target point is in a well-lit area. ij When = 0, the target point is in the object's shadow area;

[0053] Next, calculate the average pixel values ​​of the shadowed area and the area without shadow in the YCbCr image C, respectively. The calculation formula is as follows:

[0054]

[0055] Where w and h represent the height and width of the YCbCr image, respectively;

[0056] Then, add the average pixel value of the object shadow area and the average pixel value of the object shadowless area in the obtained YCbCr image C to the pixel value of the Y luminance component image D, respectively, to obtain the ratio r of the average object shadow area pixels to the average object shadowless area pixels.

[0057] Based on the obtained r value and I ij Next, calculate all pixel values ​​of the YCbCr image after removing the object's shadow region from image C. The formula is as follows:

[0058]

[0059] Where k ij =t ij *cosθ ij

[0060] Then, based on the calculated pixel values ​​of all pixels in the YCbCr image C after removing the object's shadow region, the YCbCr image C is converted into an RGB image J without the object's shadow region; that is, the object's shadow region is removed from the YCbCr image C. Figure 2 As shown.

[0061] Step 7: Apply the YOLOv5 object detection algorithm-based image detection method to the RGB image J of the object-free shadow area obtained in Step 6 to detect liquid leaks in natural gas pipelines. The specific detection method is as follows:

[0062] The first detection method involves adding an attention mechanism (ShuffleAttention) after the first convolutional layer of the CSPDarknet53 network in the Backbone layer of the basic YOLOv5 object detection algorithm. This attention mechanism consists of two sub-modules: a channel rearrangement module and an attention module. The channel rearrangement module reorders the channels of the feature map obtained after the convolutional operation of the first convolutional layer, thus preserving feature map information and reducing the computational cost and parameter count of subsequent networks. Simultaneously, the attention module applies attention weights to the feature map, increasing the attention given to important features by subsequent networks.

[0063] like Figure 3 The image shown is an image detection result based on the basic YOLOv5 object detection algorithm. The black areas in the image represent the leakage areas of the pipe liquid on the ground (i.e., areas without object shadows). There are a total of 14 areas, of which 6 are large black areas with dashed outlines. These 6 areas are the detection results of the pipe liquid leakage on the ground based on the basic YOLOv5 object detection algorithm.

[0064] like Figure 4The image shown is an image detection result of the YOLOv5 object detection algorithm based on the added attention mechanism. The black areas in the image represent the leak area of ​​liquid from the pipe on the ground (i.e., the area without object shadows), totaling 14 blocks. Among them, there are 9 larger black areas with dashed outlines. These 9 areas represent the detection result of the YOLOv5 object detection algorithm with the added attention mechanism for the leak area of ​​liquid from the pipe on the ground, which is 3 smaller than the detection result of the basic YOLOv5 object detection algorithm. The comparison shows that the image detection effect of the basic YOLOv5 object detection algorithm after adding the attention mechanism is better than that of the basic YOLOv5 object detection algorithm.

[0065] The second detection method involves adding a feature pyramid layer to each BiFPN module of the basic YOLOv5 object detection algorithm to extract more features at different scales from the image, thereby improving the accuracy of the YOLOv5 object detection algorithm in detecting small objects.

[0066] like Figure 5 The image shown is the image detection performance of the YOLOv5 object detection algorithm based on the BiFPN structure improvement. The black areas in the image represent the leak area of ​​liquid from the pipe on the ground (i.e., the area without object shadows), totaling 14 blocks. Ten of these are larger black areas with dashed outlines. These 10 blocks represent the detection results of the YOLOv5 object detection algorithm based on the BiFPN structure improvement, which adds four smaller black areas compared to the basic YOLOv5 object detection algorithm. The comparison shows that the image detection performance of the basic YOLOv5 object detection algorithm, after the BiFPN structure improvement, is superior to that based on the basic YOLOv5 object detection algorithm in detecting liquid leaks.

[0067] The third detection method: This method combines the two detection methods mentioned above in the basic YOLOv5 object detection algorithm. Specifically, it adds an attention mechanism (ShuffleAttention) after the first convolutional layer of the CSPDarknet53 network in the Backbone layer of the basic YOLOv5 object detection algorithm, and adds a feature pyramid layer to each BiFPN module of the BiFPN structure in the basic YOLOv5 object detection algorithm. For example... Figure 6The image shown is an image detection result of the YOLOv5 object detection algorithm improved based on the added attention mechanism and BiFPN structure. The black areas in the image represent the leakage area of ​​the pipe liquid on the ground (i.e., the area without object shadows), totaling 14 blocks. All 14 black areas are bounded by dashed lines. These 14 blocks represent the detection results of the YOLOv5 object detection algorithm improved based on the added attention mechanism and BiFPN structure for the leakage area of ​​the pipe liquid on the ground. Compared with the detection results of the basic YOLOv5 object detection algorithm, the YOLOv5 object detection algorithm based on the added attention mechanism, and the YOLOv5 object detection algorithm based on the BiFPN structure, the detection results of this algorithm are 8 smaller black areas, 5 smaller black areas, and 4 smaller black areas, respectively. The detection results are superior to the aforementioned image detection methods.

[0068] The first inventive point of this invention (i.e., step 7) is achieved by first calculating the position I at (i,j) in the YCbCr image. ij The pixel values ​​are calculated, and then the average pixel values ​​of the object shadow area and the object-free shadow area in the YCbCr image are calculated separately. Then, the average pixel values ​​of the object shadow area and the object-free shadow area in the YCbCr image are added to the pixel values ​​of the Y channel image to obtain the ratio r of the average object shadow pixels to the average object-free shadow pixels. Then, based on the obtained r value and I... ij Next, calculate the image pixel values ​​of the new object-free shadow area, and then convert the YCbCr image to an RGB image based on the new object-free shadow area image pixel values ​​to obtain the RGB image of the object-free shadow area. In this way, only the liquid leakage area (i.e. the object-free shadow area) in the RGB image is retained, ensuring the accuracy of liquid leakage detection based on the YOLOv5 target detection algorithm.

[0069] The second inventive point of this invention (i.e., step 8) involves using an RGB image of the liquid leakage area and applying a YOLOv5-based object detection algorithm to detect the liquid leakage. In the YOLOv5 object detection algorithm, an attention mechanism is added after the first convolutional layer of the CSPDarknet53 network in the Backbone layer. This attention mechanism consists of two sub-modules: a channel rearrangement module and an attention module. This not only preserves feature map information but also reduces the computational load and parameter count of subsequent networks. Furthermore, the attention module adds attention weight to the feature map, increasing the attention given to important features by subsequent networks. Additionally, a feature pyramid layer is added to each BiFPN module of the BiFPN structure in the YOLOv5 object detection algorithm to extract more features at different scales from the image, improving the accuracy of the YOLOv5 object detection algorithm in detecting small targets. Using these techniques simultaneously will produce more accurate detection results.

[0070] Although the embodiment uses an image of a natural gas pipeline leaking liquid on the ground as an example from the vision of an inspection robot, the present invention can be applied to images of natural gas pipeline leaks on other objects, such as building walls, and can also be applied to image detection methods for other liquid leaks, such as water, liquefied gas, gasoline, and oil; of course, it can also be used for images including the shadow area of ​​an object and its projection area, such as images including a human body and its projection area, a car and its projection area, etc.

[0071] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. An image detection method based on the YOLOv5 target detection algorithm, characterized in that, The steps include: acquiring the original captured RGB image A, and performing object detection on the RGB image A using the YOLOv5 object detection algorithm; In the YOLOv5 object detection algorithm, an attention mechanism called ShuffleAttention is added after the first convolutional layer of the Backbone layer CSPDarknet53 network. The ShuffleAttention attention mechanism consists of two sub-modules: a channel rearrangement module and an attention module. The RGB image A is an RGB image that includes regions without object shadows and regions with object shadows. Before performing object detection on the RGB image A using the YOLOv5-based object detection algorithm, a step is included to remove the object shadow regions from the RGB image, as follows: First, the RGB image A is converted into a YCbCr image C, and the position I at (i,j) in the YCbCr image C is calculated. ij The pixel value is calculated using the following formula: Where L d L e These represent the brightness of the shadowed area and the area without shadow, respectively, with units of nits and R. ij It is the surface reflectivity of the pixel. It is the angle between the direct lighting direction and the surface normal, t ij The attenuation coefficient of direct light; when t ij When t = 1, the target point is in a well-lit area; when t = 1, the target point is in a well-lit area. ij When =0, the target point is in the object's shadow area; Next, calculate the average pixel values ​​of the shadowed area and the area without shadow in the YCbCr image C, respectively. The calculation formula is as follows: Where w and h represent the height and width of the image, respectively; Then, add the average pixel value of the object's shadow area to the average pixel value of the area without object shadows, respectively, and add the pixel value of the Y channel image to obtain: the ratio of the average pixel value of the object's shadow area to the average pixel value of the area without object shadows is r. Next, calculate all pixel values ​​of the YCbCr image after removing the object's shadow region from image C. The calculation formula is as follows: wherein ; Based on the calculated pixel values ​​of all YCbCr images C after removing the object's shadow region, the YCbCr image C is converted into an RGB image J after removing the object's shadow region.

2. The image detection method according to claim 1, characterized in that, In the YOLOv5 target detection algorithm, a feature pyramid layer is added to each BiFPN module of the BiFPN structure.

3. The image detection method according to claim 1 or 2, characterized in that, The RGB image A is an RGB image B obtained through filtering and sharpening.

4. The image detection method according to claim 1 or 2, characterized by, The RGB image A is an image of the leaking area in the natural gas pipeline captured by the inspection robot.

5. The image detection method according to claim 3, wherein The RGB image A is an image of the leaking area in the natural gas pipeline captured by the inspection robot.

6. The image detection method of claim 1, wherein, The step of removing object shadow areas from the RGB image includes the following steps before the step of removing object shadow areas from the RGB image: Step 01: Obtain the Y luminance component image D in the YCbCr image C, and calculate the mean of the Y luminance component image D. The calculation formula is as follows: Where w and h are the width and height of the Y luminance component image D, respectively, and P ij Let be the brightness at position (i,j) in the Y-luminance component image D, with units of nits; Step 02: Determine the pixel composition of the Y luminance component image D and its relation to the pixel composition of the Y luminance component image D. Size relationship; Where P is the proportional adjustment coefficient; if If the shadow is clear, then the area is the shadow area of ​​the object; otherwise, the area is the shadow area of ​​no object. Step 03: Then perform binarization processing on the Y luminance component image D: set the pixels in the object shadow area to white, and set the pixels in the non-object shadow area to black; That is, we obtain a binary image E of the Y luminance component that distinguishes between the shadow of the object and the shadowless area. Then, based on the binary image E, we distinguish between the shadow area of ​​the object and the shadowless area in the YCbCr image C.

7. The image detection method according to claim 6, wherein The RGB image A is an RGB image B obtained through filtering and sharpening.

8. The image detection method of claim 6, wherein, The step of distinguishing between the object shadow region and the non-object shadow region in the YCbCr image C based on the binarized image E further includes a step: performing an opening operation on the binarized image E of the Y luminance component to obtain the final binarized image H, which distinguishes between the object shadow region and the non-object shadow region in the YCbCr image.

Images