A method for detecting cold leakage areas of large LNG storage tanks based on image stitching

Through the visible light and infrared image stitching technology collected by the drone, the problem of full display and automated analysis of large LNG storage tank wall information is solved, and high-precision cold leakage area detection is achieved, which improves detection efficiency.

CN117764822BActive Publication Date: 2024-12-03DALIAN UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311812574.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-27
Publication Date
2024-12-03
Estimated Expiration
2043-12-27

AI Technical Summary

Technical Problem

It is difficult for the prior art to fully display and automated analysis of the wall information of large LNG storage tanks. The infrared image stitching effect is poor, especially in environments with small temperature differences, which is low in characteristic point recognition, resulting in low detection accuracy and efficiency.

Method used

The drone carries the on-board image module to collect visible light and infrared images, and use spatial positioning identification and image mapping relationship to perform image segmentation and splicing to realize automatic detection of the cold leakage area of ​​large LNG storage tanks.

Benefits of technology

The intelligent detection accuracy and efficiency of the cold leakage area of ​​large LNG storage tanks is improved, and the full-scale temperature matrix splicing of tank wall information is realized and the automatic identification and area calculation of the cold leakage area is realized.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117764822B_ABST
    Figure CN117764822B_ABST
Patent Text Reader

Abstract

A method for detecting cold leakage areas of large LNG storage tanks based on image stitching belongs to the field of automated and intelligent detection of safe operation of large cryogenic containers. The method comprises the following steps: selecting and arranging multiple spatial positioning marks on the surface of the object to be detected, dividing the object to be detected into multiple image acquisition blocks; acquiring visible light and infrared images of each image block; stitching the infrared images based on the spatial positioning marks and the mapping relationship between the visible light image and the infrared image; and identifying and calculating characteristic parameters of cold leakage areas of large LNG storage tanks. The method stitches infrared images based on the spatial positioning marks and the mapping relationship between the visible light image and the infrared image under the same image acquisition module, which can realize automated processing of image data and improve the accuracy and efficiency of intelligent detection of cold leakage areas of large LNG storage tanks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of automated and intelligent detection of safe operation of large cryogenic containers, and displays the overall picture of cold leakage within the circumference of a large LNG storage tank. It specifically involves drone image acquisition and the use of spatial positioning markers and the mapping relationship between visible light images and infrared images to splice partitioned images of the object being measured taken by an infrared camera into a full-scale image. Background Art

[0002] Image stitching technology is a technology that stitches several overlapping images into a complete panoramic image. It is widely used in machine vision fields such as motion detection and tracking, augmented reality, resolution enhancement, video compression and image stabilization.

[0003] At present, the detection technology for cold leakage areas widely used in LNG receiving stations is handheld infrared thermal imaging technology, but this technology can only analyze a single infrared image. It is difficult to establish a complete picture of the tank wall information of a large LNG storage tank, which is not conducive to subsequent automated analysis and database establishment. Therefore, it is necessary to stitch infrared images.

[0004] The patent "Panoramic stitching method of infrared image of generator stator core based on visible light assistance (CN115393195A)" provides a method of infrared stitching of stator core inside large generators with visible light images. The image acquisition device is fixed on a rotating platform at the center of the stator circle, and the image acquisition module on the rotating platform is used to perform 360° partition image acquisition of the stator circumference. However, large LNG storage tanks cannot be inspected from the inside during online inspection, and a rotating platform cannot collect information on the entire wall surface, which is not suitable for large LNG storage tanks; in addition, the existing technology needs to improve the contrast of infrared images, and then extract effective feature points and stitch infrared images by feature point matching. However, the temperature difference of the wall surface of large LNG storage tanks is small, and the effectiveness of infrared image feature point recognition is low. The algorithm that only increases the contrast is not obvious, which will cause feature point matching errors. Summary of the invention

[0005] The present invention utilizes the characteristic that the features in visible light images are relatively obvious, arranges spatial positioning marks in the visible light images, and uses the mapping relationship between the visible light images and infrared images under the same image acquisition module and the fixed-distance shooting of drones to segment and splice the infrared images; it can realize the automatic processing of image data and improve the intelligent detection accuracy and efficiency of the cold leakage area of ​​large LNG storage tanks.

[0006] The technical solution adopted by the present invention is: a large-scale LNG storage tank cold leakage area detection method based on image stitching, and the detection equipment used in the detection method includes:

[0007] Spatial positioning mark: used to divide the surface of the object to be measured into partitions and serve as a reference for image stitching;

[0008] Drone: Realize the planning of fixed-distance routes on cylindrical surfaces and automated flight;

[0009] Airborne imaging module: It contains both optical and infrared lenses to collect visible light and infrared images of the same area;

[0010] The detection method includes the following steps:

[0011] S1. Select and arrange multiple spatial positioning marks on the surface of the object to be measured, and divide the object to be measured into multiple image acquisition blocks;

[0012] The width of the spatial positioning mark should satisfy the requirement that the pixel width of the spatial positioning mark in the visible light lens of the drone is greater than or equal to 4;

[0013] According to the position of the surface of the object being measured and the surrounding interference objects and the obstacle avoidance requirements of the drone, determine the distance L from the drone to the center of the block when shooting at a fixed distance;

[0014] After the shooting distance L is determined, the field of view of the infrared lens of the drone should fully cover each block when shooting at the corresponding position in the center of the block; the vertical distance from the edge of the block to the drone lens and the vertical distance from the center of the block to the drone lens should not differ by more than 5%;

[0015] S2. Collect visible light and infrared images of each image block; the block image is collected by the drone camera at a fixed distance in the center of each block;

[0016] S3. splicing the infrared image based on the spatial positioning identifier and the mapping relationship between the visible light image and the infrared image;

[0017] S3.1 uses the GUI interface to import a visible light image. The user drags the mouse to control the size of the cropping frame to cover the entire block, and generates a cropping frame with the center of the image as the base point. This cropping frame is used to crop all visible light images;

[0018] S3.2 uses Radon transform to obtain the angle at which the vertical spatial positioning mark in the area deviates from the vertical, and determines whether the image is too tilted. If the tilt angle is too large, the block image needs to be re-collected;

[0019] S3.3 uses the Sobel operator to calculate the horizontal and vertical gradients of the regional image, and finds the coordinate position information of the block boundary with the center of each image as the coordinate origin according to the set gradient threshold;

[0020] S3.4 uses the focal length multiple relationship between the infrared lens and the visible light lens, the horizontal distance between the two lenses, and the actual distance to the surface of the object to calculate the mapping matrix between the two, thereby obtaining the position information matrix of the block boundary in the infrared image, and segmenting and splicing the infrared image;

[0021] The centers of the visible light and infrared images are taken as the pixel coordinate origins respectively, and the vertical spatial positioning mark is positioned at the coordinate x1 in the visible light image, and its corresponding position x2 in the infrared image is:

[0022]

[0023] Among them, f1 and f2 are the focal lengths of the visible light and infrared lenses, respectively, both in mm, L is the distance from the drone lens to the center of the block during image acquisition, and S is the horizontal distance between the two lenses in the airborne imaging module;

[0024] S3.5 Use image processing software to export the temperature matrix corresponding to the infrared image; use the position information matrix of the block boundaries in the infrared image to segment and splice the temperature matrix to obtain the full-scale temperature matrix of the wall surface of the measured object;

[0025] S4. Identification of cold leakage areas in large LNG storage tanks and calculation of characteristic parameters;

[0026] S4.1 The two-dimensional temperature matrix obtained by using the temperature scale display, the upper limit of the horizontal axis is the angle of the center angle of the tank wall covered by all blocks, and the upper and lower limits of the vertical axis are the heights of the top and bottom horizontal space positioning marks, respectively. The position and shape of the low temperature area are observed in the figure;

[0027] S4.2 sets the temperature threshold, and the image processing software will automatically identify the number of pixels in the stitched image whose temperature is lower than the threshold and calculate the area of ​​the cooling leakage area based on the mapping relationship between the pixel space and the real space.

[0028] The beneficial effects of the present invention are:

[0029] 1. The present invention utilizes an unmanned aerial vehicle carrying an airborne imaging module to collect partition information of the object under test, and promotes this online automated detection technology to the cold leakage detection of large LNG storage tanks, which helps to further promote and improve the online detection technology of large LNG storage tanks.

[0030] 2. The present invention proposes an infrared image stitching method based on spatial positioning identification and the mapping relationship between visible light images and infrared images under the same image acquisition module, which can realize automatic processing of image data and improve the intelligent detection accuracy and efficiency of cold leakage areas of large LNG storage tanks. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the image acquisition process of the present invention.

[0032] Figure 2 Flowchart of the image stitching method.

[0033] Figure 3 Flowchart of infrared image segmentation.

[0034] Figure 4 This is an image obtained by segmenting and splicing the infrared image of a large LNG storage tank collected by the present invention.

[0035] In the figure: 1. Image processing software, 2. Drone, 3. Spatial positioning mark, 4. Drone fixed-distance route, 5. Drone image collection position, 6. Airborne imaging module. DETAILED DESCRIPTION

[0036] In order to clearly illustrate the technical features of this solution, this solution is described below through a specific implementation case. This implementation case is implemented based on the technical solution of the present invention, but the protection scope of the present invention should not be limited to the following implementation case.

[0037] Figure 1 A large-scale LNG tank cold leakage area detection device based on drone vehicle imaging splicing is shown, including:

[0038] Spatial positioning mark: used to divide the surface of the object to be measured into partitions and serve as a reference for image stitching;

[0039] Drone: It can realize the planning of fixed-distance routes on cylindrical surfaces and automatic flight;

[0040] Airborne imaging module: Contains both optical and infrared lenses, which can simultaneously collect visible light and infrared images of the same area;

[0041] Image processing software: realize digital image processing.

[0042] The present invention also provides a method for detecting cold leakage areas of large LNG storage tanks based on unmanned aerial vehicle imaging splicing, comprising the following steps:

[0043] Step 1: Select and arrange multiple spatial positioning marks on the surface of the object to be measured, and divide the object to be measured into multiple image acquisition blocks;

[0044] Step 2: Collect visible light and infrared images of each image block;

[0045] Step 3: stitching the infrared image based on the spatial positioning mark and the mapping relationship between the visible light image and the infrared image;

[0046] Step 4: Identification of cold leakage areas in large LNG storage tanks and calculation of characteristic parameters.

[0047] The selection of the spatial positioning mark should ensure that the color of the spatial positioning mark in the visible light image is significantly different from the color of the wall around the tank, which is convenient for the position identification of the spatial positioning mark. The width of the spatial positioning mark should satisfy the pixel width of the spatial positioning mark in the visible light lens of the drone is greater than or equal to 4, which is convenient for the position identification of the spatial positioning mark.

[0048] The acquisition of block images is done by the drone lens shooting at a fixed distance in the center of each block. According to the position of the surface of the object being measured and the surrounding interference objects and the obstacle avoidance requirements of the drone, the distance L from the drone to the center of the block is determined when shooting at a fixed distance.

[0049] The partition of the measured object, after the shooting distance L is determined, should satisfy the field of view of the infrared lens when the drone shoots at the corresponding position in the center of the block to fully cover each block. The measured object is a cylinder with an arc surface. After the shooting distance L is determined, the vertical distance from the edge of the block to the drone lens and the vertical distance from the center of the block to the drone lens should not differ by more than 5%. The arc surface shot can be approximately replaced by a plane for subsequent image stitching.

[0050] For the stitching of infrared images, since the size of the spatial positioning marker partitions is consistent and the drone collects images of each block at the corresponding position in the center of each block, the position of each block in the visible light image is relatively fixed. A visible light image is imported using the GUI interface, and the user drags the mouse to control the size of the cropping frame to cover the entire block, generating a cropping frame with the image center as the base point. This cropping frame is used to crop all visible light images to obtain the approximate area of ​​the block on each visible light image and exclude a large number of surrounding interference objects.

[0051] Radon transform is used to obtain the angle at which the vertical spatial positioning mark in the area deviates from the vertical, and to determine whether the image is too tilted and not suitable for stitching. The Sobel operator is used to calculate the horizontal and vertical gradients of the regional image, and the coordinate position information of the block boundary with the center of each image as the coordinate origin is found according to the set gradient threshold. The mapping matrix between the infrared lens and the visible light lens is calculated using the focal length multiple relationship between the two lenses and the actual distance to the surface of the object, thereby obtaining the position information matrix of the block boundary in the infrared image, and the infrared image is segmented and stitched.

[0052] The centers of the visible light and infrared images are taken as the pixel coordinate origins respectively, and the vertical spatial positioning mark is positioned at the coordinate x1 in the visible light image, and its corresponding position x2 in the infrared image is:

[0053]

[0054] Among them, f1 and f2 are the focal lengths of the optical and infrared lenses, respectively, both in mm, L is the distance from the drone lens to the center of the block during image acquisition, and S is the horizontal distance between the two lenses in the onboard imaging module, both in m. The quality of both images is a, in dpi.

[0055] The temperature matrix corresponding to the infrared image is exported using image processing software and saved as a raw file. The temperature matrix is ​​segmented and spliced ​​using the position information matrix of the block boundaries in the infrared image to obtain the full-scale temperature matrix of the wall of the measured object.

[0056] The calculation of characteristic parameters of the cold leakage area uses a two-dimensional temperature matrix obtained by displaying the temperature scale. The upper limit of the horizontal axis is the angle of the center angle of the tank wall covered by all blocks, and the upper and lower limits of the vertical axis are the heights of the top and bottom horizontal space positioning marks. The position and shape of the low temperature area can be observed in the figure. Set the temperature threshold, and the image processing software will automatically identify the number of pixels in the spliced ​​image with a temperature lower than the threshold and calculate the area of ​​the cold leakage area based on the mapping relationship between the pixel space and the physical space.

[0057] Figure 1 After the mid-space positioning mark 3 is arranged, the drone 2 carrying the airborne imaging module 6 is used to collect information of each partition at each waypoint 5 along the fixed-distance route 4, and finally transmit it to the image processing software 1 for digital image processing.

[0058] The specific inspection method includes the following steps:

[0059] Step 1: Determine the inspection range of UAV 2 and the distance L between it and the tank according to the frequent locations of cold leakage of large LNG storage tanks, the size of the area, the spatial range of the site, and the obstacle avoidance requirements of the UAV.

[0060] Step 2: Determine the width of the spatial positioning mark, the distance between the spatial positioning marks, and the number of partitions based on the focal length f and field of view angle of the airborne imaging module 6, and arrange the spatial positioning mark 3 on the tank surface according to the inspection range of the drone (here it is replaced by the lightning protection ring and vertical lightning protection belt carried by the LNG storage tank itself).

[0061] Step 3: Control the drone 2 carrying the airborne imaging module 6 to collect visible light images and infrared images of each block according to the fixed-distance route 3.

[0062] Step 4: The drone lens is facing the center of the block. The block size is consistent, so use the GUI window to manually mark the position of the cropping frame in the visible light image. Use this cropping frame to crop all visible light images to eliminate the interference of most objects around the block. The operation process is as follows: Figure 3shown.

[0063] Step 5: Perform Radon transform calculation on the target area to find out the angle of deviation of the vertical spatial positioning mark and determine whether the image is too tilted and unsuitable for subsequent processing. If the tilt angle is too large, the block image needs to be re-collected.

[0064] Step 6: Use the Sobel operator to calculate the gradient matrix of the visible light image, set a certain gradient threshold, identify the horizontal and vertical boundaries in the visible light image, and obtain and establish the position information matrix of the block boundaries in each visible light image.

[0065] Step 7: Derive the temperature matrix corresponding to each infrared image based on the temperature measurement distance, ambient humidity, emissivity of the object being measured, and ambient temperature during shooting.

[0066] Step 8: Use the focal length multiple relationship between the infrared lens and the visible light lens, the horizontal distance between the two lenses, and the actual distance to the object surface to calculate the position mapping matrix between the two, thereby obtaining the position information matrix of the block boundaries in the infrared image, and segmenting and splicing the temperature matrix corresponding to the infrared image.

[0067] Step 9: Use the temperature scale to display the completed two-dimensional temperature matrix. The displayed image is as follows: Figure 4 As shown in the figure, the upper limit of the horizontal axis is the angle of the center angle of the tank wall covered by all blocks, and the upper and lower limits of the vertical axis are the heights of the top and bottom horizontal space positioning marks, respectively. The position and shape of the low temperature area can be observed in the figure. At the same time, the temperature threshold is set, and the number of pixels with a temperature lower than the threshold in the spliced ​​image is calculated, and the area of ​​the cold leakage area is calculated based on the mapping relationship between the pixel space and the physical space.

Claims

1. A method for detecting cold leakage areas of large LNG storage tanks based on image stitching, characterized in that: The testing equipment used in the testing method includes: Spatial positioning mark: used to divide the surface of the object to be measured into partitions and serve as a reference for image stitching; Drone: Realize the planning of fixed-distance routes on cylindrical surfaces and automated flight; Airborne imaging module: contains both visible light and infrared lenses to collect visible light and infrared images of the same area; The detection method includes the following steps: S1. Select and arrange multiple spatial positioning marks on the surface of the object to be measured, and divide the object to be measured into multiple image acquisition blocks; The width of the spatial positioning mark should satisfy the requirement that the pixel width of the spatial positioning mark in the visible light lens of the drone is greater than or equal to 4; According to the position of the surface of the object being measured and the surrounding interference objects and the obstacle avoidance requirements of the drone, determine the distance L from the drone to the center of the block when shooting at a fixed distance; After the shooting distance L is determined, the field of view of the infrared lens of the drone should fully cover each block when shooting at the corresponding position in the center of the block; the vertical distance from the edge of the block to the drone lens and the vertical distance from the center of the block to the drone lens should not differ by more than 5%; S2. Collect visible light and infrared images of each image block; the block image is collected by the drone camera at a fixed distance from the center of each block; S3. splicing the infrared image based on the spatial positioning identifier and the mapping relationship between the visible light image and the infrared image; S3.1 uses the GUI interface to import a visible light image. The user drags the mouse to control the size of the cropping frame to cover the entire block, and generates a cropping frame with the center of the image as the base point. This cropping frame is used to crop all visible light images; S3.2 uses Radon transform to obtain the angle at which the vertical spatial positioning mark in the area deviates from the vertical, and determines whether the image is too tilted. If the tilt angle is too large, the block image needs to be re-collected; S3.3 uses the Sobel operator to calculate the horizontal and vertical gradients of the regional image, and finds the coordinate position information of the block boundary with the center of each image as the coordinate origin according to the set gradient threshold; S3.4 uses the focal length multiple relationship between the infrared lens and the visible light lens, the horizontal distance between the two lenses, and the actual distance to the surface of the object to calculate the mapping matrix between the two, thereby obtaining the position information matrix of the block boundary in the infrared image, and segmenting and splicing the infrared image; The centers of the visible light and infrared images are taken as the pixel coordinate origins respectively. The coordinates of the vertical spatial positioning mark in the visible light image are x1, and its corresponding position x2 in the infrared image is: Among them, f1 and f2 are the focal lengths of the visible light and infrared lenses, respectively, both in mm, L is the distance from the drone lens to the center of the block during image acquisition, S is the horizontal distance between the two lenses in the airborne imaging module, and the units of parameters L and S are m; a is the image quality, in dpi; S3.5 Use image processing software to export the temperature matrix corresponding to the infrared image; use the position information matrix of the block boundaries in the infrared image to segment and splice the temperature matrix to obtain the full-scale temperature matrix of the wall surface of the measured object; S4. Identification of cold leakage areas in large LNG storage tanks and calculation of characteristic parameters; S4.1 The two-dimensional temperature matrix obtained by using the temperature scale display, the upper limit of the horizontal axis is the angle of the center angle of the tank wall covered by all blocks, and the upper and lower limits of the vertical axis are the heights of the top and bottom horizontal space positioning marks, respectively. The position and shape of the low temperature area are observed in the figure; S4.2 sets the temperature threshold, and the image processing software will automatically identify the number of pixels in the stitched image whose temperature is lower than the threshold and calculate the area of ​​the cooling leakage area based on the mapping relationship between the pixel space and the real space.