A method for measuring the three-dimensional surface temperature field of large piles based on a dual-light camera

Abstract

The present invention relates to a method for measuring the three-dimensional surface temperature field of a large-scale pile using a dual-light camera. The method comprises the following steps: Step 1: Calibrate a temperature map; Step 2: Use an unmanned aerial vehicle to capture images of the site; Step 3: Create a three-dimensional surface mesh model of the pile; Step 4: Calculate the temperature of each vertex in the three-dimensional surface mesh model; Step 5: Create a three-dimensional surface temperature field mesh model of the pile and perform measurements based on this model. Compared to existing technologies, this method offers advantages such as improved accuracy and ease of use for pile temperature monitoring and spontaneous combustion warning.

Description

Technical Field

[0001] The present invention relates to a method for measuring the three-dimensional surface temperature field of a large-scale pile, in particular to a method for measuring the three-dimensional surface temperature field of a large-scale pile based on a dual-light camera. Background Art

[0002] As an important industrial raw material and fuel, coal plays a crucial role in power generation, smelting, chemical industry, building materials and other industries. However, spontaneous combustion of coal piles is a major problem in coal storage, transportation and management.

[0003] Traditional methods for monitoring and early warning of coal pile spontaneous combustion rely primarily on multi-point measurement and infrared imaging. Multi-point measurement involves manually inserting temperature sensors at multiple locations throughout the coal pile to monitor temperature changes. This is a manual, contact-based detection method. However, CO released when coal heats up is harmful to humans. Furthermore, frequent changes in the pile material necessitate repeated insertion of temperature sensors. This manual, multi-point contact detection method poses significant health and safety risks to operators and is tedious. Infrared imaging, a non-contact infrared temperature measurement method, provides 24 / 7 monitoring of the pile's temperature. It offers fast response and accurate measurement, enabling temperature monitoring of fire points, rapid fire alarms, and precise location of fires. However, current methods use handheld infrared thermal imagers to measure and capture temperature around the coal pile. However, due to the limited field of view of current temperature cameras and spatial constraints, they cannot precisely measure the entire coal pile at a thermal power plant. Even if a temperature anomaly is detected using a handheld thermal imager, its spatial location cannot be accurately determined, hindering precise control of coal pile spontaneous combustion.

[0004] After searching, Chinese patent publication number CN 112614215A discloses a method for reconstructing a three-dimensional surface temperature field. Specifically, the method involves using a projection device to emit structured light onto the surface of an object to be measured; a camera photographs the object, reconstructs it in three dimensions, and obtains a three-dimensional surface temperature field model. The three-dimensional reconstruction step includes structured light decoding and matching, and triangulation. The three-dimensional surface temperature field model is a temperature and color comparison table established based on standard sample colorimetric plate data, which automatically digitizes and analyzes the temperature information on the three-dimensional model surface. However, this existing patent uses a projection device, making it incapable of detecting large piles and obtaining test results through table lookup, resulting in low detection accuracy. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a method for measuring the three-dimensional surface temperature field of a large pile based on a dual-light camera.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] According to one aspect of the present invention, a method for measuring the three-dimensional surface temperature field of a large-scale pile based on a dual-light camera is provided, the method comprising the following steps:

[0008] Step 1: Calibrate the temperature map;

[0009] Step 2: Use a drone to take pictures of the site;

[0010] Step 3: Establish a three-dimensional surface mesh model of the pile;

[0011] Step 4: Calculate the temperature of each vertex of the three-dimensional surface mesh model;

[0012] Step 5: Establish a three-dimensional surface temperature field grid model of the pile and perform measurements based on the model.

[0013] As a preferred technical solution, the step 1 is specifically: using a temperature measuring camera to calibrate the temperature curve and solve the corresponding relationship between temperature and grayscale value.

[0014] As a preferred technical solution, the temperature curve calibration is performed by the user selecting an object for temperature calibration, and then parsing the grayscale of the temperature measurement camera image corresponding to different temperatures, and determining the temperature range according to the user's needs; based on the temperature curve, the temperature corresponding to different grayscale values can be determined.

[0015] As a preferred technical solution, the step 2 is specifically as follows:

[0016] Step 2-1: Install the dual-light camera and corresponding program on the drone;

[0017] Step 2-2: Plan the flight path of the drone so that the photos taken by the drone can cover the entire pile;

[0018] Step 2-3: Use the drone to fly along the trajectory and use the dual-light camera to synchronously capture images of the pile.

[0019] As a preferred technical solution, the image of the pile body includes a visible light image and a temperature image of the pile body.

[0020] As a preferred technical solution, the step 3 is specifically as follows:

[0021] Step 3-1, using the captured visible light image sequence to perform three-dimensional reconstruction to obtain a sparse point cloud of the site;

[0022] In step 3-2, based on the sparse point cloud, a 3D envelope surface fitting algorithm is used to create a mesh model of the 3D surface of the large pile.

[0023] As a preferred technical solution, step 4 is: transforming the coordinate system of each vertex in the three-dimensional grid model to solve the coordinates of each vertex in the pixel coordinate system of the temperature measuring camera; solving the image group in which the point appears; averaging the temperature values of the point in all images in the image group to solve the measured temperature of the point.

[0024] As a further preferred technical solution, the step 4 is specifically as follows:

[0025] Step 4-1: Calibrate the internal and external parameters of the dual-light camera to determine the internal parameters of the visible light camera and the temperature camera, as well as the external parameters between the two;

[0026] Step 4-2: Filter out the observable view group of the mesh model vertices built in step 3, and calculate the 3D spatial coordinates of each vertex in the camera coordinate system of each view in its observable image group;

[0027] Step 4-3: Transform the vertices of the mesh model of the three-dimensional surface of the pile into the visible light camera coordinate system;

[0028] P = [X, Y, Z] T

[0029] P1=[X c ,Y c ,Z c ] T

[0030]

[0031] According to the above formula, the transformation relationship between the world coordinate system and the camera coordinate system is obtained according to the camera external parameters; X c ,Y c ,Z c is the coordinate of the point in the world coordinate system, X, Y, Z are the coordinates of the point in the camera coordinate system, R, t are the rotation matrix and translation matrix of the coordinate system, P and P1 are the coordinates of the point in the world coordinate system and the camera coordinate system respectively;

[0032] Step 4-4: Perform coordinate conversion to transform the points in the visible light coordinate system into the local coordinate system of the temperature measurement camera. The coordinate conversion formula is:

[0033] P2=[X r ,Y r ,Z r ]

[0034] P2=R1P1+t1

[0035] In the above formula, P2 is the coordinate of the point in the temperature measurement camera coordinate system, X r ,Y r ,Z ris the coordinate point of the temperature measurement camera coordinate system, R1 and t1 are the rotation and translation components of the visible light coordinate system in the temperature measurement camera coordinate system;

[0036] Step 4-5: According to the formula, convert the points projected to the temperature measurement camera system into the pixel coordinate system of the temperature measurement camera;

[0037]

[0038] The change from the camera coordinate system to the imaging plane coordinate system is deduced from this formula, where u and v are the coordinates of the point in the camera's pixel coordinate system; f x , f y is the focal length of the camera, C x , C y The center point of the image pixel coordinate system;

[0039] Steps 4-6: Based on the formula in the previous step, calculate the positions of the vertices of the mesh model of the large pile's three-dimensional surface in the world coordinate system in different images in the view group. If the coordinates of the point are within the pixel size range of the image, it means that the point is captured in the image at this time. Record the mapping relationship between the point and the image. Traverse all images in the view group for the point and calculate the coordinates of the point on the image in the view group.

[0040] Step 4-7: Based on the mapping relationship between points and images, record the temperature values of each point in different images and take the average of these values to obtain the temperature measurement value of the point;

[0041] Steps 4-8: Follow the above steps to traverse each point in the large grid model and calculate the temperature value of each point.

[0042] As a preferred technical solution, step 5 is: based on the temperature information of each vertex of the three-dimensional surface mesh model obtained in step 4, the temperature information of each point is converted into RGB color information, a three-dimensional surface temperature field mesh model of the pile is established, and the three-dimensional surface temperature field of the large pile is measured based on the model.

[0043] As a further preferred technical solution, the step 5 is specifically as follows:

[0044] Combine the original 3D model with the temperature information of each vertex of the three-dimensional surface mesh model obtained in step 4, and use the temperature-color conversion algorithm to convert the temperature information of each point into RGB color values; write the RGB information corresponding to the temperature into the 3D model to construct a three-dimensional surface temperature field model of the large pile, and measure the three-dimensional surface temperature field of the large pile based on this model.

[0045] Compared with the prior art, the present invention has the following advantages:

[0046] 1) The present invention constructs a three-dimensional surface temperature field of a large pile body based on a dual-light camera carried by an unmanned aerial vehicle (UAV) through a computer vision optimization fusion algorithm, thereby improving the accuracy and usability of pile body temperature monitoring and spontaneous combustion warning.

[0047] 2) The present invention provides a method for measuring the three-dimensional surface temperature field of a large-scale pile using a drone as a temperature sensor carrier, which can measure the temperature field of the large pile as a whole.

[0048] 3) The present invention uses temperature image sequence fusion to calculate the temperature of each vertex of the three-dimensional surface mesh model, thereby improving measurement accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a specific flow chart of the present invention;

[0050] Figure 2 Schematic diagram of visible light image sequence;

[0051] Figure 3 Schematic diagram of infrared image sequence;

[0052] Figure 4 Schematic diagram of the three-dimensional surface temperature field measurement of a large-scale pile. DETAILED DESCRIPTION

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0054] like Figure 1 As shown, the present invention provides a method for measuring the three-dimensional surface temperature field of a large pile based on a dual-light camera, comprising:

[0055] Step 1: Calibrate the temperature curve using a temperature measurement camera to determine the relationship between temperature and grayscale values.

[0056] Step 2: Use a drone to capture images of the site. First, install a dual-light camera and the corresponding software on the drone. Plan the drone's flight path so that the images captured by the drone cover the entire pile. Use the drone to fly along the path, capturing visible light and temperature images of the pile.

[0057] Step 3: Build a 3D surface mesh model of the pile. Use the visible light image taken in the previous step to perform 3D reconstruction to obtain a sparse point cloud of the pile. Perform 3D envelope surface fitting on the sparse point cloud to obtain a 3D surface mesh model of the pile.

[0058] Step 4: Calculate the temperature of each vertex in the 3D surface mesh model. Transform the coordinate system of each vertex in the 3D mesh model to determine the coordinates of each vertex in the temperature camera's pixel coordinate system. Calculate the image group where the point appears. Average the temperature values of the point in all images in the image group to determine the measured temperature at that point.

[0059] Step 5: Create a 3D surface temperature field mesh model for the stack. Based on the vertex temperature information of the 3D surface mesh model from the previous step, convert each vertex's temperature information into RGB color information. This creates a 3D surface temperature field mesh model for the stack.

[0060] Specifically, in step 1, the temperature curve calibration is performed by the user selecting an object for temperature calibration, and then parsing the grayscale of the temperature camera image corresponding to different temperatures, and determining the temperature range according to the user's needs. Based on the temperature curve, the temperature corresponding to different grayscale values can be determined.

[0061] The specific steps in step 2 are:

[0062] Step 2-1: Mount the dual-light camera on the drone and configure the drone flight program;

[0063] Step 2-2: Plan the drone's flight path

[0064] Steps 2-3: Fly the drone according to the path and use the dual-light camera to simultaneously capture images, including visible light images and temperature images.

[0065] The step 3 is specifically as follows:

[0066] Step 3-1: Use the captured visible light image sequence to perform three-dimensional reconstruction to obtain a sparse point cloud of the site.

[0067] Step 3-2: Based on the established sparse point cloud, use the 3D envelope surface fitting algorithm to build a mesh model of the 3D surface of the large pile

[0068] The step 4 is specifically as follows:

[0069] Step 4-1: Calibrate the internal and external parameters of the dual-light camera to determine the internal parameters of the visible light camera and temperature camera as well as the external parameters between the two.

[0070] Step 4-2:

[0071] Filter out the observable view group of the mesh model vertices built in step 3, and calculate the 3D space coordinates of each vertex in each view camera coordinate system in its observable image group

[0072] Step 4-3: Transform the vertices of the mesh model of the three-dimensional surface of the pile into the visible light camera coordinate system.

[0073] P = [X, Y, Z] T

[0074] P1=[X c ,Y c ,Z c ] T

[0075]

[0076] According to the above formula, the transformation relationship between the world coordinate system and the camera coordinate system based on the camera external parameters can be obtained. c ,Y c ,Z c is the coordinate of the point in the world coordinate system, X, Y, Z are the coordinates of the point in the camera coordinate system, R, t are the rotation matrix and translation matrix of the coordinate system.

[0077] Step 4-4: Perform coordinate conversion to transform the points in the visible light coordinate system into the local coordinate system of the temperature measurement camera. The coordinate conversion formula is:

[0078] P2=[X r ,Y r ,Z r ]

[0079] P2=R1P1+t1

[0080] In the above formula, P2 is the coordinate of the point in the temperature measurement camera coordinate system.

[0081] Step 4-5: According to the formula, transform the point projected to the temperature measurement camera system into the pixel coordinate system of the temperature measurement camera.

[0082]

[0083] From this formula, we can deduce the change from the camera coordinate system to the imaging plane coordinate system. In the formula, u and v are the coordinates of the point in the camera's pixel coordinate system. x , f y is the focal length of the camera, C x , C y The center point of the image pixel coordinate system.

[0084] Steps 4-6: Using the formula from the previous step, calculate the positions of the vertices of the mesh model of the large pile's 3D surface in the world coordinate system across the different images in the view group. If the coordinates of a point are within the pixel size of the image, the point is captured at that moment. Record the mapping relationship between the point and the image. Traverse all images in the view group for that point and calculate the coordinates of the point within the image in the view group.

[0085] Step 4-7: Based on the mapping relationship between points and images, the temperature values of each point in different images can be recorded, and the average of these values is the temperature measurement value of the point.

[0086] Step 4-8: Traverse each point in the large grid model according to the above steps, and calculate the temperature value of the point according to the method proposed in this patent.

[0087] The step 5 is specifically as follows:

[0088] Combine the original 3D model with the temperature information of the vertices of the 3D surface mesh model obtained in step 4. Use the temperature-to-color conversion algorithm to convert the temperature information of each point into an RGB color value. The RGB information corresponding to the temperature is written into the 3D model to construct a 3D surface temperature field model of the large pile.

[0089] Therefore, the present invention has the following specific advantages:

[0090] 1) Using a drone equipped with a camera for flight photography solves the inconvenience of handheld photography; 2) Using computer vision 3D reconstruction technology and surface fitting algorithms to establish a large-volume 3D surface mesh model of the pile; 3) Using the established 3D surface mesh model and the camera pose relationship, the temperature value of each vertex of the mesh model is inferred through the internal and external parameters of the temperature measurement camera. Since each vertex can be observed by multiple temperature images, the temperature values in different images are fused, which improves the accuracy of the 3D surface temperature field measurement of large piles.

[0091] The present invention involves 3D modeling, three-dimensional surface temperature field measurement of large-scale piles, computer vision algorithms, and the conversion of temperature and color information. It can be applied to three-dimensional surface temperature field measurement of various large-scale piles, and is particularly suitable for monitoring spontaneous combustion of coal piles. Because the present invention does not require large instruments, it is easy to use, improves the accuracy of temperature measurement cameras, and can measure the temperature field of large piles, showing promising application prospects.

[0092] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for measuring the three-dimensional surface temperature field of a large pile based on a dual-light camera, characterized in that: The method comprises the following steps: Step 1: Calibrate the temperature map; Step 2: Use a drone to take pictures of the site; Step 3: Establish a three-dimensional surface mesh model of the pile; Step 4: Calculate the temperature of each vertex of the three-dimensional surface mesh model; Step 5: Establish a three-dimensional surface temperature field grid model of the pile and perform measurements based on the model; The step 2 is specifically as follows: Step 2-1: Install the dual-light camera and corresponding program on the drone; Step 2-2: Plan the flight path of the drone so that the photos taken by the drone can cover the entire pile; Step 2-3: Use the drone to fly along the trajectory and use a dual-light camera to synchronously capture images of the pile; The images captured of the pile body include visible light images and temperature images of the pile body; Step 4 comprises: performing coordinate transformation on each vertex in the three-dimensional grid model to obtain the coordinates of each vertex in the temperature camera pixel coordinate system; obtaining the image group in which the point appears; averaging the temperature values of the point in all images in the image group to obtain the measured temperature of the point; The step 4 is specifically as follows: Step 4-1: Calibrate the internal and external parameters of the dual-light camera to determine the internal parameters of the visible light camera and the temperature camera, as well as the external parameters between the two; Step 4-2: Filter out the observable view group of the mesh model vertices built in step 3, and calculate the 3D spatial coordinates of each vertex in the camera coordinate system of each view in its observable image group; Step 4-3: Transform the vertices of the mesh model of the three-dimensional surface of the pile into the visible light camera coordinate system; P=[X,Y,Z] T P1=[X c ,Y c ,Z c ] T According to the above formula, the transformation relationship between the world coordinate system and the camera coordinate system is obtained according to the camera external parameters; X c ,Y c ,Z c is the coordinate of the point in the world coordinate system, X, Y, Z are the coordinates of the point in the camera coordinate system, R, t are the rotation matrix and translation matrix of the coordinate system, P and P1 are the coordinates of the point in the world coordinate system and the camera coordinate system respectively; Step 4-4: Perform coordinate conversion to transform the points in the visible light coordinate system into the local coordinate system of the temperature measurement camera. The coordinate conversion formula is: P2=[X r ,Y r ,Z r ] P2=R1P1+t1 In the above formula, P2 is the coordinate of the point in the temperature measurement camera coordinate system, X r ,Y r ,Z r is the coordinate point of the temperature measurement camera coordinate system, R1 and t1 are the rotation and translation components of the visible light coordinate system in the temperature measurement camera coordinate system; Step 4-5: According to the formula, convert the points projected to the temperature measurement camera system into the pixel coordinate system of the temperature measurement camera; The change from the camera coordinate system to the imaging plane coordinate system is deduced from this formula, where u and v are the coordinates of the point in the camera's pixel coordinate system; f x , f y is the focal length of the camera, C x , C y The center point of the image pixel coordinate system; Steps 4-6: Based on the formula in the previous step, calculate the positions of the vertices of the mesh model of the large pile's three-dimensional surface in the world coordinate system in different images in the view group. If the coordinates of the point are within the pixel size range of the image, it means that the point is captured in the image at this time. Record the mapping relationship between the point and the image. Traverse all images in the view group for the point and calculate the coordinates of the point on the image in the view group. Step 4-7: Based on the mapping relationship between points and images, record the temperature values of each point in different images and take the average of these values to obtain the temperature measurement value of the point; Steps 4-8: Follow the above steps to traverse each point in the large grid model and calculate the temperature value of each point.

2. The method for measuring the three-dimensional surface temperature field of a large pile based on a dual-light camera according to claim 1, characterized in that: The step 1 is specifically as follows: using a temperature measuring camera to calibrate the temperature curve and solve the corresponding relationship between temperature and grayscale value.

3. The method for measuring the three-dimensional surface temperature field of a large pile based on a dual-light camera according to claim 2, characterized in that: The temperature curve calibration is performed by the user selecting an object for temperature calibration, and then parsing the grayscale of the temperature camera image corresponding to different temperatures, and determining the temperature range according to the user's needs; based on the temperature curve, the temperature corresponding to different grayscale values can be determined.

4. The method for measuring the three-dimensional surface temperature field of a large pile based on a dual-light camera according to claim 1, characterized in that: The step 3 is specifically as follows: Step 3-1, using the captured visible light image sequence to perform three-dimensional reconstruction to obtain a sparse point cloud of the site; Step 3-2: Based on the established sparse point cloud, a three-dimensional envelope surface fitting algorithm is used to establish a mesh model of the three-dimensional surface of the large pile.

5. The method for measuring the three-dimensional surface temperature field of a large pile based on a dual-light camera according to claim 1, characterized in that: The step 5 is as follows: according to the temperature information of each vertex of the three-dimensional surface mesh model obtained in step 4, the temperature information of each point is converted into RGB color information, a three-dimensional surface temperature field mesh model of the pile is established, and the three-dimensional surface temperature field of the large pile is measured based on the model.

6. A method for measuring the three-dimensional surface temperature field of a large pile based on a bifocal camera according to claim 1 or 5, characterized in that: The step 5 is specifically as follows: Combine the original 3D model with the temperature information of each vertex of the 3D surface mesh model obtained in step 4, and use the temperature-color conversion algorithm to convert the temperature information of each point into RGB color values; The RGB information corresponding to the temperature is written into the 3D model to construct a three-dimensional surface temperature field model of a large pile, and the three-dimensional surface temperature field of the large pile is measured based on the model.

Images