Multi-point temperature correction method, device and system based on visible light and thermal infrared

Abstract

The present invention discloses a multi-point temperature correction method, device and system based on visible light and thermal infrared, and relates to the field of temperature measurement technology. The method comprises: obtaining a reference visible light image and a reference thermal infrared image when a visible light camera and a thermal infrared camera are jointly calibrated to generate joint calibration parameters of the visible light camera and the thermal infrared camera; constructing a temperature compensation curve according to the actual temperature values ​​measured by the thermal infrared camera at different distances from the same heat source; extracting visible light feature points of a target object in a target visible light image and thermal infrared feature points of a target object in a target thermal infrared image; matching a target feature point pair from the visible light feature points and the thermal infrared feature points according to the joint calibration parameters; and correcting the initial temperature value of the target object measured by the thermal infrared camera according to the target feature point pair, the joint calibration parameters and the temperature compensation curve. The present invention combines the calibration of visible light cameras and thermal infrared cameras and feature point matching to measure distance, and performs temperature correction on the temperature measurement result according to the distance, thereby improving the accuracy and reliability of temperature measurement, while controlling costs.

Description

Technical Field

[0001] The present invention relates to the field of temperature measurement technology, and in particular to a multi-point temperature correction method, device and system based on visible light and thermal infrared. Background Art

[0002] In the existing technology, thermal infrared cameras are widely used in temperature measurement. Due to their non-contact, fast response and large area coverage, they are widely used in the fields of industry, medical treatment, building inspection, etc.; while visible light is mainly used for applications such as object detection. However, the existing technology still has the following problems and disadvantages:

[0003] 1. The influence of distance on temperature measurement:

[0004] Existing temperature measurement methods use visible light and thermal infrared images for registration, but the temperature measurement results are often inaccurate due to changes in distance. At the same time, when thermal infrared radiation propagates in the air, it will be affected by absorption and scattering, which may lead to temperature measurement errors, especially when the distance changes greatly.

[0005] 2. Accuracy and resolution:

[0006] Most existing temperature measurement methods use visible light and thermal infrared images for registration to obtain the temperature of an object. However, due to possible differences in distance, the measured temperature is also inaccurate. This method also requires relatively high infrared accuracy and resolution, which will further increase the cost and is not suitable for large-scale popularization. At the same time, most existing methods use thermal infrared cameras, and their temperature measurement accuracy and spatial resolution are limited by the technical level of the sensor.

[0007] 3. Increased cost of depth cameras:

[0008] Currently, there is also a method of using depth cameras for temperature measurement. Depth cameras can provide distance information between the object and the camera, thereby improving the accuracy of temperature measurement; however, depth camera equipment is expensive, which adds additional cost and complexity to the system, which is not conducive to large-scale popularization and application.

[0009] Therefore, the existing temperature measurement technology has certain limitations and shortcomings when facing complex and changeable measurement environments. Summary of the invention

[0010] The technical problem to be solved by the present invention is to provide a multi-point temperature correction method, device and system based on visible light and thermal infrared, which can improve the accuracy and reliability of temperature measurement.

[0011] In order to solve the above technical problems, the present invention provides a multi-point temperature correction method based on visible light and thermal infrared, including: obtaining a reference visible light image and a reference thermal infrared image when a visible light camera and a thermal infrared camera are jointly calibrated to generate joint calibration parameters of the visible light camera and the thermal infrared camera; constructing a temperature compensation curve according to the actual temperature values ​​measured by the thermal infrared camera at different distances from the same heat source; extracting visible light feature points of the target object in the target visible light image and thermal infrared feature points of the target object in the target thermal infrared image; matching target feature point pairs from the visible light feature points and the thermal infrared feature points according to the joint calibration parameters; and correcting the initial temperature value of the target object measured by the thermal infrared camera according to the target feature point pair, the joint calibration parameters and the temperature compensation curve.

[0012] As an improvement of the above scheme, the step of extracting thermal infrared feature points of the target object in the target thermal infrared image includes: obtaining a target thermal infrared image of the target object; extracting two-dimensional temperature data of the target thermal infrared image; denoising the two-dimensional temperature data to extract significant two-dimensional peak points; and extracting thermal infrared feature points based on the two-dimensional peak points.

[0013] As an improvement of the above-mentioned scheme, the step of extracting thermal infrared feature points based on the two-dimensional peak points includes: expanding to all sides with the two-dimensional peak points as the origin, and taking the area with temperature values ​​higher than the ambient temperature as the feature area of ​​the target object; and taking the geometric feature points of the feature area as the thermal infrared feature points of the target object in the target thermal infrared image.

[0014] As an improvement of the above scheme, the step of extracting visible light feature points of the target object in the target visible light image includes: acquiring the target visible light image of the target object; and taking the geometric feature points of the target object area in the target visible light image as visible light feature points respectively.

[0015] As an improvement of the above scheme, the step of matching the target feature point pair from the visible light feature points and the thermal infrared feature points according to the joint calibration parameters includes: determining the minimum depth value and the maximum depth value of the target object from the visible light camera; according to the joint calibration parameters, mapping each visible light feature point to the minimum mapping point and the maximum mapping point under the thermal infrared camera under the minimum depth value and the maximum depth value; connecting the minimum mapping point and the maximum mapping point into line segments to generate multiple baseline line segments, and the baseline line segments correspond to the visible light feature points one by one; calculating the distance of each thermal infrared feature point from each baseline line segment; and taking the visible light feature point corresponding to the baseline line segment and the thermal infrared feature point closest to the baseline line segment as the matching target feature point pair.

[0016] As an improvement of the above-mentioned scheme, the step of correcting the initial temperature value measured by the thermal infrared camera according to the target feature point pair, the joint calibration parameters and the temperature compensation curve includes: performing triangulation on the target object according to the target feature point pair and the joint calibration parameters to calculate the reference distance of the target object; extracting the temperature compensation value corresponding to the reference distance from the temperature compensation curve, wherein the temperature compensation curve records the relationship between the reference distance and the temperature compensation value; and correcting the initial temperature value measured by the thermal infrared camera according to the temperature compensation value to generate a corrected temperature value.

[0017] As an improvement of the above-mentioned scheme, the step of constructing a temperature compensation curve based on the actual temperature values ​​measured by the thermal infrared camera at different distances from the same heat source includes: respectively obtaining the reference temperature values ​​of the same heat source measured by the thermal infrared camera at different reference distances at a reference temperature, wherein the reference temperature is the reference temperature of the heat source, and the reference distance is the distance between the thermal infrared camera and the heat source; performing polynomial regression processing on the reference temperature values ​​at different reference distances according to the reference temperature to generate a temperature compensation curve, wherein the temperature compensation curve records the relationship between the reference distance and the temperature compensation value.

[0018] As an improvement of the above scheme, the step of performing polynomial regression processing on the reference temperature values ​​at different reference distances according to the reference temperature to generate a temperature compensation curve includes: obtaining the reference temperature values ​​of the same heat source measured by the thermal infrared camera at different reference temperatures and different reference distances; performing polynomial regression processing on different reference temperatures and the reference temperature values ​​at different reference distances to respectively generate initial compensation curves corresponding to different reference temperatures that vary with reference distances; adjusting the initial compensation curve according to the reference temperature and the reference temperature value to respectively construct reference compensation curves at different reference distances; performing polynomial regression processing on the reference compensation curve to generate a temperature compensation curve.

[0019] Correspondingly, the present invention also provides a multi-point temperature correction device based on visible light and thermal infrared, comprising a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the steps of the above-mentioned multi-point temperature correction method based on visible light and thermal infrared when executing the computer program.

[0020] Correspondingly, the present invention also provides a multi-point temperature correction system based on visible light and thermal infrared, including a thermal infrared calibration plate, a visible light camera, a thermal infrared camera, a heat source and the above-mentioned multi-point temperature correction device based on visible light and thermal infrared.

[0021] The implementation of the present invention has the following beneficial effects:

[0022] The present invention is based on a multi-point temperature correction method of visible light and thermal infrared. By combining the calibration of a visible light camera and a thermal infrared camera and matching characteristic points for distance measurement, temperature correction is performed on the temperature measurement result according to the distance, thereby improving the accuracy and reliability of temperature measurement and controlling the cost at the same time.

[0023] Furthermore, the present invention adopts a depth-based feature matching algorithm, which significantly reduces the mapping range of feature points and improves the efficiency and robustness of the matching process;

[0024] The present invention adopts a temperature correction algorithm based on distance, which can greatly improve the accuracy of temperature measurement after reducing the influence of distance. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of an embodiment of the multi-point temperature correction method based on visible light and thermal infrared of the present invention;

[0026] Figure 2 is a schematic diagram of an initial compensation curve in the present invention;

[0027] Figure 3 It is a schematic diagram of a reference compensation curve and a temperature compensation curve in the present invention;

[0028] Figure 4 It is a schematic diagram of the structure of an embodiment of the multi-point temperature correction system based on visible light and thermal infrared of the present invention. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings.

[0030] See also Figure 1 , Figure 1 The flowchart of an embodiment of the multi-point temperature correction method based on visible light and thermal infrared of the present invention is shown, comprising:

[0031] S101, obtaining a reference visible light image and a reference thermal infrared image during joint calibration of a visible light camera and a thermal infrared camera to generate joint calibration parameters of the visible light camera and the thermal infrared camera;

[0032] The present invention jointly calibrates a visible light camera and a thermal infrared camera by using a thermal infrared calibration plate to generate joint calibration parameters of the visible light camera and the thermal infrared camera. The specific steps include:

[0033] (1) Obtaining the reference visible light image and reference thermal infrared image of the thermal infrared calibration plate when jointly calibrating the visible light camera and the thermal infrared camera;

[0034] During calibration, the relative positions of the visible light camera and the thermal infrared camera are first fixed; then, the thermal infrared calibration plate is heated and a series of corresponding reference visible light images a1, a2, …, a are obtained. n and reference thermal infrared images b1, b2, …, b n .

[0035] (2) performing checkerboard corner point detection on the reference visible light image and the reference thermal infrared image respectively, so as to locate the reference visible light image and the reference thermal infrared image respectively;

[0036] Checkerboard corner detection is performed on all reference visible light images and reference thermal infrared images. It is necessary to ensure that the reference visible light image a i And reference thermal infrared image b i The positions of the corner points of the chessboard correspond.

[0037] (3) Calibrate the reference visible light image and the reference thermal infrared image after positioning, and obtain the calibration parameters of the visible light camera and the thermal infrared camera;

[0038] The reference visible light image and the reference thermal infrared image are calibrated separately using the camera calibration tool in Matlab to obtain the internal and external parameters cameraParams1 of the reference visible light camera and the internal and external parameters cameraParams2 of the thermal infrared camera.

[0039] (4) The visible light camera calibration parameters and the thermal infrared camera calibration parameters are jointly calibrated to obtain the fusion calibration parameters of the visible light camera and the thermal infrared camera.

[0040] In practical applications, the relative positions of the visible light camera and the thermal infrared camera [R, T] can be obtained first:

[0041] [R,T]=estimateInitialTranslationAndRotation(cameraParams1,cameraParams2)

[0042] Then obtain the fusion calibration parameters cameraParams of the visible light camera and thermal infrared camera:

[0043] cameraParams=stereoParameters(cameraParams1,cameraParams2,R,T)

[0044] Among them, R is the rotation matrix, T is the translation vector, cameraParams1 is the internal and external parameters of the reference visible light camera, and cameraParams2 is the internal and external parameters of the thermal infrared camera.

[0045] S102, constructing a temperature compensation curve according to actual temperature values ​​measured by the thermal infrared camera at different distances from the same heat source;

[0046] Specifically, according to the actual temperature values ​​measured by the thermal infrared camera at different distances from the same heat source, the steps of constructing the temperature compensation curve include:

[0047] (1) Obtaining the reference temperature values ​​of the same heat source at different reference distances at reference temperature measured by a thermal infrared camera;

[0048] In practical applications, the reference temperature values ​​of the same heat source at different reference distances at a single reference temperature can be measured to simplify the measurement; the reference temperature values ​​of the same heat source at different reference temperatures and different reference distances can also be measured to improve the accuracy of the temperature compensation curve;

[0049] It should be noted that the reference temperature is the reference temperature of the heat source, and the reference distance is the distance between the thermal infrared camera and the heat source.

[0050] Accordingly, the steps of generating the reference temperature value include:

[0051] (1.1) Obtain the actual temperature value within a preset time period at a single / different reference temperature and reference distance measured by a thermal infrared camera;

[0052] When measuring, set the heat source to a single / different reference temperature T i The devices are placed at different reference distances from the thermal infrared camera, and the thermal infrared camera is used to measure the actual temperature values ​​within a preset time at a single / different reference temperature and reference distance.

[0053] (1.2) construct temperature curves at single / different reference temperatures and reference distances according to the actual temperature values;

[0054] (1.3) Smoothing the temperature curves respectively;

[0055] (1.4) Calculate the average temperature value of the actual temperature values ​​in the temperature curve after smoothing, and use the average temperature value as the reference temperature value under different reference temperatures and reference distances;

[0056] The actual temperature values ​​within the preset time are plotted into a temperature curve and smoothed, and the average value of the actual temperature values ​​is taken as the reference distance d i The reference temperature value t i ,Right now:

[0057]

[0058] Among them, t ij At the reference distance d iThe actual temperature value of the jth measurement under the condition of n is the number of measurements within the preset time.

[0059] (2) performing polynomial regression processing according to the reference temperature at different reference distances to generate a temperature compensation curve;

[0060] The specific steps include:

[0061] (2.1) Obtaining the reference temperature values ​​of the same heat source measured by a thermal infrared camera at different reference distances at a single / different reference temperatures;

[0062] (2.2) The single / different reference temperatures and the reference temperature values ​​corresponding to the reference temperatures at different reference distances are subjected to polynomial regression processing to generate initial compensation curves corresponding to the single / different reference temperatures as the reference distance changes (see Figure 2 );

[0063] (2.3) The initial compensation curve is adjusted according to the reference temperature and the base temperature to construct the base compensation curves at different reference distances (see Figure 3 );

[0064] (2.4) The reference compensation curve is processed by polynomial regression to generate a temperature compensation curve (see Figure 3 );

[0065] For example, the reference distance d at different reference temperatures i and (t i -T i ) to perform polynomial regression to obtain the temperature compensation curve:

[0066] T c =f(d)=a0+a1d+…+a n d n

[0067] Among them, T c is the temperature compensation value of the thermal infrared camera, d is the reference distance, a0, a1, …, a n are the regression coefficients determined from the calibration data.

[0068] From the above, it can be seen that the temperature compensation curve records the relationship between the reference distance and the temperature compensation value.

[0069] S103, extracting visible light feature points of the target object in the target visible light image and thermal infrared feature points of the target object in the target thermal infrared image;

[0070] Accordingly, the step of extracting the visible light feature points of the target object in the target visible light image includes:

[0071] (1) obtaining a target visible light image of a target object;

[0072] (2) The geometric feature points of the target object area in the target visible light image are respectively used as visible light feature points.

[0073] It should be noted that the geometric feature point may be a pixel center point or a mass point, etc., but is not limited thereto and may be selected according to actual conditions.

[0074] In practical applications, machine learning neural network methods such as target detection or segmentation can be used to obtain the pixel center point or mass point of the target object, that is, the visible light feature point P = {p i},i=1,2,…,m.

[0075] In addition, due to the resolution limitation of the thermal infrared camera, it is difficult to obtain accurate feature points of the target object. Therefore, the steps of extracting the thermal infrared feature points of the target object in the target thermal infrared image include:

[0076] (1) Acquire a target thermal infrared image of the target object;

[0077] (2) extracting two-dimensional temperature data of the target thermal infrared image;

[0078] (3) De-noising the two-dimensional temperature data to extract significant two-dimensional peak points;

[0079] In practical applications, the mean smoothing function cv2.blur in the OpenCV library can be used to achieve smoothing:

[0080]

[0081] Where T is the two-dimensional temperature data before smoothing; T smoothed is the smoothed two-dimensional temperature data; is the size of the filter kernel, which can be set and adjusted according to the resolution of the thermal infrared camera.

[0082] In addition, the two-dimensional peak point can be obtained by local maximum detection:

[0083]

[0084] in,

[0085] M is the point set of two-dimensional peak points;

[0086] (x, y) is the image coordinate, which represents a pixel point;

[0087] N(x,y) is the neighborhood of pixel (x,y);

[0088] (x′, y′) is the pixel coordinate point in the neighborhood N (x, y) point set.

[0089] (4) Extracting thermal infrared feature points based on two-dimensional peak points;

[0090] Accordingly, the step of extracting thermal infrared feature points according to the two-dimensional peak points includes:

[0091] (4.1) Expand around the two-dimensional peak point as the origin, and take the area with a temperature higher than the ambient temperature as the characteristic area of ​​the target object;

[0092] Take all the selected two-dimensional peak points as the origin and expand them in all directions to find the area where the temperature value is obviously higher than the ambient temperature as the characteristic area of ​​the target object:

[0093] R i ={(x,y)|T smoothed (x,y)>T env +ΔT}

[0094] Among them, R i is the characteristic area, T env is the ambient temperature, and ΔT is the temperature threshold, which is used to distinguish between biological temperature and ambient temperature.

[0095] (4.2) The geometric feature points of the feature area are respectively used as the thermal infrared feature points of the target object in the target thermal infrared image.

[0096] It should be noted that the geometric feature point may be a center point or a mass point, etc., but is not limited thereto and may be selected according to actual conditions.

[0097] Select each feature area R i The center point or mass point Q = {q j}, i = 1, 2, …, n are the thermal infrared feature point coordinates of the target object in the thermal infrared image.

[0098] S104, matching target feature point pairs from visible light feature points and thermal infrared feature points according to the joint calibration parameters;

[0099] The present invention can use a depth-based feature matching algorithm to obtain corresponding matching target feature point pairs under visible light and thermal infrared dual cameras, and the specific steps include:

[0100] (1) Determine the minimum depth value D of the target object from the visible light camera min and the maximum depth D max ;

[0101] It should be noted that the minimum depth value D minIt refers to the minimum depth of the target object within the camera's field of view, and the maximum depth value D max It refers to the maximum depth within which the target object can move within the camera's field of view.

[0102] (2) According to the joint calibration parameters, each visible light feature point is mapped to the minimum mapping point and the maximum mapping point under the thermal infrared camera at the minimum depth value and the maximum depth value respectively;

[0103] The minimum depth value D of the visible light feature point is calculated using the joint calibration parameters obtained in step S101. min and the maximum depth value D max In this case, it is mapped to the mapping point under the thermal infrared camera. The specific mapping relationship can be expressed as:

[0104]

[0105] in,

[0106] Φ is the mapping function, which is determined by the camera calibration parameters;

[0107] p i is the visible light feature point;

[0108] p i ′ is the minimum mapping point;

[0109] p i ″ is the maximum mapping point;

[0110] D min is the minimum depth value;

[0111] D max is the maximum depth value;

[0112] C is the joint calibration parameter, including the intrinsic parameter matrix K of the visible light camera v , the distortion parameter D of the visible light camera v , the internal parameter matrix K of the thermal infrared camera T , the distortion parameter D of the thermal infrared camera T , the rotation matrix R of the thermal infrared camera relative to the visible light, and the translation vector T of the thermal infrared camera relative to the visible light.

[0113] Furthermore, the step of mapping the visible light feature points to the mapping points under the thermal infrared camera includes:

[0114] (2.1) Dedistort the coordinates of the visible light feature points and convert them into normalized coordinates P v ′:

[0115] P v ′ = cv2.undistortPoints(P v ,Kv ,D v )

[0116] Among them, P v is the coordinate of the visible light feature point.

[0117] (2.2) Convert normalized coordinates to homogeneous coordinates

[0118]

[0119] (2.3) Convert the homogeneous coordinates into the thermal infrared camera coordinate system:

[0120]

[0121] Wherein, Z is a predefined depth value.

[0122] (2.4) Normalized and converted into the homogeneous coordinates P of the thermal infrared camera coordinate system T ′:

[0123]

[0124] (2.5) According to the joint calibration parameters, the normalized homogeneous coordinates are projected onto the thermal infrared camera to obtain the corresponding mapping point coordinates P T :

[0125] P T =cv2.projectPoints(P T ′,K T ,D T )

[0126] (3) connecting the minimum mapping point and the maximum mapping point into line segments respectively to generate multiple reference line segments;

[0127] It should be noted that the reference line segments correspond to the visible light feature points one by one. i ′ and the maximum mapping point p i ′ are connected into line segments, and m corresponding reference line segments s1, s2, …, s m ,in:

[0128]

[0129] (4) Calculate the distance between each thermal infrared feature point and each reference line segment respectively;

[0130] The distance between the thermal infrared feature point and the m reference line segments in step S103 is calculated.

[0131] (5) The visible light feature point corresponding to the reference line segment and the thermal infrared feature point closest to the reference line segment are respectively used as the matching target feature point pair.

[0132] The Hungarian matching algorithm can be used to obtain the thermal infrared feature points closest to the m corresponding reference line segments:

[0133]

[0134] Among them, d ij Represents the thermal infrared feature point q j To the baseline segment s i distance.

[0135] Finally, the baseline segment s i The corresponding visible light feature point p i and distance to the baseline segment s i The nearest thermal infrared feature point q j That is, the target feature point pair is matched; that is, the reference line segment is matched one by one with the visible light feature point and the thermal infrared feature point.

[0136] Therefore, the present invention adopts a depth-based feature matching algorithm, which significantly reduces the mapping range of feature points and improves the efficiency and robustness of the matching process.

[0137] S105, correcting the initial temperature value of the target object measured by the thermal infrared camera according to the target feature point pair, the joint calibration parameters and the temperature compensation curve.

[0138] The present invention performs triangulation measurement on all target feature point pairs and obtains reference distances, thereby performing correction according to a temperature compensation curve. The specific steps include:

[0139] (1) triangulate the target object based on the target feature point pair and the joint calibration parameters to calculate the reference distance of the target object;

[0140] Use the joint calibration parameters to perform triangulation and calculate the distance d:

[0141] d = cv2.triangulatePoints(p i ,q j ,cameraparams)

[0142] Among them, p i is the visible light feature point, q j It is the thermal infrared feature point.

[0143] (2) extracting a temperature compensation value corresponding to the reference distance from the temperature compensation curve;

[0144] It should be noted that the temperature compensation curve records the relationship between the reference distance and the temperature compensation value;

[0145] (3) Correcting the initial temperature value measured by the thermal infrared camera according to the temperature compensation value to generate a corrected temperature value.

[0146] Use the temperature compensation curve to correct the measured initial temperature value:

[0147] T obj =T r -T c

[0148] Among them, T c is the temperature compensation value of the thermal infrared camera at the reference distance d, T r is the initial temperature value directly measured by the thermal infrared camera, T obj The target temperature is the corrected temperature value obtained according to the temperature compensation curve.

[0149] Therefore, the present invention adopts a distance-based temperature correction algorithm, which can greatly improve the accuracy of temperature measurement after reducing the influence of distance.

[0150] In summary, the present invention matches the feature points extracted from the visible light image and the thermal infrared image (visible light feature points and thermal infrared feature points), so as to more accurately extract the target feature point pairs while controlling the cost, thereby obtaining the reference distance and performing temperature correction, effectively improving the accuracy and reliability of temperature measurement.

[0151] Correspondingly, the present invention also discloses a multi-point temperature correction device based on visible light and thermal infrared, including a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the steps of the above-mentioned multi-point temperature correction system based on visible light and thermal infrared when executing the computer program.

[0152] like Figure 4 As shown, the present invention also discloses a multi-point temperature correction system 100 based on visible light and thermal infrared, which includes a thermal infrared calibration plate 1, a visible light camera 2, a thermal infrared camera 3, a heat source 4 and the above-mentioned multi-point temperature correction device 5 based on visible light and thermal infrared.

[0153] It should be noted that the present invention jointly calibrates the visible light camera and the thermal infrared camera with the help of a thermal infrared calibration board 1, a visible light camera 2, and a thermal infrared camera 3; at the same time, a temperature compensation curve is constructed with the help of a heat source 4, a visible light camera 2, and a thermal infrared camera 3; finally, the data is input into a multi-point temperature correction device 5 based on visible light and thermal infrared to achieve temperature correction.

[0154] Therefore, the multi-point temperature correction system 100 based on visible light and thermal infrared of the present invention performs distance measurement by combining the calibration of visible light camera and thermal infrared camera and feature point matching, and performs temperature correction on the temperature measurement result according to the distance, thereby improving the accuracy and reliability of temperature measurement while controlling costs.

[0155] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A multi-point temperature correction method based on visible light and thermal infrared, characterized in that: include: Acquire a reference visible light image and a reference thermal infrared image during joint calibration of a visible light camera and a thermal infrared camera to generate joint calibration parameters of the visible light camera and the thermal infrared camera; Constructing a temperature compensation curve according to actual temperature values ​​measured by the thermal infrared camera at different distances from the same heat source; Extracting visible light feature points of the target object in the target visible light image and thermal infrared feature points of the target object in the target thermal infrared image; According to the joint calibration parameters, matching target feature point pairs from the visible light feature points and thermal infrared feature points; The initial temperature value of the target object measured by the thermal infrared camera is corrected according to the target feature point pair, the joint calibration parameters and the temperature compensation curve.

2. The multi-point temperature correction method based on visible light and thermal infrared as claimed in claim 1, characterized in that: The steps of extracting thermal infrared feature points of a target object in a target thermal infrared image include: Acquire a target thermal infrared image of a target object; Extracting two-dimensional temperature data of the target thermal infrared image; Performing denoising on the two-dimensional temperature data to extract significant two-dimensional peak points; Thermal infrared feature points are extracted according to the two-dimensional peak points.

3. The multi-point temperature correction method based on visible light and thermal infrared as claimed in claim 2, characterized in that: The step of extracting thermal infrared feature points according to the two-dimensional peak points comprises: Expanding to all sides with the two-dimensional peak point as the origin, and taking the area with a temperature value higher than the ambient temperature as the characteristic area of ​​the target object; The geometric feature points of the feature area are respectively used as thermal infrared feature points of the target object in the target thermal infrared image.

4. The multi-point temperature correction method based on visible light and thermal infrared as claimed in claim 1, characterized in that: The step of extracting visible light feature points of the target object in the target visible light image includes: Acquire a target visible light image of a target object; The geometric feature points of the target object area in the target visible light image are respectively used as visible light feature points.

5. The multi-point temperature correction method based on visible light and thermal infrared as claimed in claim 1, characterized in that: The step of matching a target feature point pair from the visible light feature points and the thermal infrared feature points according to the joint calibration parameters comprises: Determine a minimum depth value and a maximum depth value of the target object from the visible light camera; According to the joint calibration parameters, each visible light feature point is mapped to the minimum mapping point and the maximum mapping point under the thermal infrared camera under the minimum depth value and the maximum depth value respectively; Connecting the minimum mapping point and the maximum mapping point into line segments respectively to generate a plurality of reference line segments, wherein the reference line segments correspond to the visible light feature points one by one; Calculate the distance between each thermal infrared feature point and each reference line segment respectively; The visible light feature point corresponding to the reference line segment and the thermal infrared feature point closest to the reference line segment are respectively used as a matching target feature point pair.

6. The multi-point temperature correction method based on visible light and thermal infrared as claimed in claim 1, characterized in that: The step of correcting the initial temperature value measured by the thermal infrared camera according to the target feature point pair, the joint calibration parameter and the temperature compensation curve comprises: Performing triangulation on the target object according to the target feature point pair and the joint calibration parameters to calculate a reference distance of the target object; Extracting a temperature compensation value corresponding to the reference distance from the temperature compensation curve, wherein the temperature compensation curve records a relationship between the reference distance and the temperature compensation value; The initial temperature value measured by the thermal infrared camera is corrected according to the temperature compensation value to generate a corrected temperature value.

7. The multi-point temperature correction method based on visible light and thermal infrared as claimed in claim 1, characterized in that: The step of constructing a temperature compensation curve according to the actual temperature values ​​measured by the thermal infrared camera at different distances from the same heat source comprises: Respectively obtaining reference temperature values ​​of the same heat source measured by the thermal infrared camera at different reference distances at a reference temperature, wherein the reference temperature is a reference temperature of the heat source, and the reference distance is a distance between the thermal infrared camera and the heat source; A polynomial regression process is performed according to the reference temperature at different reference distances to generate a temperature compensation curve, wherein the temperature compensation curve records the relationship between the reference distance and the temperature compensation value.

8. The multi-point temperature correction method based on visible light and thermal infrared as claimed in claim 7, characterized in that: The step of performing polynomial regression processing according to the reference temperature values ​​at different reference distances to generate a temperature compensation curve comprises: Obtaining reference temperature values ​​of the same heat source measured by the thermal infrared camera at different reference temperatures and different reference distances; Performing polynomial regression processing on different reference temperatures and the reference temperature values ​​at different reference distances to generate initial compensation curves corresponding to different reference temperatures and varying with reference distances; The initial compensation curve is adjusted according to the reference temperature and the base temperature value to respectively construct base compensation curves at different reference distances; The reference compensation curve is subjected to polynomial regression processing to generate a temperature compensation curve.

9. A multi-point temperature correction device based on visible light and thermal infrared, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A multi-point temperature correction system based on visible light and thermal infrared, characterized in that: It comprises a thermal infrared calibration plate, a visible light camera, a thermal infrared camera, a heat source and the multi-point temperature correction device based on visible light and thermal infrared as described in claim 9.

Images