Multi-target and long-distance field domain temperature measurement method based on dual-spectrum imaging

Abstract

The invention discloses a multi-target and long-distance field domain temperature measurement method based on dual-spectrum imaging. A visible light camera module and an infrared camera module are included. Obtaining internal parameters and external parameters in the visible light camera module and the infrared camera module, and optimizing the internal parameters and the external parameters; performing quick image feature search and image feature stereo matching on the acquired visible light image and infrared image to realize identification and positioning during image measurement; registering and fusing the visible light image and the infrared image by adopting a double-spectrum image fusion algorithm to obtain a fused image; carrying out image intersection measurement positioning and temperature measurement area stereo matching on the image; performing systematic edge calculation on the image by adopting a data-based intelligent analysis algorithm, and analyzing a temperature change trend according to the obtained image data to obtain the temperature of the measured object; the current running state of the detected power equipment is evaluated, the equipment fault is diagnosed, and the application end is used for storing, displaying and processing the detection result; and the temperature detection accuracy of the power equipment is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-spectral temperature measurement systems, and in particular to a multi-target, long-distance field temperature measurement method based on dual-spectrum imaging. Background Art

[0002] Substations are usually large in area and have a high frequency of temperature detection. The detection of abnormal temperature of substation equipment relies on handheld infrared thermometers, which have low temperature measurement efficiency and high risk factor. Inspection robots equipped with temperature measurement systems are difficult to adapt to the complex and changeable substation environment. Therefore, more manpower and material resources are required, which greatly increases the cost of monitoring abnormal temperature of substation equipment.

[0003] The field temperature measurement system of substation equipment needs to achieve high-precision, long-distance temperature monitoring. The temperature monitoring needs to be continuous and real-time, and can perform online monitoring and early warning of various equipment in the station to achieve automatic monitoring. However, the existing temperature monitoring system that relies only on infrared detectors has low temperature measurement accuracy. The temperature measurement system that integrates visible light and infrared images is mostly used for close-range temperature detection. The temperature monitoring accuracy and temperature measurement distance of substation equipment have great limitations. Today, the temperature monitoring of key parts of substation equipment is still in a semi-automatic state. Most of the temperature monitoring devices that have been put into production are specially developed and designed for a certain type of equipment, and their uses are relatively single.

[0004] When diagnosing and identifying faults in substation equipment, the temperature of electrical equipment will vary due to differences in transmission power and environment. For accurate diagnosis, it is necessary to analyze the temperature data of equipment operation over a period of time. The temperature measurement system needs to store temperature measurement data and compare different temperature characteristics. However, the existing methods for identifying and diagnosing substation equipment faults based on temperature data have relatively simple analysis factors and cannot achieve accurate diagnosis and identification of substation equipment faults. Summary of the invention

[0005] Purpose of the invention: In order to overcome the deficiencies in the prior art, the present invention provides a multi-target, long-distance field temperature measurement method based on dual-spectrum imaging. By optimizing the internal and external parameters of the visible light camera and the infrared camera in the field temperature measurement equipment, the collected infrared images and visible light images are processed to obtain the temperature of the object being measured, and whether the power equipment is faulty is analyzed based on the temperature.

[0006] Technical solution: To achieve the above purpose, the present invention provides a multi-target, long-distance field temperature measurement method based on dual-spectrum imaging, including a field temperature measurement device, wherein the field temperature measurement device includes a visible light camera module and an infrared camera module; and includes the following steps:

[0007] S1. Using infrared corner detection algorithm, binocular visible light camera calibration algorithm and visible light and infrared camera calibration algorithm based on feature point extraction, obtain the internal and external parameters in the visible light camera module and infrared camera module, and optimize them;

[0008] S2, image recognition and matching algorithm based on deep learning and feature engineering, which can quickly search for image features and stereo matching of image features of collected visible light images and infrared images, and realize recognition and positioning during image measurement;

[0009] S3, using an image fusion algorithm based on IHS transform and wavelet transform and combining the high- and low-frequency part fusion rules to obtain a dual-spectrum image fusion algorithm, the dual-spectrum image fusion algorithm aligns and fuses the visible light image and the infrared image to obtain a fused image;

[0010] S4, performing image intersection measurement and positioning and temperature measurement area stereo matching on the image processed in S2 based on a high-precision intersection measurement and positioning algorithm and a temperature measurement area stereo matching algorithm;

[0011] S5, using a data-based intelligent analysis algorithm, and performing system edge computing based on the fused image in S3 and the processed image in S4, analyzing the temperature change trend based on the acquired image data to obtain the temperature of the object under test; and evaluating the current operating status and the status of the operating status of the object under test, and diagnosing the current fault of the object under test;

[0012] S6. Upload the temperature of the object under test, the operating status of the object under test, and the fault result data obtained in S5 to the application end, and the application end stores, displays, and processes the detection results.

[0013] Further, in S1, in the infrared corner point detection algorithm, one or more colored or shaped markers are set on the collected checkerboard image to mark the initial position, and the homography matrix H obtained by using the initial pixel coordinates of the unit grid in the pixel coordinate system and the world coordinates set in the world coordinate system is expanded outward to the entire checkerboard area to complete the corner point detection; including the following steps:

[0014] S1-1, setting one or more colored or shaped markers on the checkerboard calibration board to mark the initial position;

[0015] S1-2, using the infrared camera module to collect the chessboard calibration plate image and perform preliminary image processing;

[0016] S1-3, detecting the pixel coordinates of the pre-set mark by morphological or color detection algorithm;

[0017] S1-4, according to the pixel coordinates of the marker detected in step 3, determine the pixel coordinates of the four points of the initial grid, and calculate the corresponding sub-pixel corner point coordinates through iteration;

[0018] S1-5, according to the sub-pixel corner coordinates of the four points and the world coordinates of the four points in the world coordinate system, calculate the homography matrix H;

[0019] S1-6. Multiply the homography matrix H and the world coordinates of the adjacent grid in the world coordinate system to obtain the starting rough sub-pixel corner points of the other two unknown points of the adjacent grid, and calculate the coordinates of all sub-pixel corner points in the image based on the sub-pixel corner point solution principle.

[0020] Furthermore, in the binocular visible light camera calibration algorithm, Bouguet correction and essential matrix decomposition are performed on the corresponding corner points on the original image to optimize the external parameters, so that the imaging origin coordinates of the left and right view images are consistent, the camera optical axes of the two cameras are parallel, the left and right imaging planes are coplanar, and the epipolar lines are aligned;

[0021] The Bouguet correction is performed on the corresponding corner points on the collected original image, including finding the common corner point pairs (p li ,p ri ) corresponds to the coordinate pair (P li ,P ri ); Set point p(u, v) as the pixel coordinate on the image, and P(X, Y) as the corresponding coordinate in the normalized coordinate system;

[0022] p=KP

[0023]

[0024] When the pixel point p and the camera's intrinsic parameter K are known, the corresponding coordinate P in the regular coordinate system is calculated. The calculation process is as follows:

[0025] P=K -1 p

[0026] For each pair of left and right camera corner points (p li ,p ri ), its normal coordinate system coordinate pair (P li ,P ri )for:

[0027]

[0028] Get the normal coordinate system coordinate pair of the left and right view images (P li ,P ri ) and then perform distortion removal;

[0029] The left and right view images are rotated according to the rotation relationship between the two cameras. It is known that the rotation matrix R and the translation vector T between the left and right view images satisfy the following relationship:

[0030] P li =RP ri +T

[0031] Where P li is the coordinate in the normalized coordinate system of the left camera, P ri is the coordinate in the normalized coordinate system of the right camera;

[0032] The left image is rotated by half the angle in the positive direction of R, and the right image is rotated by half the angle in the negative direction of R, so that the overlap area of ​​the left and right cameras is maximized after rotation and the impact on the reprojection error is minimized; the rotation angle RT is calculated according to the translation relationship T between the two cameras, and the two cameras are rotated by RT at the same time. The calculation process is as follows:

[0033]

[0034] The image after dedistortion and rotation is restored to the pixel coordinate system, and the coordinates of the image after dedistortion and rotation are calculated. The calculation process is as follows:

[0035]

[0036] Furthermore, the essential matrix decomposition optimizes the extrinsic parameters. The essential matrix E between the corresponding points of the left and right view images is only related to the extrinsic parameters between the left and right cameras. After Bouguet correction, the image coordinates of the corresponding corner points of the corrected left and right view images are calculated to optimize the extrinsic parameters. The rotation and translation relationship of the left and right cameras after correction is decomposed from the essential matrix E. The essential matrix E is related to the rotation R E and pan T E The relationship is as follows:

[0037] E=[T E ] × R E

[0038] In the formula, [T E ] × T E The cross-product matrix of the essential matrix E is decomposed into singular values:

[0039]

[0040] Define two matrices as follows:

[0041]

[0042] And the essential matrix E is expressed as the following two forms:

[0043] E=UZU T UWV T , let [t′] × =UZU T , R′=UWV T

[0044] E=-UZU T UW T V T , let [t′] × =-UZU T , R′=UWV T

[0045] The essential matrix E is decomposed by SVD to obtain four pairs of R E and T E , select the solution with three-dimensional meaning; superimpose the decomposed rotation matrix and translation matrix into the original external parameters; set the rotation matrix before distortion removal to R, and the translation vector to T = (t x , t y ,t z ) T ; The rotation matrix calculated by the above formula is R E , the translation vector is T E =(t Ex , t Ey , t Ez ) T , then Rnew and Tnew satisfy the following relationship:

[0046] R new =RR E

[0047] T temp =(R 1 / 2 ) -1 T E

[0048]

[0049] Furthermore, the visible light and infrared camera calibration algorithm based on feature point extraction includes the following steps:

[0050] S2-1, obtaining the overlapped portion of the visible light edge image and the infrared edge image, and performing scale-invariant feature conversion on the overlapped portion of the visible light edge image and the infrared edge image according to a scale-invariant feature conversion algorithm, to obtain feature point description information corresponding to the visible light edge image and feature point description information corresponding to the infrared edge image;

[0051] S2-2, determining matching point pairs according to the feature point description information corresponding to the visible light edge image obtained after conversion and the feature point description information corresponding to the infrared edge image; each set of matching point pairs consists of a point in the visible light edge image and a point in the infrared edge image, and the two points have a common direction, scale and position;

[0052] S2-3. An external parameter matrix is ​​calculated by combining at least four sets of matching point pairs, an internal parameter matrix of a visible light camera, and an internal parameter matrix of an infrared camera and using an image transformation model of motion parameters of the camera.

[0053] Furthermore, in S2-1, a scale-invariant feature transformation algorithm is used to detect and describe local features in an image; extreme points are found in the spatial scale as feature points, and the position, scale and rotation invariant of the feature points are extracted; the following steps are included:

[0054] S3-1, construct a scale space, use the Gaussian kernel function to scale the original image, obtain the scale representation sequence of the image at multiple scales, and extract features in this scale space to obtain feature points;

[0055] S3-2, accurate positioning of feature points, by fitting a three-dimensional quadratic function, determining the position of feature points and the scale to achieve sub-pixel accuracy, and eliminating candidate feature points with low contrast;

[0056] S3-3, determine the main direction of the feature point, and use the gradient direction distribution characteristics of the pixels in the neighborhood of the extreme point of the Gaussian scale space of the image to specify a direction parameter for each feature point, so that the operator has rotation invariance;

[0057] S3-4. Generate a feature vector descriptor. Using the size, position, and direction information of each feature point, rotate the coordinate axis of the plane coordinate system to the main direction of the feature point to ensure rotation invariance. Calculate the vector information of each feature point to obtain a 128-dimensional SIFT feature vector for each feature point to form the description information of the feature point.

[0058] Furthermore, in S2-3, the image transformation model of the motion parameters of the camera is as follows:

[0059]

[0060] Where K1 is the intrinsic parameter matrix of infrared camera C1, K2 is the intrinsic parameter matrix of visible light camera C2, R represents the rotation matrix of camera C2 relative to camera C1; T represents the displacement vector of camera C2 relative to camera C1, [RT] is the extrinsic parameter matrix, Zc1 and Zc2 represent the distances from point P to the image planes of cameras C1 and C2 respectively;

[0061]

[0062] Wherein, fcx1 is the scale factor of the u-axis direction in the image plane of the infrared camera C1, fcy1 is the scale factor of the v-axis direction in the image plane of the infrared camera C1, and (cx1, cy1) are the coordinates of the center point of the image plane of the infrared camera;

[0063] Substitute the K1 and K2 formulas into the image transformation model of the camera's motion parameters, solve R and T according to the linear equations formed by the coordinate points of the four sets of matching point pairs, and orthogonalize each element of R to obtain the external parameter matrix.

[0064] Furthermore, in S3-1, the scale-variable Gaussian function is:

[0065]

[0066] In the formula, σ represents the variance of the Gaussian normal distribution;

[0067] Assuming I(x, y) as the input image and G(x, y, σ) as the scalable Gaussian function, the scale space of an image is defined as the function L(x, y, σ):

[0068] L(x,y,σ)=G(x,y,σ)×I(x,y)

[0069] Where L represents the scale space of image I, (x, y) represents a point on image I, and σ is the scale factor;

[0070] The Gaussian difference function D(x, y, σ) is calculated by the difference of two adjacent scale images with a constant multiplication factor k:

[0071] D(x,y,σ)=G(x,y,kσ)-G(x,y,σ)×I(x,y)

[0072] =L(x,y,kσ)-L(x,y,σ)

[0073] Beneficial effects: The multi-target, long-distance field temperature measurement method based on dual-spectrum imaging of the present invention proposes to combine the advantages of infrared thermal imaging technology with visible light images, and then use them for long-distance and large-scale field temperature measurement through the registration and fusion of infrared images and visible light images, which can effectively solve the disadvantages of high cost and low efficiency of temperature anomaly detection of traditional substation equipment, and can realize high-precision, long-distance and high real-time temperature detection; the dual-spectrum long-distance fusion long-distance field temperature measurement system optimizes the structure of temperature anomaly detection in existing power systems, greatly reduces labor costs, and realizes unmanned distribution rooms. At the same time, the system's intelligent data transmission mechanism avoids waste of resources and reduces operating costs; the substation equipment field area is huge and the environment is complex. This project realizes accurate identification and temperature monitoring of substation equipment based on image recognition and matching technology based on deep learning and feature engineering, effectively improves the accuracy of temperature detection of power equipment, and realizes accurate analysis and early warning of power equipment failures. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 This is a block diagram of a multi-target, long-distance field temperature measurement method based on dual-spectrum imaging;

[0075] Figure 2 Improve the algorithm for the adopted YOLOv5 network framework structure;

[0076] Figure 3 Calibration flow chart for binocular visible light camera;

[0077] Figure 4 This is a flow chart for visible light and infrared camera calibration based on feature point extraction;

[0078] Figure 5 This is the principle diagram of sub-pixel corner point detection;

[0079] Figure 6 It is the flow chart of corner detection algorithm based on homography matrix;

[0080] Figure 7 is the original chessboard image;

[0081] Figure 8 is a checkerboard binary image;

[0082] Fig. 9 Schematic diagram of the outward expansion of the homography matrix;

[0083] Fig.10 The sub-pixel corner detection result diagram for the left and right views;

[0084] Fig.11 Schematic diagram of binocular correction. DETAILED DESCRIPTION

[0085] The present invention will be further described below in conjunction with the accompanying drawings.

[0086] like Figure 1 As shown, a multi-target, long-distance field temperature measurement method based on dual-spectrum imaging includes a field temperature measurement device, the field temperature measurement device includes a visible light camera module and an infrared camera module; the visible light camera module and the infrared camera module respectively collect visible light images and infrared images of the same power equipment; the visible light camera module includes two visible light cameras, and the infrared camera module includes an infrared camera, including the following steps:

[0087] S1. Use infrared corner detection algorithm, binocular visible light camera calibration algorithm and visible light and infrared camera calibration algorithm based on feature point extraction to obtain the internal and external parameters in the visible light camera module and the infrared camera module, and optimize them; for building a dual-spectrum temperature acquisition system, solving the internal parameters of each camera and the external parameters between the two cameras through calibration is a crucial link, which affects the accuracy of subsequent tasks of the entire visual acquisition system; both visible light cameras and infrared cameras need to be calibrated before use to obtain the corresponding internal and external parameters of the camera. The calibration of visible light cameras and infrared cameras is achieved through three technical studies: infrared corner detection algorithm, binocular visible light camera calibration repetition, and visible light and infrared camera calibration algorithm based on feature point extraction.

[0088] S2, image recognition and matching algorithm based on deep learning and feature engineering, quickly searches for image features and performs stereo matching of image features on the collected visible light images and infrared images, and realizes recognition and positioning during image measurement;

[0089] The image recognition and matching algorithm based on deep learning images and feature engineering combines the advantages of deep learning and feature engineering to solve the technical problems of image measurement and positioning; image recognition and positioning technology has always been the core of passive detection technology, non-full-size image matching technology is one of the key technologies, and fast search and image feature stereo matching are the basis of image positioning technology; image recognition and matching technology specifically includes three aspects: backbone feature extraction network, visual attention CBAM module, and loss function.

[0090] like Figure 2 As shown in the figure, the YOLOv5 network framework is used to realize target detection and recognition, and the feature extraction ability of the model is improved by changing the traditional backbone network and adding the visual attention CBAM module. CSL (Circular SmoothLabel) is used to convert the regression problem into a classification problem to avoid boundary conditions. Finally, data enhancement is performed to improve the accuracy of model detection, so as to improve the speed and accuracy of image target detection. Figure 2As shown, the algorithm improved by the YOLOv5 network framework structure is used as the network framework of the image recognition and matching algorithm based on deep learning and feature engineering. The image recognition and matching algorithm based on deep learning and feature engineering is improved by using the YOLOv5 network framework, including a backbone feature extraction network, a Neck network and a Prediction network; the visual attention CBAM module is introduced into the Neck network to extract features from the image.

[0091] S3. The image fusion algorithm based on IHS transform and wavelet transform is combined with the fusion rules of high and low frequency parts to obtain the dual-spectrum image fusion algorithm. The dual-spectrum image fusion algorithm registers and fuses the visible light image and the infrared image to obtain the fused image; dual-spectrum image registration and fusion refers to the fusion of the visible light image and the infrared image of the same source target, which is the most important step for the temperature measurement system to realize the temperature measurement function; it is necessary to unify the registration of infrared and visible light images to realize the registration and fusion of images; give full play to the thermal sensitivity of infrared cameras and the high resolution of white light cameras to solve the practical problem of high-resolution image recognition in complex operation scenes of substations. Under the same field of view, white light has a resolution of 2 million pixels and infrared has a resolution of only 300,000 pixels. The resolution difference between the two is about 7 times. Through positioning, compounding, filling, smoothing and other image algorithms, the high-definition fused image can be accurately characterized without distortion; for this purpose, it will be mainly combined from the two aspects of the image fusion algorithm principle based on IHS transform and wavelet transform and the fusion rules of high and low frequency parts.

[0092] S4, the image-based high-precision intersection measurement and positioning algorithm and the temperature measurement area stereo matching algorithm perform image intersection measurement and positioning and temperature measurement area stereo matching on the image processed in S2; the temperature infrared data attenuates in the medium with changes in factors such as distance, so compensation is required; for the realization of long-distance non-contact and accurate temperature measurement, the implementation of high-precision intersection measurement and positioning technology based on images is a prerequisite for accurate temperature measurement. The advantages of multi-spectrum are fully utilized through the intersection measurement and positioning technology. In the implementation process, the image intersection measurement technology and the temperature measurement area stereo matching technology are mainly used.

[0093] Image spatial positioning technology is an important research branch of computer vision analysis. Compared with traditional measurement methods, it has two biggest advantages: non-contact and passive. Through the accuracy of multi-camera stereo matching and calibration technology, the accurate spatial positioning of the target can be achieved. The image depth obtained by stereo matching technology is the basis of binocular stereo vision. The core of image stereo matching technology is to determine any pixel in the left or right image. How to efficiently and accurately find this pixel in the right or left image is a key problem. When solving this problem, the following constraints are given to the left and right images: epipolar constraint, uniqueness constraint, disparity continuity constraint, sequential consistency constraint, similarity constraint, smoothness constraint, left-right consistency constraint, and disparity range constraint. The goal to be achieved is: the error between the disparity map generated with the left image as the reference and the disparity map generated with the right image as the reference is the smallest. Based on the above constraints and goals, an improved local stereo matching algorithm is used to improve the matching accuracy. Specifically, the image is pre-processed with guided filtering to enhance its edge information, increase the matching cost calculation information, and increase the window's inclusion range of homogeneous pixels, thereby improving the accuracy of the disparity value. The double helix search path method combining the CIELab color space information and gray value information of the preprocessed image is used to obtain a special-shaped adaptive matching window that has good fit to the complex image boundary, strong descriptiveness, and can effectively improve the matching accuracy.

[0094] S5, using a data-based intelligent analysis algorithm, and performing system edge computing based on the fused image in S3 and the processed image in S4, analyzing the temperature change trend based on the acquired image data to obtain the temperature of the object under test; and evaluating the current operating status and operating status of the tested power equipment to diagnose the current power equipment fault;

[0095] The detection data is stored in the database and uploaded to the server. The edge computing of the system can analyze the temperature change trend based on the detection data, judge the current operating status of the tested equipment, and evaluate the operating status. In order to better evaluate the current operating status of substation equipment, the Bayesian theoretical method is used to establish a Bayesian model evaluation system; the operating status of power equipment in the power system is evaluated through three aspects: traditional status evaluation and discrimination method, Bayesian evaluation system, and Bayesian evaluation method for diagnosing equipment failures.

[0096] The purpose of the dual-spectrum remote field temperature measurement system can be to detect fault warnings for substation equipment and analyze power equipment faults based on the detected temperature of the power equipment; there are many methods for power equipment fault diagnosis, such as spectrum analysis method, parameter model method, pattern recognition method, vibration and acoustic analysis method, etc. These methods have their own characteristics and are suitable for different occasions, but in general, these methods are complex and require more information and more complex mathematical processing. Using the three-level state evaluation method, there is a general fault state between the normal and faulty operating states. This operating state will have a certain impact on the equipment, but it will not cause great harm, and it can continue to operate on the basis of enhanced detection; the introduction of the general fault state makes it necessary to diagnose the equipment based on temperature information based on the probability of the equipment being in three states, and use the maximum probability membership principle to judge whether the equipment is faulty.

[0097] S6. Upload the temperature of the object under test, the operating status of the object under test and the result data of the fault obtained in S5 to the application end, such as the fault of the power equipment and the related result data of the object under test and the power equipment to the application end, and the application end stores, displays and processes the test results; the application end can be a PC; after completing several detection and processing technologies involved in dual-spectrum fusion, design databases, communication protocols, etc. to realize the development of application end software and web version. The monitoring software of the temperature measurement system station application end is used to realize equipment management, target acquisition, target data acquisition and storage, target positioning, historical data query, alarm data display and storage, test result display, test data upload and other functions; the field temperature of the power equipment detected is displayed on the display screen, and the type and location of the power equipment are marked; and the power equipment detected by the data-based intelligent analysis algorithm is displayed whether it is faulty. If a fault occurs, an alarm is processed to notify the personnel to carry out maintenance.

[0098] The application software of the dual-spectrum remote field temperature measurement system is the PC-side detection software, and its development content includes the development of temperature measurement equipment management function, measured target management function, database function, alarm detection and management function, network communication function, etc. At the same time, Ethernet is used to transmit regional images and monitoring data, and web-side software development is carried out to realize multi-platform remote network sharing of equipment data. For system-side software development, a modular design scheme is adopted; modular design is one of the main manifestations of the openness of the system architecture, which can decompose the originally complex problems into many simpler local problems in order to simplify the development and debugging of the program. In view of the goals to be achieved by the dual-spectrum remote field temperature measurement system, a systematic module design scheme was first carried out. The dual-spectrum remote field temperature measurement software-side system contains four modules, namely online monitoring module, control module, image processing module and auxiliary diagnosis module.

[0099] The multi-target, long-distance power field temperature measurement system based on dual-spectrum imaging mainly includes the calibration of visible light camera modules and infrared camera modules, the recognition and deep matching of substation equipment images collected by the camera system, the registration and fusion of visible light images and infrared images, the intersection measurement and positioning of the temperature measurement area images, and the analysis and evaluation of the temperature data after the temperature data is collected to analyze the fault status of the entire power equipment. Finally, it is necessary to develop the application software of the multi-target, long-distance power field temperature measurement system based on dual-spectrum imaging. The multi-target, long-distance power field temperature measurement method based on dual-spectrum imaging is based on computer vision, deep learning, image recognition and positioning technology, and dual-spectrum fusion technology to perform image fusion and registration, target temperature extraction in the area to be detected, temperature characteristic analysis and temperature compensation, and abnormal high temperature extraction. Through a set of multi-spectral monitoring cameras that fully cover the substation and the outdoor equipment area of ​​the substation, the abnormal heating fault identification, early warning, and rapid and accurate positioning of abnormal heating points of many equipment in the substation are solved. The development of a multi-target, long-distance power field temperature measurement system based on dual-spectrum imaging mainly includes calibration of visible light camera modules and infrared camera modules, recognition and depth matching of substation equipment images collected by the camera system, registration and fusion of visible light images and infrared images, intersection measurement and positioning of temperature measurement area images, analysis and evaluation of temperature data after collection to analyze the fault status of the entire power equipment, and finally development of application-side software for a multi-target, long-distance power field temperature measurement system based on dual-spectrum imaging.

[0100] For building a multispectral acquisition system, solving the internal parameters of each camera and the external parameters between two cameras through calibration is a crucial step, which affects the accuracy of subsequent tasks of the entire visual acquisition system. Both visible light cameras and infrared cameras need to be calibrated before use to obtain the corresponding internal and external parameters of the camera; for this purpose, infrared corner point detection algorithm, binocular visible light camera calibration algorithm and visible light and infrared camera calibration algorithm based on feature point extraction are used to obtain and optimize the internal and external parameters.

[0101] like Figure 6 As shown, in S1, in the infrared corner point detection algorithm, in order to overcome the shortcomings of the current checkerboard corner point detection technology, a method for adaptively extending corner point detection based on a homography matrix is ​​studied; one or more colored or shaped markers are set on the collected checkerboard image to mark the initial position, and the homography matrix H obtained by using the initial pixel coordinates of the unit grid in the pixel coordinate system and the world coordinates set in the world coordinate system is expanded outward to the entire checkerboard area to complete the corner point detection; the following steps are included:

[0102] S1-1. Set one or more colored or shaped markers on the checkerboard calibration board to mark the initial position; the position characteristics of the sub-pixel corner point are: the corner point is the intersection of the edge, and the gradient direction of the corner point and the edge point is perpendicular; Figure 5 As shown, according to the characteristic of "vertical vector, product is 0" near the sub-pixel corner point, the sub-pixel corner point to be found is set as q, and the point in its vicinity is set as p i The connecting vector to q is p i -q;p i Inside the uniform region, point p i The gradient of p is 0; i On the edge, then point p i The gradient direction at p is perpendicular to i -q; for any point pi in the image, the grayscale gradient at pi is The following formula is obtained:

[0103]

[0104] Using the least squares method to solve the above equation, we get:

[0105]

[0106] The value of the q point is selected so that the error ε i The coordinates of the smallest pixel point, within the area centered on the corner point q, have an error sum of S, and the calculation process is as follows:

[0107]

[0108] Finding the exact position of the corner point is transformed into finding the point that minimizes the error and S. Using iterative optimization to solve, we get:

[0109]

[0110] S1-2, using the infrared camera module to collect the chessboard calibration plate image and perform preliminary image processing;

[0111] S1-3, detecting the pixel coordinates of the pre-set mark by morphological or color detection algorithm;

[0112] S1-4, according to the pixel coordinates of the marker detected in step 3, determine the pixel coordinates of the four points of the initial grid, and calculate the corresponding sub-pixel corner point coordinates through iteration;

[0113] S1-5, according to the sub-pixel corner coordinates of the four points and the world coordinates of the four points in the world coordinate system, calculate the homography matrix H;

[0114] S1-6. Multiply the homography matrix H and the world coordinates of the adjacent grid in the world coordinate system to obtain the starting rough sub-pixel corner points of the other two unknown points of the adjacent grid, and calculate the coordinates of all sub-pixel corner points in the image based on the sub-pixel corner point solution principle.

[0115] Before detection, you first need to set one or more colored or shaped markers on the chessboard to mark the initial position; take the circle used on the infrared calibration plate as an example:

[0116] First, five circles are distributed in the five grids in the central area of ​​the 3×3 chessboard, with a diameter slightly smaller than the side length of the unit grid. The middle circle is numbered C0, and the other four circles are numbered C1, C2, C3, and C4;

[0117] Use an infrared camera to shoot the infrared calibration plate until the corner points of the calibration plate can cover the entire imaging area of ​​the camera, and then preprocess the acquired checkerboard image. Preprocessing includes binarization commonly used in the field of image processing. The processed image will be used in the subsequent step of the checkerboard corner robustness verification algorithm. Since there are scenes other than the calibration plate pattern in the image, the direct use of the maximum inter-class variance method is not very ideal; therefore, the block binarization method is used to divide the image I into M×N blocks, each of which is named Ii; the maximum inter-class variance method is used for each block Ii to obtain the binary image binary_I. The original checkerboard image is as follows Figure 7 As shown, the checkerboard binary image is Figure 8 As shown in the figure, the binarization effect of the checkerboard corner area of ​​the image is very good, and the checkerboard texture is very clear.

[0118] The collected chessboard original image is detected by Hough circle detection through the imfindcircles function in MATLAB, and five circles in the image are detected and their pixel coordinates are recorded; the midpoints of the four lines connecting the four surrounding circles C1, C2, C3, C4 and the middle circle numbered C0 are obtained, numbered P1, P2, P3, P4, and these four points are used as the initial four corner points. According to the principle of sub-pixel corner point solution, the four sub-pixel corner points p1, p2, p3, p4 of the initial grid in the pixel coordinate system are iteratively solved. One of the four corner points of the initial grid is used as the origin of the world coordinate system. According to the known chessboard size and the custom world coordinate system origin, the points on the chessboard are assigned one by one in the world coordinate system, and all corner points on the chessboard obtain corresponding world coordinates.

[0119] After setting the markers, taking and processing the images, detecting the markers, and using the markers, the initial grid can be expanded outward according to the homography matrix H and spread to the entire checkerboard area. According to the camera imaging principle, the pixel coordinates of the projection point p of a point W (Xw, Yw, Zw) in the world coordinate system in the pixel coordinate system are (u, v), which is:

[0120]

[0121] Where Zc is the Z-axis coordinate of point W in the camera coordinate system, u0 is the image center coordinate in the x direction, v0 is the image center coordinate in the y direction, f is the focal length, R is the rotation matrix, T is the translation vector, fx is the scale factor of the u axis, fy is the scale factor of the v axis, dx is the pixel size in the x direction, and dy is the pixel size in the y direction;

[0122] Generally, let Z = 0 for points on the plane of the chessboard calibration plate, and we can get:

[0123]

[0124] Where r1, r2, and r3 are the three column vectors of the rotation matrix R, and t is the translation vector in the x and y directions.

[0125] By Z c P′=HW′, let

[0126]

[0127] Then the large linear matrix H is obtained:

[0128] H=A[r1 r2 t]

[0129] Therefore, the homography matrix H can be calculated based on the coordinates of the four points in the world coordinate system and the pixel-level coordinates of these four points.

[0130] like Fig. 9 As shown in the figure, using the homography matrix H of the initial four points and the coordinates of a pair of adjacent grid points p5 and p6 in the world coordinate system, the pixel coordinates P5 and P6 of the next two points can be obtained, and then according to the principle of sub-pixel corner point solution, new sub-pixel corner points p5 and p6 are iteratively solved. All sub-pixel corner points on the chessboard are expanded outward from the small square determined by the four points in the four directions of the positive X axis, the negative X axis, the positive Y axis, and the negative Y axis.

[0131] like Fig.10The following are the results of sub-pixel corner detection for the left and right images. This algorithm realizes the fully automatic detection of sub-pixel corners in the checkerboard image. The obtained sub-pixel corner coordinates can be directly used for solving the camera's intrinsic and extrinsic parameters. Compared with the Open CV function and the calibration toolbox that comes with MATLAB, this corner detection method has the following advantages:

[0132] The fully automatic detection of sub-pixel corners of chessboard images is realized, and the obtained sub-pixel corner coordinates can be directly used for the camera's internal and external parameter solutions. Compared with the current mainstream chessboard corner detection method, this method does not require manual intervention, and only requires the program to read the image to complete the fully automatic corner detection. The corner detection speed is fast and will not be affected by the incomplete chessboard. The collected images only need to capture the manually set marks. For complex infrared scenes or when the chessboard surface is contaminated, this method is very robust and can accurately detect sub-pixel corners on the chessboard.

[0133] In order to realize the fully automatic calibration method of binocular camera, a binocular camera calibration system was built. The system consists of a pure calibration board, a binocular stereo vision module, and a PC host computer. The binocular stereo vision module contains two monocular cameras of the same model, a camera link bus, an image acquisition card, and a power supply for the two cameras. The PC host computer contains a binocular camera calibration module, an infrared binocular camera correction module, and a parallel equipotential verification module.

[0134] like Figure 3 As shown in the binocular visible light camera calibration algorithm, binocular correction is to eliminate distortion and align the collected left and right view images based on the monocular internal parameter data obtained after camera calibration: focal length, imaging origin, distortion coefficient, and binocular relative position relationship: rotation matrix and translation vector, so that the imaging origin coordinates of the left and right view images are consistent, the optical axes of the two cameras are parallel, the left and right imaging planes are coplanar, and the epipolar lines are aligned, as shown in Fig.11 As shown in the figure, it is a schematic diagram of binocular correction. In order to achieve this goal, a binocular extrinsic parameter optimization method based on the essential matrix is ​​adopted. The optimization of the binocular camera extrinsic parameters is realized by performing Bouguet correction on the corresponding corner points on the original image and optimizing the extrinsic parameters by essential matrix decomposition.

[0135] The Bouguet correction is performed on the corresponding corner points on the collected original image, including finding the common corner point pairs (p li ,p ri ) corresponds to the coordinate pair (P li ,P ri ); the common corner point pairs on the left and right view images (p li ,p ri) are the sub-pixel corner coordinates of the left and right view images; set point p(u, v) as the pixel coordinate on the image, and P(X, Y) as the corresponding coordinate in the normalized coordinate system;

[0136] p=KP

[0137]

[0138] When the pixel point p and the camera's intrinsic parameter K are known, the corresponding coordinate P in the regular coordinate system is calculated. The calculation process is as follows:

[0139] P=K -1 p

[0140] For each pair of left and right camera corner points (p li ,p ri ), its normal coordinate system coordinate pair (P li ,P ri )for:

[0141]

[0142] Get the normal coordinate system coordinate pair of the left and right view images (P li ,P ri ), dedistortion processing is performed; taking (x, y) as the original position of the distortion point, the ideal coordinates (xcorrected, ycorrected) are obtained through several nonlinear iterations.

[0143] The left and right view images are rotated according to the rotation relationship between the two cameras. It is known that the rotation matrix R and the translation vector T between the left and right view images satisfy the following relationship:

[0144] P li =RP ri +T

[0145] Where P li is the coordinate in the normalized coordinate system of the left camera, P ri is the coordinate in the normalized coordinate system of the right camera;

[0146] The left image is rotated by half the angle in the positive direction of R, and the right image is rotated by half the angle in the negative direction of R, so that the overlap area of ​​the left and right cameras is maximized after rotation and the impact on the reprojection error is minimized; the rotation angle RT is calculated according to the translation relationship T between the two cameras, and the two cameras are rotated by RT at the same time. The calculation process is as follows:

[0147]

[0148] The image after dedistortion and rotation is restored to the pixel coordinate system, and the coordinates of the image after dedistortion and rotation are calculated. The calculation process is as follows:

[0149]

[0150] The essential matrix decomposition optimizes the external parameters. The essential matrix E between the corresponding points of the left and right view images is only related to the external parameters between the left and right cameras. After Bouguet correction, the image coordinates of the corresponding corner points of the corrected left and right view images are calculated to optimize the external parameters. The rotation and translation relationship of the left and right cameras after correction is decomposed from the essential matrix E. The essential matrix E is related to the rotation R E and pan T E The relationship is as follows:

[0151] E=[T E ] × R E

[0152] In the formula, [T E ] × T E The cross-product matrix of the essential matrix E is decomposed into singular values:

[0153]

[0154] Define two matrices as follows:

[0155]

[0156] And the essential matrix E is expressed as the following two forms:

[0157] E=UZU T UWV T , let [t′] × =UZU T , R′=UWV T

[0158] E=-UZU T UW T V T , let [t′] × =-UZU T , R′=UWV T

[0159] The essential matrix E is decomposed by SVD to obtain four pairs of R E and TE, select the solution with three-dimensional meaning; superimpose the decomposed rotation matrix and translation matrix into the original external parameters; set the rotation matrix before distortion removal to R, and the translation vector to T = (t x , t y , t z ) T ; The rotation matrix calculated by the above formula is R E , the translation vector is TE =(t Ex , tE y ,t Ez ) T , then Rnew and Tnew satisfy the following relationship:

[0160] R new =RR E

[0161] T temp =(R 1 / 2 ) -1 T E

[0162]

[0163] like Figure 4 As shown, the visible light and infrared camera calibration algorithm based on feature point extraction includes the following steps:

[0164] S2-1, obtaining the overlapped portion of the visible light edge image and the infrared edge image, and performing scale-invariant feature conversion on the overlapped portion of the visible light edge image and the infrared edge image according to a scale-invariant feature conversion algorithm, to obtain feature point description information corresponding to the visible light edge image and feature point description information corresponding to the infrared edge image;

[0165] S2-2, determining matching point pairs according to the feature point description information corresponding to the visible light edge image obtained after conversion and the feature point description information corresponding to the infrared edge image; each set of matching point pairs consists of a point in the visible light edge image and a point in the infrared edge image, and the two points have a common direction, scale and position;

[0166] S2-3. An external parameter matrix is ​​calculated by combining at least four sets of matching point pairs, an internal parameter matrix of a visible light camera, and an internal parameter matrix of an infrared camera and using an image transformation model of motion parameters of the camera.

[0167] In S2-1, a scale-invariant feature conversion algorithm is used to detect and describe local features in an image; extreme points are found in the spatial scale as feature points, and the position, scale and rotation invariant of the feature points are extracted; the following steps are included:

[0168] S3-1, constructing a scale space, using a Gaussian kernel function to scale the original image, obtaining a scale representation sequence of the image at multiple scales, and extracting features in this scale space to obtain feature points; in S3-1, the scale-variable Gaussian function is:

[0169]

[0170] In the formula, σ represents the variance of the Gaussian normal distribution;

[0171] Assuming I(x, y) as the input image and G(x, y, σ) as the scalable Gaussian function, the scale space of an image is defined as the function L(x, y, σ):

[0172] L(x,y,σ)=G(x,y,σ)×I(x,y)

[0173] Where L represents the scale space of image I, (x, y) represents a point on image I, and σ is the scale factor;

[0174] The Gaussian difference function D(x, y, σ) is calculated by the difference of two adjacent scale images with a constant multiplication factor k:

[0175] D(x,y,σ)=G(x,y,kσ)-G(x,y,σ)×I(x,y)

[0176] =L(x,y,kσ)-L(x,y,σ)

[0177] S3-2. Accurately locate the feature points by fitting a three-dimensional quadratic function to determine the position of the feature points and the scale to achieve sub-pixel accuracy, and eliminate candidate feature points with low contrast; in addition, the DoG operator will produce a strong edge response, so it is necessary to remove unstable edge response points.

[0178] S3-3, determine the main direction of the feature point, use the gradient direction distribution characteristics of the pixels in the neighborhood of the extreme point in the Gaussian scale space of the image to specify the direction parameter for each feature point, so that the operator has rotation invariance; calculate the local image gradient angle and amplitude centered on the extreme point, the amplitude is the modulus, and the calculation formula is as follows:

[0179]

[0180] In the formula, the scale used by L is the scale of each key point; sampling is performed in the neighborhood window centered on the feature point, and the gradient direction of the neighborhood pixels is counted using the histogram. The peak of the histogram corresponds to the main direction of the feature point. In the gradient direction histogram, if there is a peak with an energy equivalent to 80% of the main peak, it is used as the auxiliary direction of the feature point.

[0181] S3-4. Generate a feature vector descriptor. Using the size, position, and direction information of each feature point, rotate the coordinate axis of the plane coordinate system to the main direction of the feature point to ensure rotation invariance. Calculate the vector information of each feature point to obtain a 128-dimensional SIFT feature vector for each feature point to form the description information of the feature point.

[0182] Take an 8×8 window centered on the feature point and divide it into 16 2×2 sub-areas; calculate the gradient direction histogram in 8 directions on each 2×2 small block, and draw the accumulated value of each gradient direction to form a seed point; in the 8×8 window, a feature point is composed of 4×4, a total of 16 seed points, and each seed point has 8 direction vector information, so a feature point corresponds to 16×8, a total of 128 data, that is, a 128-dimensional SIFT feature vector is finally formed. These feature vectors are used to distinguish feature point pairs, and the corresponding feature vectors form the description information of the corresponding feature points.

[0183] In S2-3, the image transformation model of the motion parameters of the camera is as follows:

[0184]

[0185] Where K1 is the intrinsic parameter matrix of infrared camera C1, K2 is the intrinsic parameter matrix of visible light camera C2, R represents the rotation matrix of camera C2 relative to camera C1; T represents the displacement vector of camera C2 relative to camera C1, [RT] is the extrinsic parameter matrix, Zc1 and Zc2 represent the distances from point P to the image planes of cameras C1 and C2 respectively;

[0186]

[0187] Wherein, fcx1 is the scale factor of the u-axis direction in the image plane of the infrared camera C1, fcy1 is the scale factor of the v-axis direction in the image plane of the infrared camera C1, and (cx1, cy1) are the coordinates of the center point of the image plane of the infrared camera;

[0188] Substitute the K1 and K2 formulas into the image transformation model of the camera's motion parameters, and solve R and T according to the linear equations formed by the coordinate points of the four sets of matching point pairs; and orthogonalize each element of R to obtain the extrinsic parameter matrix. When the focal length of the camera changes, such as a visible light camera or an infrared camera, the extrinsic parameter matrix can be determined without adjusting the camera position, which improves the efficiency of parameter calibration.

[0189] The above is only a description of the preferred embodiments of the present invention. Ordinary technicians in this technical field can make several modifications and optimizations based on the above disclosure without departing from the above basic principles. These improvements and optimizations should be regarded as the understood protection scope of the present invention.

Claims

1. A multi-target, long-distance field temperature measurement method based on dual-spectrum imaging, characterized in that: The method comprises a field temperature measuring device, wherein the field temperature measuring device comprises a visible light camera module and an infrared camera module; and comprises the following steps: S1. Using infrared corner detection algorithm, binocular visible light camera calibration algorithm and visible light and infrared camera calibration algorithm based on feature point extraction, obtain the internal and external parameters in the visible light camera module and infrared camera module, and optimize them; S2, image recognition and matching algorithm based on deep learning and feature engineering, which can quickly search for image features and stereo matching of image features of collected visible light images and infrared images, and realize recognition and positioning during image measurement; S3, using an image fusion algorithm based on IHS transform and wavelet transform and combining the high- and low-frequency part fusion rules to obtain a dual-spectrum image fusion algorithm, the dual-spectrum image fusion algorithm aligns and fuses the visible light image and the infrared image to obtain a fused image; S4, performing image intersection measurement and positioning and temperature measurement area stereo matching on the image processed in S2 based on a high-precision intersection measurement and positioning algorithm and a temperature measurement area stereo matching algorithm; S5, using a data-based intelligent analysis algorithm, and performing system edge computing based on the fused image in S3 and the processed image in S4, analyzing the temperature change trend based on the acquired image data to obtain the temperature of the object under test; and evaluating the current operating status and the status of the operating status of the object under test, and diagnosing the current fault of the object under test; S6. Upload the temperature of the object under test, the operating status of the object under test, and the fault result data obtained in S5 to the application end, and the application end stores, displays, and processes the detection results.

2. The multi-target, long-distance field temperature measurement method based on dual-spectrum imaging according to claim 1 is characterized in that: In S1, in the infrared corner point detection algorithm, one or more colored or shaped markers are set on the collected checkerboard image to mark the initial position, and the homography matrix H obtained by using the initial pixel coordinates of the unit grid in the pixel coordinate system and the world coordinates set in the world coordinate system is expanded outward to the entire checkerboard area to complete the corner point detection; including the following steps: S1-1, setting one or more colored or shaped markers on the checkerboard calibration board to mark the initial position; S1-2, using the infrared camera module to collect the chessboard calibration plate image and perform preliminary image processing; S1-3, detecting the pixel coordinates of the pre-set mark by morphological or color detection algorithm; S1-4, according to the pixel coordinates of the marker detected in step 3, determine the pixel coordinates of the four points of the initial grid, and calculate the corresponding sub-pixel corner point coordinates through iteration; S1-5, according to the sub-pixel corner point coordinates of the four points and the world coordinates of the four points in the world coordinate system, calculate the homography matrix H; S1-6. Multiply the homography matrix H and the world coordinates of the adjacent grid in the world coordinate system to obtain the starting rough sub-pixel corner points of the other two unknown points of the adjacent grid, and calculate the coordinates of all sub-pixel corner points in the image based on the sub-pixel corner point solution principle.

3. The multi-target, long-distance field temperature measurement method based on dual-spectrum imaging according to claim 2 is characterized in that: In the binocular visible light camera calibration algorithm, Bouguet correction and essential matrix decomposition are performed on the corresponding corner points on the original image to optimize the external parameters, so that the imaging origin coordinates of the left and right view images are consistent, the camera optical axes of the two cameras are parallel, the left and right imaging planes are coplanar, and the epipolar lines are aligned; The Bouguet correction is performed on the corresponding corner points on the collected original image, including finding the common corner point pairs (p li ,p ri ) corresponds to the coordinate pair (P li ,P ri ); Set point p(u, v) as the pixel coordinate on the image, and P(X, Y) as the corresponding coordinate in the normalized coordinate system; p=KP When the pixel point p and the camera's intrinsic parameter K are known, the corresponding coordinate P in the regular coordinate system is calculated. The calculation process is as follows: P=K -1 p For each pair of left and right camera corner points (p li , p ri ), its normal coordinate system coordinate pair (P li , P ri )for: Get the normal coordinate system coordinate pair of the left and right view images (P li , P ri ) and then perform distortion removal; The left and right view images are rotated according to the rotation relationship between the two cameras. It is known that the rotation matrix R and the translation vector T between the left and right view images satisfy the following relationship: P li =RP ri +T Where P li is the coordinate in the normalized coordinate system of the left camera, P ri is the coordinate in the normalized coordinate system of the right camera; Rotate the left image by half the angle in the positive direction of R, and rotate the right image by half the angle in the negative direction of R, so that the overlap area of ​​the left and right cameras is maximized after rotation and the error of reprojection is minimized; calculate the rotation angle R according to the translation relationship T between the two cameras T , and rotate both cameras simultaneously by R T , the calculation process is as follows: The image after dedistortion and rotation is restored to the pixel coordinate system, and the coordinates of the image after dedistortion and rotation are calculated. The calculation process is as follows:

4. The multi-target, long-distance field temperature measurement method based on dual-spectrum imaging according to claim 3 is characterized in that: The essential matrix decomposition optimizes the external parameters. The essential matrix E between the corresponding points of the left and right view images is only related to the external parameters between the left and right cameras. After Bouguet correction, the image coordinates of the corresponding corner points of the corrected left and right view images are calculated to optimize the external parameters. The rotation and translation relationship of the left and right cameras after correction is decomposed from the essential matrix E. The essential matrix E is related to the rotation R E and pan T E The relationship is as follows: E=[T E ] × R E In the formula, [T E ] × T E The cross-product matrix of the essential matrix E is decomposed into singular values: Define two matrices as follows: ZW=∑ And the essential matrix E is expressed as the following two forms: E = UZU T UWVT, let [t'] × = UZU T , R' = UWV T E = -UZU T UW T V T Let [t′] × = -UZU T and R′ = UWV T The essential matrix E is decomposed by SVD to obtain four pairs of R E and T E , select the solution with three-dimensional meaning; The decomposed rotation matrix and translation matrix are superimposed on the original external parameters; the rotation matrix before distortion removal is set to R, and the translation vector is set to T = (t x ,t y ,t z );The rotation matrix calculated by the above formula is R E , the translation vector is T E =(t Ex ,t Ey ,t Ez ) T , then Rnew and Tnew satisfy the following relationship: R new =RR E T temp =(R 1 / 2 ) -1 T E 5. The multi-target, long-distance field temperature measurement method based on dual-spectrum imaging according to claim 1 is characterized in that: The visible light and infrared camera calibration algorithm based on feature point extraction includes the following steps: S2-1, obtaining the overlapped portion of the visible light edge image and the infrared edge image, and performing scale-invariant feature conversion on the overlapped portion of the visible light edge image and the infrared edge image according to a scale-invariant feature conversion algorithm, to obtain feature point description information corresponding to the visible light edge image and feature point description information corresponding to the infrared edge image; S2-2, determining matching point pairs according to the feature point description information corresponding to the visible light edge image obtained after conversion and the feature point description information corresponding to the infrared edge image; each set of matching point pairs consists of a point in the visible light edge image and a point in the infrared edge image, and the two points have a common direction, scale and position; S2-3. An external parameter matrix is ​​calculated by combining at least four sets of matching point pairs, an internal parameter matrix of a visible light camera, and an internal parameter matrix of an infrared camera and using an image transformation model of motion parameters of the camera.

6. The multi-target, long-distance field temperature measurement method based on dual-spectrum imaging according to claim 5 is characterized in that: In S2-1, a scale-invariant feature conversion algorithm is used to detect and describe local features in an image; extreme points are found in the spatial scale as feature points, and the position, scale and rotation invariant of the feature points are extracted; the following steps are included: S3-1, construct a scale space, use the Gaussian kernel function to scale the original image, obtain the scale representation sequence of the image at multiple scales, and extract features in this scale space to obtain feature points; S3-2, accurate positioning of feature points, by fitting a three-dimensional quadratic function, determining the position of feature points and the scale to achieve sub-pixel accuracy, and eliminating candidate feature points with low contrast; S3-3, determine the main direction of the feature point, and use the gradient direction distribution characteristics of the pixels in the neighborhood of the extreme point of the Gaussian scale space of the image to specify a direction parameter for each feature point, so that the operator has rotation invariance; S3-4. Generate a feature vector descriptor. Using the size, position, and direction information of each feature point, rotate the coordinate axis of the plane coordinate system to the main direction of the feature point to ensure rotation invariance. Calculate the vector information of each feature point to obtain a 128-dimensional SIFT feature vector for each feature point to form the description information of the feature point.

7. The multi-target, long-distance field temperature measurement method based on dual-spectrum imaging according to claim 5 is characterized in that: In S2-3, the image transformation model of the motion parameters of the camera is as follows: Where K1 is the intrinsic parameter matrix of infrared camera C1, K2 is the intrinsic parameter matrix of visible light camera C2, R represents the rotation matrix of camera C2 relative to camera C1; T represents the displacement vector of camera C2 relative to camera C1, [RT] is the extrinsic parameter matrix, Zc1 and Zc2 represent the distances from point P to the image planes of cameras C1 and C2 respectively; Wherein, fcx1 is the scale factor of the u-axis direction in the image plane of the infrared camera C1, fcy1 is the scale factor of the v-axis direction in the image plane of the infrared camera C1, and (cx1, cy1) are the coordinates of the center point of the image plane of the infrared camera; Substitute the K1 and K2 formulas into the image transformation model of the camera's motion parameters, solve R and T according to the linear equations formed by the coordinate points of the four sets of matching point pairs, and orthogonalize each element of R to obtain the external parameter matrix.

8. The multi-target, long-distance field temperature measurement method based on dual-spectrum imaging according to claim 6 is characterized in that: In S3-1, the scale-variable Gaussian function is: In the formula, σ represents the variance of the Gaussian normal distribution; Assuming I(x, y) as the input image and G(x, y, σ) as the scalable Gaussian function, the scale space of an image is defined as the function L(x, y, σ): L(x,y,σ)=G(x,y,σ)×I(x,y) Where L represents the scale space of image I, (x, y) represents a point on image I, and σ is the scale factor; The Gaussian difference function D(x, y, σ) is calculated by the difference of two adjacent scale images with a constant multiplication factor k: D(x,y,σ)=G(x,y,kσ)-G(x,y,σ)×I(x,y) =L(x,y,kσ)-L(x,y,σ).

Images