Binocular ranging temperature compensation method based on fusion of rank image and YCbCr image

Abstract

The invention provides a binocular ranging temperature compensation method based on fusion of a rank image and a YCbCr image. The binocular ranging temperature compensation method comprises the following steps: calibrating and correcting an acquired binocular visible light image and an infrared image; converting the corrected visible light image into a rank image; a YCbCr image is obtained; according to the rank image and the YCbCr image, performing image matching by using a difference function fused by a local feature difference value and a color difference value to obtain a disparity map; obtaining the three-dimensional position of each point by using a triangulation method according to the obtained disparity map and calibration parameters of the camera; identifying the target terminal according to the corrected infrared image to obtain a position area of the target terminal, and extracting temperature information in the identified position area of the target terminal to obtain a measured temperature value of the target terminal; and obtaining a distance value of the target terminal in the infrared image, and compensating the measurement temperature of the target terminal by using the distance value. The accurate distance value of the cable terminal can be obtained, and the interference of distance factors on the temperature value measured by the infrared camera is eliminated.

Description

Technical Field

[0001] The invention relates to the technical field of cable terminal temperature measurement, and in particular to a binocular ranging temperature compensation method based on rank image and YCbCr image fusion. Background Art

[0002] When the electric energy transmission reaches the destination, the cable terminal is the power component that leads the cable from the air and is installed in the substation or on the ground pile, platform or tower. During the operation of the power system, the cable terminal may fail, resulting in abnormal changes in the temperature of the cable terminal. If the cable terminal area with abnormal temperature is not checked in time, the power system may not operate normally, and in serious cases, major safety accidents may occur, causing economic losses. Therefore, real-time monitoring of the cable terminal temperature is one of the most important means to eliminate safety hazards in the power system. The most commonly used cable terminal temperature monitoring method in the power system is to use traditional temperature sensors. However, these traditional sensors often need to be installed inside or outside the cable terminal, which not only brings difficulty to the installation, but also causes certain damage to the equipment itself. Therefore, an infrared camera can be used for temperature measurement. The temperature measured by the infrared camera is closely related to the distance of the target object. Therefore, an effective binocular ranging method is needed to compensate the measured temperature of the infrared camera based on the distance. Summary of the invention

[0003] In order to solve the problem that the temperature measurement method in the prior art cannot meet the needs of practical applications, the purpose of the present invention is to provide a binocular ranging temperature compensation method based on rank image and YCbCr image fusion, which can effectively improve the stability and accuracy of binocular ranging in different environments, thereby realizing temperature compensation.

[0004] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0005] A binocular ranging temperature compensation method based on rank image and YCbCr image fusion includes the following steps:

[0006] Collect binocular visible light images and infrared images, and perform image calibration and correction to obtain the internal and external parameters of the camera and eliminate distortion and parallax;

[0007] converting the rectified visible light image into a rank image;

[0008] Convert the corrected visible light image from RGB space to YCbCr color space to obtain a YCbCr image;

[0009] According to the rank image and the YCbCr image, the disparity map is obtained by performing image matching using a difference function that combines local feature difference values ​​and color difference values;

[0010] According to the obtained disparity map and the calibration parameters of the camera, the three-dimensional position of each point is obtained using triangulation;

[0011] According to the corrected infrared image, the target terminal is identified by using a deep learning model to obtain the location area of ​​the target terminal, and the temperature information is extracted in the identified target terminal location area to obtain the measured temperature value of the target terminal;

[0012] The infrared image is registered based on the binocular visible light image to ensure that the target terminal in the infrared image is consistent with the target terminal in the three-dimensional measurement result in spatial position, the distance value of the target terminal in the infrared image is obtained, and the measured temperature of the target terminal is compensated using the distance value.

[0013] According to a binocular ranging temperature compensation method based on rank image and YCbCr image fusion provided by the present invention, the binocular visible light image and infrared image are collected, and image calibration and correction are performed, including:

[0014] The checkerboard is photographed from multiple angles by a binocular camera, and the corner points of the checkerboard image are detected using a corner detection algorithm to obtain calibrated corner point data; the internal and external parameters of the left and right cameras are calculated using the least squares method; the target area including the cable terminal is photographed by a binocular camera to obtain a left image and a right image including the cable terminal; the correction algorithm is applied to perform distortion correction and epipolar correction on the image including the cable terminal to obtain a corrected image.

[0015] According to a binocular ranging temperature compensation method based on rank image and YCbCr image fusion provided by the present invention, the conversion of the corrected visible light image into a rank image comprises:

[0016] The corrected visible light image is converted into left and right grayscale images; the left grayscale image and the right grayscale image are converted into rank images respectively using the rank transformation algorithm. The rank transformation formula is expressed as the following formula:

[0017]

[0018] Where R(x,y) is the value of (x,y) in the rank image, N is a neighborhood of size m×n centered at (x,y), (x',y') is a point in the neighborhood N, and g(x',y') is the grayscale value of (x',y') in the grayscale image;

[0019] Then save the left rank image and the right rank image.

[0020] According to a binocular ranging temperature compensation method based on rank image and YCbCr image fusion provided by the present invention, the method converts the corrected visible light image from RGB space to YCbCr color space, comprising:

[0021] The corrected visible light image is converted into a YCbCr image by calculating the converted YCbCr image according to the RGB to YCbCr formula, wherein the RGB to YCbCr formula includes:

[0022] Y(x,y)=(0.257*R(x,y)+0.564*G(x,y)+0.098*B+16) / 255

[0023] C b (x,y)=(0.148*R(x,y)-0.291*G(x,y)+0.439*B(x,y)+128) / 255

[0024] C r (x,y)=(0.439*R(x,y)-0.2368*G(x,y)-0.071*B(x,y)+128) / 255

[0025] Among them, Y(x,y), Cb(x,y), and Cr(x,y) are the YCbCr three-channel values ​​of (x,y), and R(x,y), G(x,y), and B(x,y) are the RGB three-channel values ​​of (x,y).

[0026] According to a binocular ranging temperature compensation method based on rank image and YCbCr image fusion provided by the present invention, the image matching is performed using a difference function fused with a local feature difference value and a color difference value, comprising:

[0027] Select the left rank image (x, y) as the first base point, and extract a 1*11 region S centered on the coordinates of the first base point in the left rank image and the left YCbCr image respectively. LR and S LY ; Select the right image (x+d,y) as the second base point, and extract the 1*11 region S centered on the coordinates of the second base point in the right rank image and the right YCbCr image respectively RR and S RY ;

[0028] Then, the local feature difference value is calculated, and the formula is:

[0029]

[0030] Among them, C rank (x, y, d) is the local feature difference value, d represents the distance between the first base point and the second base point, (x', y') is S LR One point in r L (x',y') is the value of the left rank image (x',y'), r R (x'+d,y') is the value of the right rank image (x'+d,y').

[0031] According to a binocular ranging temperature compensation method based on rank image and YCbCr image fusion provided by the present invention, the image matching is performed using a difference function fused with a local feature difference value and a color difference value, comprising:

[0032] Calculate the color difference value, the formula is:

[0033]

[0034] Among them, C y (x,y,d) is the color difference value, (x”,y”) is S LY A point in the image, Y, Cb, Cr are the three-channel values ​​in the YCbCr image, P(x", y") is the intermediate variable of the weighted combination of the three channels, P L (x”, y”) is the intermediate variable of the left YCbCr image, P R (x”+d,y”) is the intermediate variable of the right YCbCr image;

[0035] Keep the first base point fixed, move the second base point to the right, traverse the current row, and calculate the local feature difference values ​​between the first base point and the second base point in sequence;

[0036] The local feature difference value and the color difference value are merged to obtain the matching difference value;

[0037] Select the second base point corresponding to the minimum matching difference value as the matching point of the first base point, and select the distance between the two points as the disparity;

[0038] The first base point moves, and the above steps are repeated until the first base point traverses the entire left image to obtain a disparity map corresponding to the left rank image.

[0039] According to a binocular ranging temperature compensation method based on rank image and YCbCr image fusion provided by the present invention, when the local feature difference value and the color difference value are fused to obtain the matching difference value, the specific formula is as follows:

[0040]

[0041] Among them, cost(i,j,d) is the matching difference between the left rank image position (i,j) and the right rank image position (i+d,j), C y (i,j,d) is the color difference between the left YCbCr image (i,j) and the right YCbCr image (i+d,j). count_y is the number of corresponding peak positions of the one-dimensional region centered on (i,j) in the left YCbCr image and the one-dimensional region centered on (i+d,j) in the right YCbCr image. C rank(i,j,d) is the local feature difference value between the position (i,j) of the left rank image and the position (i+d,j) of the right rank image, and count_rank is the number of corresponding peak positions of the one-dimensional area centered on (i,j) of the rank image corresponding to the left image and the one-dimensional area centered on (i+d,j) of the rank image corresponding to the right image.

[0042] According to a binocular ranging temperature compensation method based on rank image and YCbCr image fusion provided by the present invention, after obtaining the disparity map, the disparity corresponding to each point is obtained according to the disparity map; the focal length of the binocular camera and the distance between the two cameras are obtained according to the parameter matrix of the binocular camera; the parallax, focal length and camera distance are substituted into the distance calculation formula to obtain the distance, and the distance calculation formula is expressed as the following formula:

[0043]

[0044] Among them, d r is the distance, f is the focal length of the binocular camera, d c is the distance between the binocular cameras, and d is the parallax.

[0045] According to a binocular ranging temperature compensation method based on rank image and YCbCr image fusion provided by the present invention, the distance value is used to compensate the measured temperature of the target terminal, including:

[0046] According to the external parameter matrix of the infrared camera and binocular camera obtained during calibration, the corresponding area of ​​the position area of ​​the cable terminal in the infrared image in the disparity map of the left image is found, and then the distance value of the target area is obtained; according to the distance value of the target area, the distance compensation formula is applied to compensate the temperature value of the target.

[0047] According to a binocular ranging temperature compensation method based on rank image and YCbCr image fusion provided by the present invention, the distance compensation formula includes:

[0048] T(x,y)=T c (x,y)+a0+a1L(x,y)+a2L(x,y) 2 +a3L(x,y) 3 +a4L(x,y) 4

[0049] Among them, T(x,y) is the accurate temperature value corresponding to (x,y) in the infrared image, T c (x, y) is the accurate temperature value corresponding to (x, y) in the infrared image, L(x, y) is the distance corresponding to (x, y) in the infrared image, and a0, a1, a2, a3, and a4 are fitting coefficients.

[0050] It can be seen that compared with the prior art, the method provided by the present invention has the following beneficial effects:

[0051] 1. The present invention can accurately obtain the distance value of the cable terminal, effectively eliminate the interference of distance on temperature, reduce errors, and realize accurate extraction of the cable terminal temperature.

[0052] 2. The present invention adopts a deep learning model to identify cable terminals, which can make the extraction of cable terminal areas sufficiently accurate and reduce interference from non-target areas;

[0053] 3. By collecting binocular visible light images and performing image calibration and correction, the present invention can accurately obtain the internal and external parameters of the camera, effectively eliminate distortion and parallax, thereby improving the accuracy of image matching and further improving the ranging accuracy.

[0054] 4. The present invention combines the local feature difference values ​​and color difference values ​​of the rank image and the YCbCr image for image matching, and can obtain a more accurate disparity map, providing more reliable data support for subsequent triangulation.

[0055] 5. The present invention uses a deep learning model to perform target terminal recognition on the corrected infrared image, which can more accurately determine the location area of ​​the target terminal and extract temperature information within this area, thereby obtaining a more accurate measured temperature value.

[0056] 6. The present invention aligns the infrared image based on the binocular visible light image to ensure that the target terminal in the infrared image is consistent with the target terminal in the three-dimensional measurement result in spatial position, which can improve the accuracy of temperature measurement and can also use the distance value to compensate for the measured temperature to eliminate the temperature measurement error caused by the distance change.

[0057] 7. The present invention combines multiple technical means such as visible light images, infrared images and deep learning to achieve effective fusion of multi-source information, which not only improves the overall performance of the system, but also enhances the system's adaptability to complex environments and multiple targets.

[0058] 8. The present invention introduces a temperature compensation mechanism, which can effectively solve the measurement error problem caused by temperature changes and improve the stability and reliability of the system.

[0059] 9. The present invention realizes rapid processing and analysis of image data by optimizing the image processing flow and algorithm design, which can improve the real-time performance of the system and enable it to respond quickly and accurately complete measurement tasks in practical applications.

[0060] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 The present invention is a flowchart of an embodiment of a binocular ranging temperature compensation method based on rank image and YCbCr image fusion. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0064] See also Figure 1 The present invention provides a binocular ranging temperature compensation method based on rank image and YCbCr image fusion, the method comprising the following steps:

[0065] Step S1, collecting binocular visible light images and infrared images, and performing image calibration and correction to obtain the internal and external parameters of the camera and eliminate distortion and parallax;

[0066] Step S2, converting the corrected visible light image into a rank image;

[0067] Step S3, converting the corrected visible light image from the RGB space to the YCbCr color space to obtain a YCbCr image;

[0068] Step S4, performing image matching based on the rank image and the YCbCr image using a difference function that fuses local feature difference values ​​and color difference values ​​to obtain a disparity map;

[0069] Step S5, using triangulation to obtain the three-dimensional position of each point according to the obtained disparity map and the calibration parameters of the camera;

[0070] Step S6, according to the corrected infrared image, using a deep learning model to identify the target terminal, obtain the location area of ​​the target terminal, extract temperature information in the identified target terminal location area, and obtain the measured temperature value of the target terminal;

[0071] Step S7, registering the infrared image based on the binocular visible light image to ensure that the target terminal in the infrared image is consistent with the target terminal in the three-dimensional measurement result in spatial position, obtaining the distance value of the target terminal in the infrared image, and using the distance value to compensate for the measured temperature of the target terminal.

[0072] In the above step S1, a checkerboard is photographed from multiple angles by a binocular camera, and the corner points of the checkerboard image are detected by a corner point detection algorithm to obtain calibrated corner point data; the internal parameters and external parameters of the left and right cameras are obtained by using the least squares method; the target area including the cable terminal is photographed by the binocular camera to obtain a left image and a right image including the cable terminal; and a correction algorithm is applied to perform distortion correction and epipolar correction on the image including the cable terminal to obtain a corrected image.

[0073] Specifically, a high-precision checkerboard calibration plate is provided, and images of the calibration plate are taken from multiple different angles and positions to ensure that all possible viewing angles and tilts are included.

[0074] Use the corner detection algorithm in image processing libraries such as OpenCV to detect the checkerboard corners in each image to ensure that the number and quality of the detected corners meet the calibration requirements.

[0075] Use the least squares method or other optimization algorithms, such as OpenCV's cv2.calibrateCamera function, to calculate the camera's internal parameters (focal length, optical center, etc.) and external parameters (rotation matrix and translation vector), calibrate the left and right cameras respectively, and obtain their respective parameters.

[0076] A binocular camera is used to shoot the target area including the cable terminal, and the left image and the right image are obtained simultaneously. The internal parameters obtained by camera calibration are used to perform distortion correction on the left and right images.

[0077] In this embodiment, epipolar correction is to make the corresponding points of the left and right cameras in the corrected image on the same horizontal line, thereby simplifying the stereo matching process. In OpenCV, epipolar correction can be achieved by cv2.stereoRectify function and cv2.initUndistortRectifyMap combined with cv2.remap. Epipolar correction involves calculating the reprojection matrix based on the internal and external parameters obtained by calibration, and then applying these matrices to the distortion-corrected image.

[0078] Finally, the rectified image is analyzed to ensure that the epipolar lines are well aligned and there is no obvious distortion, and the rectified image is applied for subsequent image processing tasks such as object detection, measurement, etc.

[0079] In the above step S2, the corrected visible light image is converted into left and right grayscale images; the left grayscale image and the right grayscale image are converted into rank images respectively using a rank transformation algorithm, and the rank transformation formula is expressed as the following formula:

[0080]

[0081]

[0082] Where R(x,y) is the value of (x,y) in the rank image, N is a neighborhood of size m×n centered at (x,y), (x',y') is a point in the neighborhood N, and g(x',y') is the grayscale value of (x',y') in the grayscale image;

[0083] Then save the left rank image and the right rank image.

[0084] In the above step S3, the corrected visible light image is converted from the RGB color space to the YCbCr color space, including:

[0085] The corrected visible light image is converted into a YCbCr image by calculating the converted YCbCr image according to the RGB to YCbCr formula, wherein the RGB to YCbCr formula includes:

[0086] Y(x,y)=(0.257*R(x,y)+0.564*G(x,y)+0.098*B+16) / 255

[0087] C b (x,y)=(0.148*R(x,y)-0.291*G(x,y)+0.439*B(x,y)+128) / 255

[0088] C r (x,y)=(0.439*R(x,y)-0.2368*G(x,y)-0.071*B(x,y)+128) / 255

[0089] Among them, Y(x,y), Cb(x,y), and Cr(x,y) are the YCbCr three-channel values ​​of (x,y), and R(x,y), G(x,y), and B(x,y) are the RGB three-channel values ​​of (x,y).

[0090] In the above step S4, image matching is performed using a difference function that fuses local feature difference values ​​and color difference values, including:

[0091] Select the left rank image (x, y) as the first base point, and extract the 1*11 region S centered on the coordinates of the first base point in the left rank image and the left YCbCr image respectively. LR and S LY(If the area exceeds the image boundary, fill it with 0); select the right image (x+d, y) as the second base point, and extract the 1*11 area S centered on the coordinates of the second base point in the right rank image and the right YCbCr image respectively RR and S RY (If the area exceeds the image boundary, fill it with 0);

[0092] Then, the local feature difference value is calculated, and the formula is:

[0093]

[0094] Among them, C rank (x, y, d) is the local feature difference value, d represents the distance between the first base point and the second base point, (x', y') is S LR One point in r L (x',y') is the value of the left rank image (x',y'), r R (x'+d,y') is the value of the right rank image (x'+d,y').

[0095] Calculate the color difference value, the formula is:

[0096]

[0097] Among them, C y (x,y,d) is the color difference value, (x”,y”) is S LY A point in the image, Y, Cb, Cr are the three-channel values ​​in the YCbCr image, P(x", y") is the intermediate variable of the weighted combination of the three channels, P L (x”, y”) is the intermediate variable of the left YCbCr image, P R (x”+d,y”) is the intermediate variable of the right YCbCr image;

[0098] Keep the first base point fixed, move the second base point to the right, traverse the current row, and calculate the local feature difference values ​​between the first base point and the second base point in sequence;

[0099] The local feature difference value and the color difference value are merged to obtain the matching difference value;

[0100] Select the second base point corresponding to the minimum matching difference value as the matching point of the first base point, and select the distance between the two points as the disparity;

[0101] The first base point moves, and the above steps are repeated until the first base point traverses the entire left image to obtain a disparity map corresponding to the left rank image.

[0102] In this embodiment, when the local feature difference value and the color difference value are merged to obtain the matching difference value, the specific formula is as follows:

[0103]

[0104] Among them, cost(i,j,d) is the matching difference between the left rank image position (i,j) and the right rank image position (i+d,j), C y (i,j,d) is the color difference between the left YCbCr image (i,j) and the right YCbCr image (i+d,j). count_y is the number of corresponding peak positions of the one-dimensional region centered on (i,j) in the left YCbCr image and the one-dimensional region centered on (i+d,j) in the right YCbCr image. C rank (i,j,d) is the local feature difference value between the position (i,j) of the left rank image and the position (i+d,j) of the right rank image, and count_rank is the number of corresponding peak positions of the one-dimensional area centered on (i,j) of the rank image corresponding to the left image and the one-dimensional area centered on (i+d,j) of the rank image corresponding to the right image.

[0105] In the above step S5, after obtaining the disparity map, the disparity corresponding to each point is obtained according to the disparity map; the focal length of the binocular camera and the distance between the two cameras are obtained according to the parameter matrix of the binocular camera; the disparity, focal length and camera distance are substituted into the distance calculation formula to obtain the distance, and the distance calculation formula is expressed as the following formula:

[0106]

[0107] Among them, d r is the distance, f is the focal length of the binocular camera, d c is the distance between the binocular cameras, and d is the parallax.

[0108] In the above step S5, the triangulation method is based on the fact that when two cameras observe the same scene from different angles, the projection positions of the same point in the scene on the imaging planes of the two cameras will be different. By calculating the parallax between the two projection points and combining the internal and external parameters of the camera, the position of the point in three-dimensional space can be calculated.

[0109] In the above step S6, the grayscale image converted from the infrared thermal image containing the cable terminal taken by the infrared camera is input into the trained cable terminal recognition model based on the improved YOLOV5 for recognition to obtain the location area of ​​the cable terminal; the location area of ​​the cable terminal is extracted from the infrared image to obtain the measured temperature value of the cable area.

[0110] Preferably, the deep learning model is a cable terminal recognition model based on improved YOLOV5, which is an infrared target recognition algorithm model constructed with YOLOV5 as the basic framework (specifically, an S-Ghost module is set in the backbone network of the YOLOV5 target recognition network to replace the first two general convolutional layers; an RSK module with an attention mechanism is set at the starting position of the neck network; a loss function including confidence loss, classification loss and positioning loss is constructed), and the training set and validation set of the sample data set are trained and optimized, and the model index is tested using the test set to obtain the optimal target recognition model; of course, it can also be an existing infrared recognition model such as an infrared target recognition algorithm model constructed with YOLOV5 as the basic framework, and the grayscale image of the infrared image taken by the infrared camera is input into the deep learning model to obtain the location area (w, h, x, y, a) of the cable terminal to reduce the interference of the non-target area. Wherein, w and h are the width and height of the detection frame; x and y are the position coordinates of the center of the detection frame; a is the overall rotation angle of the detection frame.

[0111] In the above step S7, according to the external parameter matrix of the infrared camera and the binocular camera obtained during calibration, the corresponding area of ​​the position area of ​​the cable terminal in the infrared image in the disparity map of the left image is found, and then the distance value of the target area is obtained; according to the distance value of the target area, the distance compensation formula is applied to compensate the temperature value of the target. Preferably, the temperature compensation formula can be obtained by collecting a large amount of temperature measurement data at different distances and temperatures, and then fitting and calculating by the least squares method, and finally obtaining a distance-based compensation formula, and the compensation formula is as follows:

[0112] T(x,y)=T c (x,y)+a0+a1L(x,y)+a2L(x,y) 2 +a3L(x,y) 3 +a4L(x,y) 4

[0113] Among them, T(x,y) is the accurate temperature value corresponding to (x,y) in the infrared image, T c (x, y) is the accurate temperature value corresponding to (x, y) in the infrared image, L(x, y) is the distance corresponding to (x, y) in the infrared image, and a0, a1, a2, a3, and a4 are fitting coefficients.

[0114] In summary, the present invention can accurately obtain the distance value of the cable terminal, effectively eliminate the interference of distance on temperature, reduce errors, and realize accurate extraction of the cable terminal temperature; the present invention adopts a deep learning model to identify the cable terminal, which can make the extraction of the cable terminal area sufficiently accurate and reduce the interference of non-target areas;

[0115] Furthermore, by collecting binocular visible light images and performing image calibration and correction, the present invention can accurately obtain the internal and external parameters of the camera, effectively eliminate distortion and parallax, thereby improving the accuracy of image matching and further improving the ranging accuracy.

[0116] Furthermore, the present invention combines the local feature difference values ​​and color difference values ​​of the rank image and the YCbCr image for image matching, and can obtain a more accurate disparity map, providing more reliable data support for subsequent triangulation.

[0117] Furthermore, the present invention uses a deep learning model to perform target terminal recognition on the corrected infrared image, which can more accurately determine the location area of ​​the target terminal and extract temperature information within this area, thereby obtaining a more accurate measured temperature value.

[0118] Furthermore, the present invention aligns the infrared image based on the binocular visible light image to ensure that the target terminal in the infrared image is consistent with the target terminal in the three-dimensional measurement result in spatial position, which can improve the accuracy of temperature measurement and can also use the distance value to compensate for the measured temperature to eliminate the temperature measurement error caused by the distance change.

[0119] Furthermore, the present invention combines multiple technical means such as visible light images, infrared images and deep learning to achieve effective fusion of multi-source information, which not only improves the overall performance of the system, but also enhances the system's adaptability to complex environments and multiple targets.

[0120] Furthermore, the present invention introduces a temperature compensation mechanism, which can effectively solve the measurement error problem caused by temperature changes and improve the stability and reliability of the system.

[0121] Furthermore, the present invention realizes rapid processing and analysis of image data by optimizing the image processing flow and algorithm design, which can improve the real-time performance of the system and enable it to respond quickly and accurately complete measurement tasks in practical applications.

[0122] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. A binocular ranging temperature compensation method based on rank image and YCbCr image fusion, characterized in that: The following steps are involved: Collect binocular visible light images and infrared images, and perform image calibration and correction to obtain the internal and external parameters of the camera and eliminate distortion and parallax; converting the rectified visible light image into a rank image; Convert the corrected visible light image from RGB space to YCbCr color space to obtain a YCbCr image; According to the rank image and the YCbCr image, the disparity map is obtained by performing image matching using a difference function that combines local feature difference values ​​and color difference values; According to the obtained disparity map and the calibration parameters of the camera, the three-dimensional position of each point is obtained using triangulation; According to the corrected infrared image, the target terminal is identified by using a deep learning model to obtain the location area of ​​the target terminal, and the temperature information is extracted in the identified target terminal location area to obtain the measured temperature value of the target terminal; The infrared image is registered based on the binocular visible light image to ensure that the target terminal in the infrared image is consistent with the target terminal in the three-dimensional measurement result in spatial position, the distance value of the target terminal in the infrared image is obtained, and the measured temperature of the target terminal is compensated using the distance value.

2. The method according to claim 1, characterized in that The collecting of binocular visible light images and infrared images and image calibration and correction include: The checkerboard is photographed from multiple angles by a binocular camera, and the corner points of the checkerboard image are detected using a corner detection algorithm to obtain calibrated corner point data; the internal and external parameters of the left and right cameras are calculated using the least squares method; the target area including the cable terminal is photographed by a binocular camera to obtain a left image and a right image including the cable terminal; the correction algorithm is applied to perform distortion correction and epipolar correction on the image including the cable terminal to obtain a corrected image.

3. The method according to claim 1, characterized in that The converting the corrected visible light image into a rank image comprises: The corrected visible light image is converted into left and right grayscale images; the left grayscale image and the right grayscale image are converted into rank images respectively using the rank transformation algorithm. The rank transformation formula is expressed as the following formula: Where R(x,y) is the value of (x,y) in the rank image, N is a neighborhood of size m×n centered at (x,y), (x',y') is a point in the neighborhood N, and g(x',y') is the grayscale value of (x',y') in the grayscale image; Then save the left rank image and the right rank image.

4. The method according to claim 1, characterized in that: The step of converting the corrected visible light image from the RGB color space to the YCbCr color space includes: The corrected visible light image is converted into a YCbCr image by calculating the converted YCbCr image according to the RGB to YCbCr formula, wherein the RGB to YCbCr formula includes: Y(x,y)=(0.257*R(x,y)+0.564*G(x,y)+0.098*B+16) / 255 C b (x,y)=(0.148*R(x,y)-0.291*G(x,y)+0.439*B(x,y)+128) / 255 C r (x,y)=(0.439*R(x,y)-0.2368*G(x,y)-0.071*B(x,y)+128) / 255 Among them, Y(x,y), Cb(x,y), and Cr(x,y) are the YCbCr three-channel values ​​of (x,y), and R(x,y), G(x,y), and B(x,y) are the RGB three-channel values ​​of (x,y).

5. The method according to claim 1, characterized in that The image matching is performed using a difference function that fuses local feature difference values ​​and color difference values, including: Select the left rank image (x, y) as the first base point, and extract the 1*11 region S centered on the coordinates of the first base point in the left rank image and the left YCbCr image respectively. LR and S LY ; Select the right image (x+d,y) as the second base point, and extract the 1*11 region S centered on the coordinates of the second base point in the right rank image and the right YCbCr image respectively RR and S RY ; Then, the local feature difference value is calculated, and the formula is: Among them, C rank (x, y, d) is the local feature difference value, d is the distance between the first base point and the second base point, (x', y') is S LR One point in r L (x',y') is the value of the left rank image (x',y'), r R (x'+d,y') is the value of the right rank image (x'+d,y').

6. The method according to claim 5, characterized in that The image matching is performed using a difference function that fuses local feature difference values ​​and color difference values, including: Calculate the color difference value, the formula is: Among them, C y (x,y,d) is the color difference value, (x”,y”) is S LY A point in the image, Y, Cb, Cr are the three-channel values ​​in the YCbCr image, P(x", y") is the intermediate variable of the weighted combination of the three channels, P L (x”, y”) is the intermediate variable of the left YCbCr image, P R (x”+d,y”) is the intermediate variable of the right YCbCr image; Keep the first base point fixed, move the second base point to the right, traverse the current row, and calculate the local feature difference values ​​between the first base point and the second base point in sequence; The local feature difference value and the color difference value are merged to obtain the matching difference value; Select the second base point corresponding to the minimum matching difference value as the matching point of the first base point, and select the distance between the two points as the disparity; The first base point moves, and the above steps are repeated until the first base point traverses the entire left image to obtain a disparity map corresponding to the left rank image.

7. The method according to claim 6, characterized in that: When the local feature difference value and the color difference value are fused to obtain the matching difference value, the specific formula is as follows: Among them, cost(i,j,d) is the matching difference between the left rank image position (i,j) and the right rank image position (i+d,j), C y (i,j,d) is the color difference between the left YCbCr image (i,j) and the right YCbCr image (i+d,j). count_y is the number of corresponding peak positions of the one-dimensional region centered on (i,j) in the left YCbCr image and the one-dimensional region centered on (i+d,j) in the right YCbCr image. C rank (i,j,d) is the local feature difference value between the position (i,j) of the left rank image and the position (i+d,j) of the right rank image, and count_rank is the number of corresponding peak positions of the one-dimensional area centered on (i,j) of the rank image corresponding to the left image and the one-dimensional area centered on (i+d,j) of the rank image corresponding to the right image.

8. The method according to claim 1, characterized in that: After obtaining the disparity map, the disparity corresponding to each point is obtained according to the disparity map; the focal length of the binocular camera and the distance between the two cameras are obtained according to the parameter matrix of the binocular camera; the disparity, focal length and camera distance are substituted into the distance calculation formula to obtain the distance, which is expressed as the following formula: Among them, d r is the distance, f is the focal length of the binocular camera, d c is the distance between the binocular cameras, and d is the parallax.

9. The method according to claim 1, characterized in that: The method of compensating the measured temperature of the target terminal by using the distance value includes: According to the external parameter matrix of the infrared camera and binocular camera obtained during calibration, the corresponding area of ​​the position area of ​​the cable terminal in the infrared image in the disparity map of the left image is found, and then the distance value of the target area is obtained; according to the distance value of the target area, the distance compensation formula is applied to compensate the temperature value of the target.

10. The method according to claim 9, characterized in that: The distance compensation formula includes: T(x,y)=T c (x,y)+a0+a1L(x,y)+a2L(x,y) 2 +a3L(x,y) 3 +a4L(x,y) 4 Among them, T(x,y) is the accurate temperature value corresponding to (x,y) in the infrared image, T c (x, y) is the accurate temperature value corresponding to (x, y) in the infrared image, L(x, y) is the distance corresponding to (x, y) in the infrared image, and a0, a1, a2, a3, and a4 are fitting coefficients.

Images