Thermal infrared sensor calibration method and device, electronic equipment and storage medium

Abstract

The application provides a kind of thermal infrared sensor calibration method, device, electronic equipment and storage medium, it is related to sensor calibration technical field, the method comprises: the original image is input into segmentation model, obtains the binary image corresponding to original image of segmentation model output, original image is the thermal infrared image of the calibration board collected by the to-be-calibrated thermal infrared sensor, image reconstruction is carried out on the binary image, obtains the contrast enhancement image corresponding to original image, the visual features of the corner point of the checkerboard pattern in the contrast enhancement image are enhanced, obtain the feature enhancement image corresponding to original image, the corner point of the checkerboard pattern in the feature enhancement image is identified, based on the position information of the identified corner point, the internal and external parameter values of the to-be-calibrated thermal infrared sensor are obtained.The application can more accurately and efficiently realize the calibration of thermal infrared sensor, can improve the robustness of thermal infrared sensor calibration, and can realize the calibration of thermal infrared sensor in complex scene.

Description

Technical Field

[0001] This invention relates to the field of sensor calibration technology, and in particular to a method, apparatus, electronic device, and storage medium for calibrating a thermal infrared sensor. Background Technology

[0002] In practical applications of thermal infrared sensors, in order to ensure the accuracy and reliability of the thermal infrared images acquired by the thermal infrared sensors, it is usually necessary to calibrate the thermal infrared sensors by acquiring data.

[0003] Traditional calibration boards in related technologies typically feature a black and white checkerboard pattern. In traditional thermal infrared sensor calibration methods, corner points of the checkerboard pattern in the thermal infrared image of the aforementioned traditional calibration board, acquired using a thermal infrared sensor, can be detected and located. The thermal infrared sensor can then be calibrated based on the positional information of these corner points.

[0004] However, for thermal infrared images acquired by thermal infrared sensors, since the temperature of objects is usually continuous, the boundaries of objects in thermal infrared images are not very clear compared to visible light images. Therefore, when calibrating thermal infrared sensors using the aforementioned traditional calibration methods, the corner points of the checkerboard pattern in the thermal infrared images of the traditional calibration board acquired by the thermal infrared sensor lack sufficient clarity and contrast. This makes it impossible for corner detection algorithms to reliably identify and locate the corner points of the checkerboard pattern in the thermal infrared images of the traditional calibration board, resulting in the difficulty of accurately calibrating thermal infrared sensors using traditional methods. Furthermore, thermal infrared sensors are easily affected by environmental factors during data acquisition. In complex thermal imaging environments (such as non-uniform temperature, low contrast, or noise interference), the corner points of the checkerboard pattern in the thermal infrared images of the traditional calibration board acquired by the thermal infrared sensor may drift and / or become blurred, further reducing the accuracy of thermal infrared sensor calibration. Therefore, how to more accurately calibrate thermal infrared sensors is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for calibrating a thermal infrared sensor, thereby addressing the shortcomings of existing technologies in accurately calibrating thermal infrared sensors and improving the accuracy of thermal infrared sensor calibration.

[0006] This invention provides a method for calibrating a thermal infrared sensor, comprising the following steps.

[0007] The original image is input into the segmentation model to obtain the binarized image corresponding to the original image output by the segmentation model. The original image is the thermal infrared image of the calibration plate collected by the thermal infrared sensor to be calibrated. The calibration plate is provided with a checkerboard pattern. The segmentation model is obtained after training based on the sample image and the corresponding binarized image. The sample image is the thermal infrared image of the calibration plate collected by the sample thermal infrared sensor.

[0008] Image reconstruction is performed on the binarized image corresponding to the original image to obtain a contrast-enhanced image corresponding to the original image;

[0009] The visual features of the corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image are enhanced to obtain the feature-enhanced image corresponding to the original image;

[0010] The corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image are identified, and the intrinsic and extrinsic parameter values ​​of the thermal infrared sensor to be calibrated are obtained based on the position information of the identified corner points.

[0011] According to a thermal infrared sensor calibration method provided by the present invention, the segmentation model includes: an encoder module, a thermal boundary sensing module, and a decoder module;

[0012] The step of inputting the original image into the segmentation model and obtaining the binarized image output by the segmentation model includes:

[0013] The original image is input into the encoder module and the thermal boundary sensing module respectively to obtain the multi-scale features of the original image output by the encoder module and the first thermal feature map corresponding to the original image output by the thermal boundary sensing module.

[0014] The multi-scale features of the original image and the first thermal feature map corresponding to the original image are input into the decoder module to obtain the binarized image corresponding to the original image output by the decoder module.

[0015] According to a thermal infrared sensor calibration method provided by the present invention, the step of reconstructing the binarized image corresponding to the original image to obtain a contrast-enhanced image corresponding to the original image includes:

[0016] Data fusion is performed on the original image and the corresponding binarized image to obtain the first target image;

[0017] The first target image is locally contrast-enhanced using an adaptive histogram equalization algorithm to obtain the second target image;

[0018] Data fusion is performed on the first thermal feature map corresponding to the second target image and the original image to obtain the contrast-enhanced image corresponding to the original image.

[0019] According to a thermal infrared sensor calibration method provided by the present invention, the step of performing feature enhancement on the visual features of the corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image to obtain the feature-enhanced image corresponding to the original image includes:

[0020] Based on the Soberfeldman operator and the Gaussian Laplacian operator, multi-scale features of the edges of the checkerboard pattern in the contrast-enhanced image corresponding to the original image are extracted to obtain the multi-scale edge map corresponding to the original image;

[0021] Calculate the local average contrast of the multi-scale edge map corresponding to the original image, and then construct the feature fusion map corresponding to the original image based on the local average contrast of the multi-scale edge map corresponding to the original image and the multi-scale edge map corresponding to the original image.

[0022] The contrast-enhanced image corresponding to the original image is input into the thermal boundary sensing module to obtain the second thermal feature map corresponding to the original image output by the thermal boundary sensing module. The feature fusion map corresponding to the original image and the second thermal feature map corresponding to the original image are fused to obtain the feature-enhanced image corresponding to the original image.

[0023] According to a thermal infrared sensor calibration method provided by the present invention, the step of identifying the corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image includes:

[0024] The feature-enhanced image corresponding to the original image is input into the corner recognition model to obtain the corner probability heatmap of the feature-enhanced image corresponding to the original image output by the corner recognition model. The corner recognition model is constructed based on a high-resolution network model and is obtained after training based on the feature-enhanced image corresponding to the sample image and the corner probability heatmap of the feature-enhanced image corresponding to the sample image.

[0025] The position information of the corner points of the checkerboard pattern is extracted from the corner probability heatmap of the feature-enhanced image corresponding to the original image using a non-maximum suppression algorithm.

[0026] According to a thermal infrared sensor calibration method provided by the present invention, the sample image is obtained based on the following steps:

[0027] Acquire the thermal infrared image of the calibration plate collected by the sample thermal infrared sensor as the original sample image. The sample thermal infrared sensor is the calibrated thermal infrared sensor.

[0028] After data augmentation is performed on the original sample image, the data-augmented original sample image is determined as the sample image. The data augmentation includes at least one of adding noise, geometric transformation, and illumination adjustment.

[0029] The present invention also provides a thermal infrared sensor calibration device, comprising the following modules:

[0030] The image binarization module is used to input the original image into the segmentation model and obtain the binarized image corresponding to the original image output by the segmentation model. The original image is the thermal infrared image of the calibration plate collected by the thermal infrared sensor to be calibrated. The calibration plate is provided with a checkerboard pattern. The segmentation model is obtained after training based on the sample image and the corresponding binarized image. The sample image is the thermal infrared image of the calibration plate collected by the sample thermal infrared sensor.

[0031] The image reconstruction module is used to reconstruct the image from the binarized image corresponding to the original image to obtain the contrast-enhanced image corresponding to the original image.

[0032] The feature enhancement module is used to enhance the visual features of the corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image, so as to obtain the feature-enhanced image corresponding to the original image.

[0033] The parameter calculation module is used to identify the corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image, and to obtain the internal and external parameter values ​​of the thermal infrared sensor to be calibrated based on the position information of the identified corner points.

[0034] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the thermal infrared sensor calibration method described above.

[0035] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the thermal infrared sensor calibration method as described above.

[0036] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the thermal infrared sensor calibration method described above.

[0037] The thermal infrared sensor calibration method, apparatus, electronic device, and storage medium provided by this invention involve inputting an original image into a segmentation model to obtain a binarized image corresponding to the original image output by the segmentation model. The original image is a thermal infrared image of a calibration board acquired by the thermal infrared sensor to be calibrated, and the calibration board has a checkerboard pattern. The segmentation model is obtained after training based on sample images and their corresponding binarized images, where the sample images are thermal infrared images of the calibration board acquired by sample thermal infrared sensors. Image reconstruction is performed on the binarized image corresponding to the original image to obtain a contrast-enhanced image corresponding to the original image. Feature enhancement is performed on the visual features of the corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image to obtain a feature-enhanced image corresponding to the original image. The corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image are identified. Based on the position information of the identified corner points, the intrinsic and extrinsic parameter values ​​of the thermal infrared sensor to be calibrated are obtained. This method can achieve more accurate and efficient calibration of thermal infrared sensors, improve the robustness of thermal infrared sensor calibration, and enable calibration of thermal infrared sensors in complex scenarios, thus having broad application prospects. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating the thermal infrared sensor calibration method provided by the present invention.

[0040] Figure 2 This is a schematic diagram of the thermal infrared sensor calibration device provided by the present invention.

[0041] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0043] In the description of the invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0044] In the description of this application, the terms "first," "second," etc., are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, in the description of this application, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0045] It should be noted that a thermal infrared sensor is a device that measures temperature based on the principle of infrared radiation. Due to its high temperature sensitivity, high resolution, high accuracy, and non-contact temperature detection capabilities, thermal infrared sensors play an important role in fields such as industrial inspection, building energy conservation, medical diagnosis, and agricultural monitoring.

[0046] Thermal infrared sensors generate thermal infrared images by detecting the infrared radiation emitted by objects, thus creating visual information based on temperature distribution, independent of visible light. Therefore, thermal infrared sensors are widely used for continuous real-time temperature monitoring of environments and objects, and can provide stable imaging even in low-light or no-light conditions.

[0047] In practical applications of thermal infrared sensors, in order to ensure the accuracy and reliability of the thermal infrared images acquired by the thermal infrared sensors, it is usually necessary to calibrate the thermal infrared sensors by acquiring data.

[0048] By calibrating the thermal infrared sensor, the imaging characteristics of the thermal infrared sensor (such as focal length, optical center and distortion coefficient in the intrinsic parameters and camera position and attitude in the extrinsic parameters) can be corrected, reducing errors caused by perspective distortion or geometric distortion.

[0049] Traditional calibration plates in related technologies typically feature a black and white checkerboard pattern. In traditional thermal infrared sensor calibration methods, corner points of the checkerboard pattern in the thermal infrared image of the calibration plate acquired by the thermal infrared sensor can be detected and located. Based on the positional information of these corner points, the intrinsic and extrinsic parameters of the thermal infrared sensor can then be calculated. Therefore, accurately detecting the corner points of the checkerboard pattern in the thermal infrared image of the calibration plate acquired by the thermal infrared sensor is one of the key steps in traditional thermal infrared sensor calibration methods.

[0050] However, the imaging mechanism of thermal infrared sensors relies on the infrared radiation distribution of an object rather than reflected light. Therefore, the contrast of the checkerboard pattern in a traditional calibration plate is typically low in the thermal infrared image of that plate, significantly increasing the difficulty of detecting the corner points of the checkerboard pattern. Furthermore, since the temperature distribution of an object is usually continuous, the edges of the object in the thermal infrared image are not as sharp as those in a visible light image. Consequently, the corner points of the checkerboard pattern in the thermal infrared image of the traditional calibration plate acquired by the thermal infrared sensor lack sufficient sharpness and contrast, making it impossible for corner detection algorithms to reliably identify and locate the corner points of the checkerboard pattern in the thermal infrared image of the traditional calibration plate. If there is a deviation in corner detection, the calculated intrinsic and extrinsic parameters of the thermal infrared sensor will have significant errors, thus affecting the imaging accuracy of the thermal infrared sensor. This error is particularly noticeable in high-precision scenarios such as industrial inspection and agricultural monitoring.

[0051] Furthermore, thermal infrared sensors are susceptible to environmental factors during data acquisition. For example, when performing outdoor measurements, changes in ambient temperature, wind speed, and humidity can affect the imaging performance of the thermal infrared sensor. This can lead to corner shifts and / or blurring in the checkerboard images acquired by the sensor in complex thermal imaging environments (such as non-uniform temperature, low contrast, or noise interference). Consequently, the calibration results of thermal infrared sensors lack stability in variable environments, especially in outdoor applications where changes in ambient temperature, humidity, and lighting significantly impact calibration accuracy.

[0052] In related technologies, image processing techniques such as image enhancement and filtering denoising can improve the corner detection accuracy of checkerboard patterns in the thermal infrared images of the aforementioned traditional calibration plates. However, for the thermal infrared images of the aforementioned traditional calibration plates acquired by thermal infrared sensors in complex thermal imaging environments, it is difficult to achieve ideal results, especially when faced with changes in external environmental factors, it exhibits significant limitations.

[0053] In summary, traditional thermal infrared sensor calibration methods still have significant limitations when calibrating thermal infrared sensors. These limitations mainly manifest in the low accuracy of identifying corner points of checkerboard patterns in thermal infrared images using traditional calibration boards, insufficient robustness, inability to meet calibration accuracy requirements, and strong limitations in handling different environments and imaging conditions. Many applications of thermal infrared sensors (such as industrial inspection, building energy efficiency assessment, and medical testing) have stringent requirements for calibration accuracy. The aforementioned traditional thermal infrared sensor calibration methods fail to provide sufficient high precision and stability, becoming a bottleneck restricting their widespread application in high-precision fields and failing to meet the high-precision calibration needs of industrial inspection, building temperature measurement, and medical diagnosis.

[0054] To address this issue, the present invention provides a thermal infrared sensor calibration method to solve common problems in thermal infrared sensor calibration, such as low contrast, noise interference, and corner blurring, thereby improving the accuracy and robustness of thermal infrared sensor calibration.

[0055] The thermal infrared sensor calibration method provided by this invention reconstructs a checkerboard image using a semantic segmentation model, resulting in a significant contrast between the white squares and the background area. Through a deep learning segmentation model, the checkerboard region in a low-contrast image can be accurately separated, generating a high-contrast binarized image. This effectively enhances the visual features of the checkerboard, making corner locations clearer and facilitating subsequent corner detection. Compared to traditional image enhancement methods, the segmentation model exhibits stronger adaptability and generalization capabilities in feature extraction, and can stably output segmentation results under varying temperature distributions and noise conditions, thus providing reliable technical support for checkerboard reconstruction in complex environments.

[0056] The thermal infrared sensor calibration method provided by this invention addresses the problem of indistinct checkerboard corner points in thermal infrared images by employing feature enhancement and deep learning-based keypoint detection methods. Through a series of feature enhancement operations, such as edge detection and contrast enhancement, the checkerboard corner point regions are made more prominent in the image. Keypoint detection models (such as HRNet) are used to accurately locate the corner points, ensuring the accuracy of corner point coordinates during calibration. The combination of feature enhancement and deep learning-based keypoint detection effectively suppresses noise interference in thermal infrared images, improves the stability of corner point detection, and further reduces calibration errors.

[0057] The following is combined with Figure 1 This invention describes the thermal infrared sensor calibration method provided by the present invention.

[0058] Figure 1 This is a flowchart illustrating the thermal infrared sensor calibration method provided by the present invention, as shown below. Figure 1As shown, the method includes the following steps: Step 101: Input the original image into the segmentation model and obtain the binarized image corresponding to the original image output by the segmentation model. The original image is the thermal infrared image of the calibration plate collected by the thermal infrared sensor to be calibrated. The calibration plate is set with a checkerboard pattern. The segmentation model is obtained after training based on the sample image and the corresponding binarized image. The sample image is the thermal infrared image of the calibration plate collected by the sample thermal infrared sensor.

[0059] It should be noted that the execution subject of this embodiment of the invention is a thermal infrared sensor calibration device. This thermal infrared sensor calibration device can be configured in electronic devices such as computers or servers.

[0060] Specifically, the thermal infrared sensor to be calibrated is the calibration object of the thermal infrared sensor calibration method provided by this invention. The thermal infrared sensor to be calibrated can be determined based on actual needs. In the implementation of this invention, no specific limitations are made on the thermal infrared sensor to be calibrated.

[0061] In this embodiment of the invention, the thermal infrared image of the calibration plate acquired by the thermal infrared sensor to be calibrated can be determined as the original image. The calibration plate in this embodiment of the invention has a checkerboard pattern.

[0062] Optionally, in this embodiment of the invention, the checkerboard pattern on the calibration board is composed of alternating black and white square blocks.

[0063] The original image can be acquired in various ways in this embodiment of the invention. For example, the original image can be acquired based on user input; or, the original image can be received from other electronic devices.

[0064] After acquiring the original image, it can be input into the segmentation model. The segmentation model can extract the checkerboard pattern from the original image and generate a high-contrast binarized image, providing a more accurate data foundation for subsequent corner detection of the checkerboard pattern.

[0065] It should be noted that the segmentation model in this embodiment of the invention is obtained by training an initial model using sample images as training samples and the corresponding binarized images as sample labels. The sample images can be thermal infrared images of the calibration plate acquired by other calibrated thermal infrared sensors.

[0066] The binarized image corresponding to the sample image can be obtained by technicians marking the area where the checkerboard pattern is located in the sample image, and then performing binarization processing on the sample image with the marked checkerboard pattern area.

[0067] As an optional embodiment, the segmentation model includes: an encoder module, a thermal boundary sensing module, and a decoder module.

[0068] The original image is input into the segmentation model to obtain the binarized image output by the segmentation model, including: inputting the original image into the encoder module and the thermal boundary sensing module respectively, and obtaining the multi-scale features of the original image output by the encoder module and the first thermal feature map corresponding to the original image output by the thermal boundary sensing module.

[0069] The multi-scale features of the original image and the first thermal feature map corresponding to the original image are input into the decoder module to obtain the binarized image corresponding to the original image output by the decoder module.

[0070] Specifically, the segmentation model in this embodiment of the invention is an improved U-Net network model. The segmentation model in this embodiment of the invention adds a Thermal Boundary Attention Module (TBA) to the U-Net network model, which can effectively capture the temperature difference characteristics of the area where the checkerboard pattern is located in the original image, and improve the segmentation performance of the segmentation model for low-contrast areas in the original image.

[0071] It should be noted that the encoder module is used to perform convolution, pooling, and normalization operations on the original image to extract multi-scale features of the original image.

[0072] Accordingly, the encoder module includes convolutional layers, pooling layers, and normalization operation layers. The formula for the encoder module is expressed as follows:

[0073] F enc =f enc [I(x,y)]

[0074] Among them, F enc The multi-scale features of the original image are represented by I; I represents the original image; I(x,y) represents the pixel value of the point with coordinates (x,y) in the original image; f enc [*] indicates the encoder function.

[0075] The thermal boundary sensing module is used to obtain the gradient information and global contrast of the original image, and then generate the first thermal feature map corresponding to the original image by combining the gradient information and global contrast of the original image.

[0076] The formula for the thermal boundary sensing module is expressed as follows:

[0077]

[0078] Wherein, TBA(I) represents the first thermal feature map corresponding to the original image; This represents the gradient information of the original image; w1 and w2 represent the learnable model weights. Softmax(*) represents the Softmax activation function.

[0079] The decoder module is used to fuse the multi-scale features of the original image and the first thermal feature map corresponding to the original image by combining skip connections and upsampling operations, gradually restoring the resolution of the original image, thereby obtaining the binarized image corresponding to the original image.

[0080] The formula for the decoder module is expressed as follows:

[0081] M(x, y) = σ{f dec [F enc +TBA(I)]}

[0082] Among them, f dec [*] denotes the decoder function; σ{*} denotes the Sigmoid activation function; M denotes the binarized image corresponding to the original image; M(x, y) denotes the pixel value of the point with coordinates (x, y) in the binarized image corresponding to the original image.

[0083] As an optional embodiment, the sample image is obtained based on the following steps: acquiring the thermal infrared image of the calibration plate collected by the sample thermal infrared sensor as the original sample image, wherein the sample thermal infrared sensor is the calibrated thermal infrared sensor.

[0084] After data augmentation of the original sample image, the data-augmented original sample image is determined as the sample image. Data augmentation includes at least one of adding noise, geometric transformation, and illumination adjustment.

[0085] It should be noted that, in order to ensure that the segmentation model can adapt to a variety of complex environments and improve the generalization performance of the segmentation model, the embodiments of the present invention have designed a variety of data augmentation strategies and a scientific training process, so that the segmentation model can not only perform well under standard experimental conditions, but also cope with complex real-world scenarios.

[0086] In this embodiment of the invention, the thermal infrared image of the calibration plate acquired by the sample thermal infrared sensor is used as the original sample image. Then, the original sample image can be data-enhanced by adding at least one of noise, geometric transformation and illumination adjustment.

[0087] To enhance the robustness of the segmentation model to noise in thermal infrared images, various noise simulation methods can be used to generate different types of noise, such as Gaussian noise N(0, σ). 2 This involves the superposition of random signals, including salt-and-pepper noise and simulated sensor interference. These noises are randomly added to the original sample images to create sample images that broadly cover different noise levels.

[0088] The geometric transformations in these embodiments may include, but are not limited to, rotation (range ±30°), scaling (0.7x-1.3x), translation (±15% of image size), affine transformation, and perspective transformation. These transformations are used to simulate checkerboard images from different viewpoints to enhance the adaptability of the segmentation model to changes in the position of the thermal infrared sensor.

[0089] In this embodiment of the invention, the lighting conditions of the original sample image can be adjusted by gamma transformation (γ∈[0.5,2.0]) and random adjustment of brightness or contrast to adapt to possible light interference in reality.

[0090] In this embodiment of the invention, a multi-stage optimization strategy is employed when training the initial model using sample images as training samples and the corresponding binarized images as sample labels. In the initial stage, a low learning rate (e.g., 0.001) is used for stable training to ensure rapid model convergence; later, the learning rate is dynamically adjusted to further improve the accuracy of the segmentation boundaries.

[0091] In this embodiment of the invention, the cross-entropy loss function L is used. BCE and Dice loss function L Dice The combination of these factors serves as the loss function for the segmentation model, balancing the overall accuracy and edge sensitivity of the model. During training, cross-validation is used to evaluate the segmentation accuracy of the model in real time, ensuring that overfitting does not occur on the validation set.

[0092] The loss function of the segmentation model can be expressed by the following formula:

[0093] L=λ1L BCE +λ2L Dice

[0094] Among them, L BCE L represents the cross-entropy loss function; Dice Let λ represent the Dice loss function; λ1 and λ2 represent the balancing weights.

[0095] In this embodiment of the invention, when training the initial model using sample images as training samples and the corresponding binarized images as sample labels, mixed precision training technology is used to accelerate the training speed of large-scale data while reducing the use of GPU memory.

[0096] In this embodiment of the invention, through data augmentation and scientific training, the segmentation model can accurately segment the region containing the checkerboard pattern in the original image under low contrast and high noise conditions.

[0097] Step 102: Reconstruct the image from the binarized image corresponding to the original image to obtain the contrast-enhanced image corresponding to the original image.

[0098] Specifically, after obtaining the binarized image M corresponding to the original image, in this embodiment of the invention, the region where the checkerboard pattern is located in the binarized image M corresponding to the original image can be further reconstructed by means of numerical calculation, mathematical statistics and deep learning technology to obtain the contrast-enhanced image corresponding to the original image, so as to solve the problem of blurry and unclear boundary in the region where the checkerboard pattern is located in the thermal infrared image of the calibration plate in related technologies.

[0099] As an optional embodiment, image reconstruction is performed on the binarized image corresponding to the original image to obtain a contrast-enhanced image corresponding to the original image, including: data fusion of the original image and the binarized image corresponding to the original image to obtain a first target image.

[0100] Specifically, in this embodiment of the invention, a first target image can be obtained by calculating a weighted sum to fuse the original image I and the corresponding binarized image M. The specific calculation formula is as follows:

[0101] I re (x,y)=M(x,y)·I(x,y)+[1-M(x,y)]·I bg (x, y)

[0102] Among them, I re Indicates the first target image; I re (x, y) represents the pixel value of the point with coordinates (x, y) in the first target image; I bg (x, y) represents the background correction value, which can be calculated by statistically analyzing the background temperature distribution of the original image.

[0103] By fusing the original image I with its corresponding binarized image M, the interference of the background on the segmented region can be effectively reduced.

[0104] The first target image is locally contrast-enhanced using an adaptive histogram equalization algorithm to obtain the second target image.

[0105] It's important to note that Contrast Limited Adaptive Histogram Equalization (CLAHE) is an image enhancement algorithm designed to improve local contrast while avoiding over-enhancement or noise amplification that can occur with global histogram equalization. CLAHE enhances local contrast by dividing the image into small blocks (called "tiles") and performing histogram equalization on each block individually. This method significantly improves the visual quality of images, particularly in medical images, low-light images, or images with uneven contrast.

[0106] Acquire the first target image I re Then, an adaptive histogram equalization algorithm can be used to process the first target image I. re Perform local contrast enhancement to ensure the first target image I re The checkerboard pattern is effectively highlighted in low-to-medium contrast regions. The adaptive histogram equalization algorithm achieves local enhancement through block equalization, making it suitable for the characteristics of thermal infrared images. The specific calculation formula is as follows:

[0107] I clahe (x,y)=CLANE[I re (x, y)]

[0108] Among them, I clahe (x, y) represents the second target image; CLANE[*] represents the adaptive histogram equalization algorithm.

[0109] Data fusion is performed on the first thermal feature map corresponding to the second target image and the original image to obtain the contrast-enhanced image corresponding to the original image.

[0110] Specifically, acquire the second target image I clahe After (x,y), the first thermal feature map TBA(I) corresponding to the original image output by the thermal boundary sensing module in the segmentation model and the second target image I can be used. clahe Nonlinear fusion is performed on (x,y) to further optimize the second target image I. clahe The visual effect of the edge of the checkerboard pattern in (x,y) is calculated using the following formula:

[0111] I enh (x,y)=α·I clahe (x,y)+β·TBA(I)

[0112] Among them, I enh Indicates the contrast-enhanced image corresponding to the original image; I enh(x,y) represents the pixel value of the point with coordinates (x,y) in the contrast-enhanced image corresponding to the original image; α and β represent adjustment coefficients used to control the degree of boundary enhancement. The specific values ​​of α and β can be predefined based on prior knowledge and / or actual conditions.

[0113] This invention embodiment obtains a first target image by fusing data between the original image and its corresponding binarized image. Then, it uses an adaptive histogram equalization algorithm to enhance the local contrast of the first target image to obtain a second target image. Subsequently, it fuses data between the second target image and the first thermal feature map corresponding to the original image to obtain a contrast-enhanced image corresponding to the original image. The checkerboard pattern has a clearer boundary visually, which can significantly improve the accuracy and reliability of the corner points of the subsequent checkerboard pattern.

[0114] Step 103: Perform feature enhancement on the visual features of the corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image to obtain the feature-enhanced image corresponding to the original image.

[0115] Specifically, obtain the contrast-enhanced image I corresponding to the original image. enh Subsequently, contrast-enhanced image I corresponding to the original image can be obtained through numerical computation, mathematical statistics, and deep learning techniques. enh The visual features of the corner points of the checkerboard pattern are enhanced. Through feature enhancement, the saliency of the corner points of the checkerboard pattern in the original image obtained in complex scenes can be improved. At the same time, multi-stage feature fusion is emphasized to improve the accuracy and reliability of the corner points of the subsequent checkerboard pattern.

[0116] As an optional embodiment, feature enhancement is performed on the visual features of the corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image to obtain a feature-enhanced image corresponding to the original image. This includes: extracting multi-scale features of the edges of the checkerboard pattern in the contrast-enhanced image corresponding to the original image based on the Soberfeldman operator and the Gaussian Laplacian operator to obtain a multi-scale edge map corresponding to the original image.

[0117] It's important to note that the Sobel operator, also known as the Sobel-Federman operator or Sobel filter, is a classic edge detection algorithm in image processing. The Sobel operator works based on the first-order discrete difference of the image's grayscale function. It contains two sets of 3x3 matrices, used to detect edges in the horizontal and vertical directions of the image, respectively. The horizontal template is [[-1,0,1],[-2,0,2],[-1,0,1]], while the vertical template is [[-1,-2,-1],[0,0,0],[1,2,1]]. By convolving these two templates with the image, the gradients in the X and Y directions can be calculated. Further combining these gradients yields the magnitude and direction of the gradient. When the gradient value of a pixel exceeds a preset threshold, that pixel is identified as an edge point.

[0118] The Laplacian of Gaussian (LoG) operator is a commonly used edge detection algorithm in image processing. It's derived by using the Laplacian operator to extract edges, building upon the Gaussian function. It combines the advantages of Gaussian smoothing filters and Laplacian sharpening filters, first smoothing the image to suppress noise, and then performing edge detection on the smoothed image.

[0119] In this embodiment of the invention, the contrast-enhanced image I corresponding to the original image is extracted by combining the Sobel operator and the LoG operator. enh The multi-scale features of the edges of the checkerboard pattern are calculated using the following formula:

[0120]

[0121] Among them, G s The Gaussian kernel representing scale *s* is used to smooth gradient responses at different resolutions; *s* represents a positive integer greater than 0; *S* represents the number of scales, which can be determined based on prior knowledge and / or actual conditions; w s The weights representing scale s can be dynamically adjusted according to environmental characteristics to adapt to different disturbance conditions; I edge Represents the multi-scale edge map corresponding to the original image; I edge (x, y) represents the pixel value of the point with coordinates (x, y) in the multi-scale edge map corresponding to the original image.

[0122] Calculate the local average contrast of the multi-scale edge map corresponding to the original image, and then construct the feature fusion map corresponding to the original image based on the local average contrast of the multi-scale edge map corresponding to the original image and the multi-scale edge map corresponding to the original image.

[0123] Specifically, the multi-scale edge map I corresponding to the original image is obtained.edge Then, the multi-scale edge map I corresponding to the original image can be calculated based on the following formula. edge Local average contrast:

[0124]

[0125] Among them, C local (x, y) represents the multi-scale edge map I corresponding to the original image. edge The local average contrast of the point with coordinates (x, y).

[0126] Obtain the multi-scale edge map I corresponding to the original image edge The local average contrast C of the point with coordinates (x, y) local After (x, y), the multi-scale edge map I corresponding to the original image can be obtained. edge The local average contrast C of the point with coordinates (x, y) local (x,y) corresponds to the pixel value I of the point with coordinates (x,y) in the multi-scale edge map corresponding to the original image. edge Data fusion is performed on (x, y) to obtain the multi-scale edge map I corresponding to the original image. edge Feature enhancement is applied to the corner areas of the checkerboard pattern. The specific calculation formula is as follows:

[0127] I fusion (x,y)=I edge (x,y)·[1+C local (x, y)]

[0128] Among them, I fusion Represents the feature fusion map corresponding to the original image; I fusion (x,y) represents the pixel value of the point with coordinates (x,y) in the feature fusion map corresponding to the original image.

[0129] The contrast-enhanced image corresponding to the original image is input into the thermal boundary sensing module to obtain the second thermal feature map corresponding to the original image output by the thermal boundary sensing module. The feature fusion map corresponding to the original image and the second thermal feature map corresponding to the original image are fused to obtain the feature-enhanced image corresponding to the original image.

[0130] Specifically, obtain the feature fusion map I corresponding to the original image. fusion Subsequently, to further enhance the salience of the corner points of the checkerboard pattern, in this embodiment of the invention, a contrast-enhanced image I corresponding to the original image can be used. enh Input the thermal boundary sensing module to obtain the second thermal feature map TBA(I) corresponding to the original image output by the thermal boundary sensing module. enhFurthermore, by calculating a weighted sum, the feature fusion map I corresponding to the original image can be obtained. fusion The second thermal feature map TBA(I) corresponding to the original image enh Data fusion is performed to obtain the feature-enhanced image corresponding to the original image. The specific calculation formula is as follows:

[0131] I final (x, y) = η·I fusion (x, y) + μ·TBA(I) enh )

[0132] Among them, I final Represents the feature-enhanced image corresponding to the original image; I final (x,y) represents the pixel value of the point with coordinates (x,y) in the feature-enhanced image corresponding to the original image; η and μ represent the parameters used to control edge response and thermal contribution; the specific values ​​of η and μ can be determined based on prior knowledge and / or actual conditions.

[0133] This invention extracts multi-scale features of the checkerboard pattern edges in the contrast-enhanced image corresponding to the original image based on the Sober-Federman operator and the Gaussian-Laplace operator. After obtaining the multi-scale edge map corresponding to the original image, the local average contrast of the multi-scale edge map corresponding to the original image is calculated. Then, based on the local average contrast of the multi-scale edge map corresponding to the original image and the multi-scale edge map corresponding to the original image, a feature fusion map corresponding to the original image is constructed. The contrast-enhanced image corresponding to the original image is then input into a thermal boundary sensing module to obtain a second thermal feature map corresponding to the original image output by the thermal boundary sensing module. Data fusion is performed on the feature fusion map corresponding to the original image and the second thermal feature map corresponding to the original image to obtain a feature-enhanced image corresponding to the original image. The feature clarity and signal-to-noise ratio of the corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image are significantly improved, which can provide a more accurate data basis for the calibration of the thermal infrared sensor to be calibrated.

[0134] Step 104: Identify the corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image, and obtain the internal and external parameter values ​​of the thermal infrared sensor to be calibrated based on the position information of the identified corner points.

[0135] Specifically, obtain the feature-enhanced image I corresponding to the original image. final Subsequently, feature-enhanced images (I) corresponding to the original image can be identified through numerical computation, mathematical statistics, and deep learning techniques. final The corner points of the chessboard image.

[0136] As an optional embodiment, identifying the corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image includes: inputting the feature-enhanced image corresponding to the original image into a corner recognition model, obtaining a corner probability heatmap of the feature-enhanced image corresponding to the original image output by the corner recognition model, wherein the corner recognition model is constructed based on a high-resolution network model and is obtained after training based on the feature-enhanced image corresponding to the sample image and the corner probability heatmap of the feature-enhanced image corresponding to the sample image.

[0137] Specifically, High-Resolution Network (HRNet) is an advanced deep learning model specifically designed for visual recognition tasks such as pose estimation, semantic segmentation, object detection, and image classification. The core idea of ​​the HRNet model is to maintain high-resolution feature representations throughout the entire network process to improve spatial accuracy and semantic expressiveness. Unlike traditional convolutional neural networks (such as ResNet and U-Net), the HRNet model maintains a high-resolution feature stream throughout the entire network process, rather than recovering resolution through downsampling and then upsampling. This design allows the HRNet model to capture more detailed visual information and improve recognition accuracy.

[0138] It should be noted that the method for obtaining the feature-enhanced image corresponding to the sample image in this embodiment of the invention is different from the method for obtaining the feature-enhanced image I corresponding to the original image. final The specific steps for obtaining the feature enhancement image corresponding to the sample image are the same as those in the above embodiments, and will not be repeated in the embodiments of this invention.

[0139] It should be noted that the angle probability heatmap of the feature-enhanced image corresponding to the sample image can be obtained by technicians after annotating the feature-enhanced image corresponding to the sample image.

[0140] In this embodiment of the invention, after constructing the initial corner recognition model based on the HRNet model, the feature-enhanced image corresponding to the sample image can be used as the training sample, and the corner probability heatmap of the feature-enhanced image corresponding to the sample image can be used as the sample label to train the initial corner recognition model and obtain the trained corner recognition model.

[0141] In this embodiment of the invention, the initial model for corner point recognition is trained based on the mean squared error loss function. The formula for the mean squared error loss function is as follows:

[0142]

[0143] Among them, L MSE This represents the mean squared error loss function value; H represents the probability that the point with coordinates (x, y) in the corner probability heatmap of the feature-enhanced image corresponding to the i-th sample image is an angle; i (x, y) represents the probability that the point with coordinates (x, y) in the predicted corner probability heatmap of the feature-enhanced image corresponding to the i-th sample image output by the initial corner recognition model during training is an angle after the feature-enhanced image corresponding to the i-th sample image is input into the initial corner recognition model during training; i represents a positive integer greater than 0; N represents the number of sample images.

[0144] The trained initial model for corner detection can be represented by the following formula:

[0145] H(x, y) = HRNet[I final (x, y)]

[0146] Where H(x, y) represents the probability that the point with coordinates (x, y) in the corner probability heatmap of the feature-enhanced image corresponding to the original image is an angle; HRNet[*] represents the HRNet model function.

[0147] The nonmaximum suppression algorithm is used to extract the position information of the corner points of the checkerboard pattern from the corner point probability heatmap of the feature-enhanced image corresponding to the original image.

[0148] Specifically, after obtaining the corner probability heatmap of the feature-enhanced image corresponding to the original image, the non-maximum suppression (NMS) algorithm can be used to extract the position information of the checkerboard pattern from the corner probability heatmap of the feature-enhanced image corresponding to the original image. The specific calculation formula is as follows:

[0149]

[0150] Where K represents the set of corner points of the checkerboard pattern in the feature-enhanced image corresponding to the extracted original image; N(x i y i ) represents the point (x) i y i The local neighborhood centered on ) is used to filter the local maximum values ​​of the heatmap values ​​during this process; (x i ,y i H(x) represents the coordinates of the i-th pixel; i ,y i ) indicates that at coordinate (x i ,y i The heatmap value at the location is the probability value that the pixel is a corner point.

[0151] It's important to note that reprojection error minimization is a crucial algorithm in computer vision and robotics, playing a key role, particularly in Simultaneous Localization and Mapping (SLAM) and 3D reconstruction. Reprojection error refers to the error between the observed pixel coordinates (i.e., the observed projected position) in an image and the position obtained by projecting the observed pixel coordinates based on the currently estimated camera pose and 3D feature points. The goal of reprojection error minimization algorithms is to minimize this error by adjusting the camera pose and the positions of the 3D feature points.

[0152] After obtaining the set K of corner points K of the checkerboard pattern in the feature-enhanced image corresponding to the original image, in this embodiment of the invention, the intrinsic and extrinsic parameters of the thermal infrared sensor to be calibrated can be calculated based on the position information of the corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image, using a minimum reprojection error algorithm. The formula for the camera projection model is expressed as follows:

[0153]

[0154] Where ρ represents the scale factor, P represents the intrinsic parameter matrix, R represents the rotation matrix in the extrinsic parameter matrix, t represents the translation vector in the extrinsic parameter matrix, and (u,v) are the pixel coordinates in the chessboard corner image, provided by the corner set K, and associated with the true coordinates (X,Y,Z) of the chessboard corner in three-dimensional space through the camera projection model.

[0155] The reprojection error is defined as:

[0156]

[0157] Where, p i Indicates the actual detected corner point. This represents the corner point calculated by projection. The optimization objective is to minimize e using the Levenberg-Marquardt method, ultimately obtaining the intrinsic and extrinsic parameter values ​​of the thermal infrared sensor to be calibrated.

[0158] This invention, through its embodiment, inputs an original image into a segmentation model to obtain a binarized image corresponding to the original image output by the segmentation model. The original image is a thermal infrared image of a calibration board acquired by a thermal infrared sensor to be calibrated, with a checkerboard pattern on the calibration board. The segmentation model is obtained after training based on sample images and their corresponding binarized images, where the sample images are thermal infrared images of the calibration board acquired by the sample thermal infrared sensor. Image reconstruction is performed on the binarized image corresponding to the original image to obtain a contrast-enhanced image. Feature enhancement is performed on the visual features of the corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image to obtain a feature-enhanced image corresponding to the original image. The corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image are identified. Based on the position information of the identified corner points, the intrinsic and extrinsic parameter values ​​of the thermal infrared sensor to be calibrated are obtained. This method enables more accurate and efficient calibration of thermal infrared sensors, improves the robustness of thermal infrared sensor calibration, and allows for calibration of thermal infrared sensors in complex scenarios, showing broad application prospects.

[0159] The thermal infrared sensor calibration method provided in this invention solves the problem of checkerboard region segmentation in low-contrast and complex temperature difference environments by introducing a thermal boundary sensing module (TBA) into the segmentation model. The segmentation model utilizes a U-Net architecture, extracting multi-scale features through an encoder and combining skip connections to achieve high-resolution reconstruction, resulting in clearer checkerboard boundaries. Further integration of binarization and enhancement operations effectively improves the contrast and boundary features of the checkerboard image, providing high-quality input data for subsequent corner detection.

[0160] The thermal infrared sensor calibration method provided by this invention addresses the issue of insignificant corner features of checkerboard patterns in thermal infrared images by designing a multi-stage feature enhancement method, including multi-scale edge extraction, local contrast enhancement, and thermal boundary optimization. By combining gradient response and local contrast analysis, an enhanced image with clear boundaries and prominent corner features is generated. The thermal boundary sensing module further enhances the salience of low temperature difference regions, making it particularly suitable for environments with complex noise and temperature difference distributions.

[0161] The thermal infrared sensor calibration method provided by this invention innovatively employs the HRNet high-resolution deep learning model for keypoint detection. HRNet ensures high-precision detection capability of checkerboard corner points through multi-scale feature fusion and high-resolution feature preservation, reliably locating corner points even in complex backgrounds. The probabilistic heatmap output by the network extracts corner point coordinates through non-maximum suppression and provides high-precision input parameters for calibration calculations.

[0162] The thermal infrared sensor calibration method provided by this invention significantly improves the corner detection accuracy under low contrast and complex temperature difference conditions by combining deep learning image segmentation and key point detection technology, reduces the estimation deviation of internal and external parameters caused by the accumulation of errors in traditional methods, and comprehensively improves the reliability of calibration results.

[0163] The thermal infrared sensor calibration method provided by this invention can adapt to complex environmental conditions such as different temperature gradients, illumination changes, and noise interference. The segmentation model and feature enhancement module significantly optimize the checkerboard region in the thermal infrared image, achieving stable corner detection and parameter estimation even in non-ideal imaging environments.

[0164] The thermal infrared sensor calibration method provided by this invention automatically extracts checkerboard features and optimizes parameters using deep learning technology, significantly simplifying the complex manual adjustments and preprocessing steps in traditional calibration methods. Furthermore, combined with the hardware-accelerated deep learning inference capabilities of modern technology, the calibration process can be completed quickly, making it suitable for large-scale deployments and real-time applications.

[0165] The thermal infrared sensor calibration method provided by this invention is compatible with various practical application environments, including high-precision indoor industrial testing, outdoor agricultural monitoring, and building energy efficiency assessment. Regardless of whether the temperature distribution is uniform or the background noise conditions are complex, this method maintains high applicability and result stability.

[0166] Figure 2 This is a schematic diagram of the thermal infrared sensor calibration device provided by the present invention. The following is in conjunction with… Figure 2 The thermal infrared sensor calibration device provided by this invention will be described below. The thermal infrared sensor calibration device described below can be referred to in correspondence with the thermal infrared sensor calibration method provided by this invention described above. For example... Figure 2 As shown, there are image binarization module 201, image reconstruction module 202, feature enhancement module 203, and parameter calculation module 204.

[0167] The image binarization module 201 is used to input the original image into the segmentation model and obtain the binarized image corresponding to the original image output by the segmentation model. The original image is the thermal infrared image of the calibration board collected by the thermal infrared sensor to be calibrated. The calibration board is set with a checkerboard pattern. The segmentation model is obtained after training based on the sample image and the corresponding binarized image. The sample image is the thermal infrared image of the calibration board collected by the sample thermal infrared sensor.

[0168] The image reconstruction module 202 is used to reconstruct the image from the binarized image corresponding to the original image to obtain the contrast-enhanced image corresponding to the original image.

[0169] The feature enhancement module 203 is used to enhance the visual features of the corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image, so as to obtain the feature-enhanced image corresponding to the original image.

[0170] The parameter calculation module 204 is used to identify the corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image, and to obtain the internal and external parameter values ​​of the thermal infrared sensor to be calibrated based on the position information of the identified corner points.

[0171] Specifically, the image binarization module 201, the image reconstruction module 202, the feature enhancement module 203, and the parameter calculation module 204 are electrically connected.

[0172] The thermal infrared sensor calibration device in this embodiment of the invention obtains a binarized image corresponding to the original image by inputting the original image into a segmentation model. The original image is a thermal infrared image of a calibration board collected by the thermal infrared sensor to be calibrated, and the calibration board is provided with a checkerboard pattern. The segmentation model is obtained after training based on sample images and their corresponding binarized images. The sample images are thermal infrared images of the calibration board collected by sample thermal infrared sensors. Image reconstruction is performed on the binarized image corresponding to the original image to obtain a contrast-enhanced image corresponding to the original image. Feature enhancement is performed on the visual features of the corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image to obtain a feature-enhanced image corresponding to the original image. The corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image are identified. Based on the position information of the identified corner points, the intrinsic and extrinsic parameter values ​​of the thermal infrared sensor to be calibrated are obtained. This device can achieve more accurate and efficient calibration of thermal infrared sensors, improve the robustness of thermal infrared sensor calibration, and enable calibration of thermal infrared sensors in complex scenarios, thus having broad application prospects.

[0173] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 can call logic instructions in the memory 330 to execute a thermal infrared sensor calibration method. This method includes: inputting an original image into a segmentation model to obtain a binarized image corresponding to the original image output by the segmentation model. The original image is a thermal infrared image of a calibration board acquired by the thermal infrared sensor to be calibrated. The calibration board has a checkerboard pattern. The segmentation model is obtained after training based on sample images and their corresponding binarized images. The sample images are thermal infrared images of the calibration board acquired by the sample thermal infrared sensor. The processor 310 performs image reconstruction on the binarized image corresponding to the original image to obtain a contrast-enhanced image corresponding to the original image. The processor 310 performs feature enhancement on the visual features of the corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image to obtain a feature-enhanced image corresponding to the original image. The processor 310 identifies the corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image, and based on the identified corner point position information, obtains the intrinsic and extrinsic parameter values ​​of the thermal infrared sensor to be calibrated.

[0174] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0175] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the thermal infrared sensor calibration method provided by the above methods. The method includes: inputting an original image into a segmentation model to obtain a binarized image corresponding to the original image output by the segmentation model. The original image is a thermal infrared image of a calibration board collected by the thermal infrared sensor to be calibrated. A checkerboard pattern is set on the calibration board. The segmentation model is obtained after training based on a sample image and the corresponding binarized image. The sample image is a thermal infrared image of the calibration board collected by a sample thermal infrared sensor. Image reconstruction is performed on the binarized image corresponding to the original image to obtain a contrast-enhanced image corresponding to the original image. Feature enhancement is performed on the visual features of the corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image to obtain a feature-enhanced image corresponding to the original image. The corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image are identified, and the intrinsic and extrinsic parameter values ​​of the thermal infrared sensor to be calibrated are obtained based on the position information of the identified corner points.

[0176] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program performs the thermal infrared sensor calibration method provided by the above methods. The method includes: inputting an original image into a segmentation model to obtain a binarized image corresponding to the original image output by the segmentation model. The original image is a thermal infrared image of a calibration board acquired by the thermal infrared sensor to be calibrated. A checkerboard pattern is set on the calibration board. The segmentation model is obtained after training based on a sample image and the corresponding binarized image. The sample image is a thermal infrared image of the calibration board acquired by a sample thermal infrared sensor. The method includes: performing image reconstruction on the binarized image corresponding to the original image to obtain a contrast-enhanced image corresponding to the original image; performing feature enhancement on the visual features of the corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image to obtain a feature-enhanced image corresponding to the original image; identifying the corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image; and obtaining the intrinsic and extrinsic parameter values ​​of the thermal infrared sensor to be calibrated based on the position information of the identified corner points.

[0177] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0178] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of calibrating a thermal infrared sensor, characterized by, The method comprises the following steps: inputting an original image into a segmentation model to obtain a binary image corresponding to the original image output by the segmentation model, the original image being a thermal infrared image of a calibration board collected by a to-be-calibrated thermal infrared sensor, the calibration board being provided with a checkerboard pattern, the segmentation model being obtained by training based on a sample image and a binary image corresponding to the sample image, the sample image being a thermal infrared image of the calibration board collected by a sample thermal infrared sensor; performing image reconstruction on the binary image corresponding to the original image to obtain a contrast-enhanced image corresponding to the original image; performing feature enhancement on visual features of corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image to obtain a feature-enhanced image corresponding to the original image; identifying the corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image, and obtaining internal and external parameter values of the to-be-calibrated thermal infrared sensor based on position information of the identified corner points; the segmentation model comprises an encoder module, a thermal boundary perception module and a decoder module; the step of inputting the original image into the segmentation model to obtain the binary image output by the segmentation model comprises the following steps: inputting the original image into the encoder module and the thermal boundary perception module respectively to obtain multi-scale features of the original image output by the encoder module and a first thermal feature map corresponding to the original image output by the thermal boundary perception module; the thermal boundary perception module is used to generate the first thermal feature map corresponding to the original image by combining gradient information and global contrast of the original image after obtaining the gradient information and the global contrast of the original image; inputting the multi-scale features of the original image and the first thermal feature map corresponding to the original image into the decoder module to obtain the binary image corresponding to the original image output by the decoder module.

2. The thermal infrared sensor calibration method of claim 1, wherein, the step of performing image reconstruction on the binary image corresponding to the original image to obtain the contrast-enhanced image corresponding to the original image comprises the following steps: performing data fusion on the original image and the binary image corresponding to the original image to obtain a first target image; performing local contrast enhancement on the first target image by using an adaptive histogram equalization algorithm to obtain a second target image; performing data fusion on the second target image and the first thermal feature map corresponding to the original image to obtain the contrast-enhanced image corresponding to the original image.

3. The thermal infrared sensor calibration method of claim 1, wherein, the step of performing feature enhancement on visual features of corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image to obtain a feature-enhanced image corresponding to the original image comprises the following steps: extracting multi-scale features of edges of the checkerboard pattern in the contrast-enhanced image corresponding to the original image based on Sobel-Freeman operators and Gaussian Laplacian operators to obtain a multi-scale edge map corresponding to the original image; calculate a local average contrast of a multi-scale edge map corresponding to the original image, and then construct a feature fusion map corresponding to the original image based on the local average contrast of the multi-scale edge map corresponding to the original image and the multi-scale edge map corresponding to the original image; input the contrast-enhanced image corresponding to the original image into the heat boundary perception module, obtain a second heat feature map corresponding to the original image output by the heat boundary perception module, and perform data fusion on the feature fusion map corresponding to the original image and the second heat feature map corresponding to the original image to obtain a feature-enhanced image corresponding to the original image.

4. The thermal infrared sensor calibration method of claim 1, wherein, The identification of the corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image comprises: input the feature-enhanced image corresponding to the original image into a corner point identification model, obtain a corner point probability heat map of the feature-enhanced image corresponding to the original image output by the corner point identification model, and the corner point identification model is constructed based on a high-resolution network model and obtained after training based on the feature-enhanced image corresponding to the sample image and the corner point probability heat map of the feature-enhanced image corresponding to the sample image; extract the position information of the corner points of the checkerboard pattern from the corner point probability heat map of the feature-enhanced image corresponding to the original image using a non-maximum suppression algorithm.

5. The method of calibrating a thermal infrared sensor according to any one of claims 1 to 4, characterized in that, The sample image is obtained based on the following steps: obtain a thermal infrared image of the calibration board collected by a sample thermal infrared sensor as an original sample image, and the sample thermal infrared sensor is a calibrated thermal infrared sensor; after data enhancement is performed on the original sample image, determine the original sample image after data enhancement as the sample image, and the data enhancement includes at least one of adding noise, geometric transformation, and illumination adjustment.

6. A thermal infrared sensor calibration device, characterized by comprise: an image binarization module configured to input an original image into a segmentation model, and obtain a binarized image corresponding to the original image output by the segmentation model, wherein the original image is a thermal infrared image of a calibration board collected by a to-be-calibrated thermal infrared sensor, the calibration board is provided with a checkerboard pattern, and the segmentation model is obtained after being trained based on a sample image and a binarized image corresponding to the sample image, wherein the sample image is a thermal infrared image of the calibration board collected by a sample thermal infrared sensor; an image reconstruction module configured to perform image reconstruction on the binarized image corresponding to the original image to obtain a contrast-enhanced image corresponding to the original image; a feature enhancement module configured to perform feature enhancement on visual features of corner points of the checkerboard pattern in the contrast-enhanced image corresponding to the original image to obtain a feature-enhanced image corresponding to the original image; a parameter calculation module configured to identify the corner points of the checkerboard pattern in the feature-enhanced image corresponding to the original image, and obtain internal and external parameter values of the to-be-calibrated thermal infrared sensor based on position information of the identified corner points; the segmentation model comprises an encoder module, a heat boundary perception module, and a decoder module; the inputting of the original image into the segmentation model and the obtaining of the binarized image output by the segmentation model comprise: The original image is input into the encoder module and the thermal boundary perception module respectively, and the multi-scale features of the original image output by the encoder module and the first thermal feature map corresponding to the original image output by the thermal boundary perception module are obtained; the thermal boundary perception module is used to obtain gradient information and global contrast of the original image, and then the first thermal feature map corresponding to the original image is generated by combining the gradient information and the global contrast of the original image; The multi-scale features of the original image and the first thermal feature map corresponding to the original image are input into the decoder module, and a binarized image corresponding to the original image output by the decoder module is obtained.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the calibration method of the thermal infrared sensor according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the calibration method of the thermal infrared sensor according to any one of claims 1 to 5.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the calibration method of the thermal infrared sensor according to any one of claims 1 to 5.

Images