A method and device for calibrating an infrared camera, an electronic device and a storage medium

Abstract

The application discloses a kind of infrared camera's calibration method, device, equipment and storage medium, comprising: according to local blur degree determination search window, and in search window to initial checkerboard corner point is iteratively optimized, obtain subpixel level checkerboard corner point;According to subpixel level checkerboard corner point is parameterized to the infrared camera to be calibrated.It is enhanced by processing to infrared image, and the discernibility of checkerboard boundary and corner point is promoted, and by calculating the local blur degree of each initial corner point, the blur degree difference of different regions in image is quantitatively characterized, according to the blur degree adaptively determine the search window size of subpixel optimization, avoid the problem that fuzzy corner point optimization fails due to window is too small or background noise is introduced due to window is too large, in non-uniform fuzzy infrared image, high-precision, high-stability subpixel corner point positioning is realized, to significantly improve the robustness and re-projection precision of infrared camera calibration.

Description

Technical Field

[0001] This invention relates to the field of camera calibration technology, and more particularly to a calibration method, apparatus, device, and storage medium for an infrared camera. Background Technology

[0002] Infrared cameras are widely used in industrial non-destructive testing, security monitoring, and intelligent driving. To obtain the spatial geometric information of the target, the infrared camera needs to be accurately calibrated. The planar checkerboard calibration method has become a common approach due to its ease of operation and high accuracy. However, the positioning accuracy of the corner points has a direct impact on the calibration results. To further improve accuracy, sub-pixel optimization is usually performed on the initial position of the corner points.

[0003] Currently, when optimizing the initial position of corner points, in infrared imaging scenarios, due to factors such as lens depth of field, thermal radiation distribution, and environmental reflection, checkerboard images often exhibit a spatially uneven blur state. However, traditional sub-pixel optimization algorithms generally use a fixed-size search window. This "one-size-fits-all" strategy is difficult to adapt to the needs of different blur levels. If the window is too small, the corner points in the blurred area will fail to optimize or converge unstably due to insufficient gradient information. If the window is too large, the corner points in the clear area will introduce too much noise and interference from adjacent edges, resulting in positioning offset, which will affect the calibration accuracy of the subsequent camera. Summary of the Invention

[0004] This invention provides a calibration method, apparatus, device, and storage medium for infrared cameras to achieve accurate calibration of infrared cameras.

[0005] According to a first aspect of the present invention, a calibration method for an infrared camera is provided, the method comprising: acquiring an original checkerboard image captured by the infrared camera to be calibrated, and detecting the original checkerboard image to obtain initial checkerboard corner points; For each initial chessboard corner point, calculate the local sharpness index of the corresponding analysis window, and construct the local blur based on the local sharpness index; The search window is determined based on the local ambiguity, and the initial chessboard corner points are iteratively optimized within the search window to obtain sub-pixel level chessboard corner points; The parameters of the infrared camera to be calibrated are determined based on the sub-pixel level checkerboard corner points.

[0006] According to another aspect of the present invention, a calibration device for an infrared camera is provided, the device comprising: an initial checkerboard corner point acquisition module, configured to acquire an enhanced checkerboard image captured by the infrared camera to be calibrated, and to detect the enhanced checkerboard image to acquire initial checkerboard corner points; The local ambiguity construction module is used to calculate the local sharpness index of the corresponding analysis window for each initial chessboard corner point, and construct the local ambiguity based on the local sharpness index; The chessboard corner point subpixel-level optimization module is used to determine a search window based on the local ambiguity, and to iteratively optimize the initial chessboard corner points within the search window to obtain subpixel-level chessboard corner points. The parameter calibration module is used to calibrate the parameters of the infrared camera to be calibrated based on the sub-pixel level checkerboard corner points.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: one or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any embodiment of the present invention.

[0008] According to another aspect of the present invention, a storage medium for computer-executable instructions is provided, on which a computer program is stored, which, when executed by a processor, implements the method described in any of the embodiments of the present invention.

[0009] The technical solution of this invention enhances the recognizability of checkerboard boundaries and corners by performing enhancement processing on infrared images. It also quantitatively characterizes the difference in blurriness in different regions of the image by calculating the local blurriness of each initial corner. Based on this blurriness, the search window size for subpixel optimization is adaptively determined, avoiding the problems of failed optimization of blurred corners due to an excessively small window or the introduction of background noise due to an excessively large window. This achieves high-precision and high-stability subpixel corner localization in non-uniformly blurred infrared images, thereby significantly improving the robustness and reprojection accuracy of infrared camera calibration.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a calibration method for an infrared camera according to Embodiment 1 of the present invention; Figure 2 This is a flowchart of another infrared camera calibration method provided according to Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the structure of a calibration device for an infrared camera according to Embodiment 3 of the present invention; Figure 4 This is a structural block diagram of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0014] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, apparatus, product, or terminal device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or terminal devices.

[0015] Example 1 Figure 1 This is a flowchart of an infrared camera calibration method provided in Embodiment 1 of the present invention. This embodiment is applicable to the calibration of infrared cameras. The method can be executed by an infrared camera calibration device, which can be implemented in hardware and / or software, and can be integrated into an electronic device with data processing capabilities. Figure 1 As shown, the method includes: S101: Acquire the original checkerboard image captured by the infrared camera to be calibrated, and detect the original checkerboard image to obtain the initial checkerboard corner points.

[0016] Optionally, the process involves acquiring the original checkerboard image captured by the infrared camera to be calibrated, and detecting the original checkerboard image to obtain initial checkerboard corner points. This includes: acquiring the original checkerboard image captured by the infrared camera to be calibrated, and performing local contrast enhancement on the original checkerboard image to obtain an enhanced checkerboard image; and using a specified detection method to detect the enhanced checkerboard image to obtain initial checkerboard corner points. The specified detection method may include checkerboard corner point detection, coded checkerboard detection, or edge intersection detection.

[0017] Optionally, the original checkerboard image is enhanced by local contrast enhancement, including: dividing the original checkerboard image into multiple local sub-blocks and calculating the gray-level histogram of each local sub-block; cropping the gray-level counts exceeding the gray-level threshold in the gray-level histogram of each local sub-block and redistributing the cropping amount to each gray level to obtain the redistributed gray-level histogram; calculating the normalized cumulative distribution function for the redistributed gray-level histogram and establishing a local gray-level mapping; and eliminating the gray-level discontinuities between adjacent local sub-blocks through bilinear interpolation based on the local gray-level mapping to obtain the enhanced checkerboard image.

[0018] Specifically, in this embodiment, the original checkerboard image captured by the infrared camera to be calibrated is acquired, denoted as I(x,y). The original checkerboard image can be a single-channel infrared grayscale image or a multi-channel infrared image formed after pseudo-color mapping or other image processing. When the input image is a multi-channel image, it needs to be converted to a single-channel grayscale image before detecting the original checkerboard image. This is because the format of a single-channel image is simpler than that of a multi-channel image, thus avoiding interference from color information on corner detection. Furthermore, addressing the common problems of low contrast and blurred details in infrared images, this embodiment performs local contrast enhancement on the single-channel checkerboard image to obtain an enhanced checkerboard image, denoted as I'(x,y). For example, this embodiment can employ an advanced image processing method with limited contrast adaptive histogram equalization to enhance the contrast of a single-channel checkerboard image. Specifically, I(x,y) is divided into multiple local sub-blocks, and the corresponding grayscale histogram is obtained separately for each local sub-block. If there are too many pixels of a certain grayscale value, exceeding a preset grayscale threshold, the excess grayscale counts are cropped, and the cropped amount is redistributed evenly to other grayscale levels to obtain a reassigned grayscale histogram. This cropping prevents excessive amplification of noise in the image. Based on the reassigned grayscale histogram, i.e., the cropped distribution, a transformation formula, such as a normalized cumulative distribution function, is calculated to map the original grayscale values ​​to new, higher-contrast grayscale values. Furthermore, since each local sub-block is processed independently, there may be abrupt breaks at the edges when they are pieced together. This embodiment uses bilinear interpolation to smoothly transition the boundaries of adjacent local sub-blocks to obtain an enhanced checkerboard image, resulting in a more natural and smooth overall enhanced checkerboard image.

[0019] In addition, this embodiment employs a specified detection method to detect the enhanced checkerboard image I'(x,y) to obtain the initial checkerboard corner points, for example, the position of the i-th initial corner point. Furthermore, the aforementioned specified detection methods may specifically include checkerboard corner detection, coded checkerboard detection, or edge intersection detection. Checkerboard corner detection refers to the conventional method of finding corners by utilizing the alternating pattern of black and white blocks. Coded checkerboard detection refers to the method of accurately locating corners by decoding a calibration board with specific codes, such as AprilTag or ArUco, instead of using a regular checkerboard. Edge intersection detection refers to the method of first extracting straight edges from the image and then calculating the intersections of these horizontal and vertical edges as corners. Of course, this embodiment is only an example and does not limit the specific method of corner detection.

[0020] S102, calculate the local sharpness index of the corresponding analysis window for each initial chessboard corner point, and construct the local blur based on the local sharpness index.

[0021] Optionally, for each initial checkerboard corner point, a local sharpness index is calculated for the corresponding analysis window, and a local blur is constructed based on the local sharpness index. This includes: performing Laplacian calculation on the enhanced checkerboard image to obtain a response map; constructing a corresponding analysis window centered on each initial checkerboard corner point, and calculating a local sharpness index based on the response map within the analysis window, wherein the local sharpness index includes the Laplacian response mean and the Laplacian response variance; calculating the initial blur based on the Laplacian response variance, and normalizing the initial blur based on the minimum and maximum blur in the enhanced checkerboard image to obtain the local blur.

[0022] Specifically, in this embodiment, a Laplacian calculation is performed on the enhanced checkerboard image I'(x,y) obtained above to obtain the response map. Additionally, an analysis window will be constructed centered on each initial chessboard corner point. For example, for each initial corner point... A corresponding local analysis window was constructed around the center. The side length of the local analysis window can be 7 pixels, and this side length can be flexibly adjusted between 3 and 15 pixels. The actual size can be determined based on the resolution of the infrared camera and the size of the checkerboard grid; this embodiment does not limit it. For each initial checkerboard corner point, a local sharpness index is calculated based on the response map within the corresponding analysis window. For example, for the initial corner point... Calculate the local analysis window Mean Laplace response within and Laplace response variance Furthermore, a larger Laplace response variance indicates a higher initial corner point. The clearer the neighborhood, the better.

[0023] Specifically, this implementation calculates the initial ambiguity based on the Laplace response variance mentioned above. For example, the initial ambiguity corresponding to the i-th initial corner point is... ,in, To prevent the introduction of tiny positive numbers due to a denominator of zero, this embodiment also obtains the minimum value of the blurring of all initial corner points in the enhanced checkerboard image I'(x,y). and maximum value and based on the minimum value and maximum value The initial ambiguity is normalized to obtain the local ambiguity. For example, the local ambiguity for the i-th initial corner point is... When the minimum value and maximum value When they are equal or nearly equal, that is, when the blurring degree of all initial corner points in the enhanced chessboard image is exactly the same, it will... Set to a preset value or degenerate to a fixed search radius.

[0024] S103, determine the search window based on the local ambiguity, and iteratively optimize the initial chessboard corner points within the search window to obtain sub-pixel level chessboard corner points.

[0025] Optionally, a search window is determined based on local ambiguity, and the initial chessboard corner points are iteratively optimized within the search window to obtain sub-pixel level chessboard corner points. This includes: calculating an adaptive search radius for each initial chessboard corner point based on local ambiguity and pre-set minimum and maximum search radii, where the search radius is positively correlated with local ambiguity; using the initial chessboard corner points as the center and the range defined by the search radius as the search window, calculating the image gradient of each pixel within the search window, and constructing a gradient structure matrix and a target vector based on the image gradient; correcting the initial chessboard corner points based on the gradient structure matrix and the target vector to obtain updated chessboard corner points; calculating the coordinate correction amount corresponding to the updated chessboard corner points, and when the coordinate correction amount is less than a preset threshold or the current iteration number reaches the maximum iteration number, the updated chessboard corner points are taken as sub-pixel level chessboard corner points.

[0026] Specifically, in this embodiment, the search window for each initial chessboard corner point is determined based on local ambiguity, since a minimum search radius is preset. and maximum search radius Therefore, for the i-th initial corner point, the corresponding search radius is Therefore, when the local ambiguity of each initial corner point is different, the corresponding search radius will also be different, and the size of the search radius is positively correlated with the local ambiguity. Furthermore, the search radius of each initial corner point must be strictly limited to a certain value. Within a certain range, to prevent excessively large or small extreme values, when When the value is close to 0, local areas are clearer. near ;when When the value approaches 1, local areas become blurry. near .For example, 3 to 11 pixels are possible. It can take 4 to 15 pixels, and Greater than Optional in lightly blurred scenes =5、 =6; optional in medium blur scenes. =7、 =8; optional in heavily blurred scenes =9、 =10; In light, medium, and heavy blur mixed scenes, the radius range that can cover multiple blur levels can be selected, for example... =5、 =10, of course, this embodiment is only an example and does not represent the actual implementation. and The specific value to be taken is limited.

[0027] Among them, for the initial corner point For example, starting with the corner points of the initial chessboard. Centered on search radius The defined range serves as the search window, such as In the search window Inner corner point Iterative optimization is performed to obtain sub-pixel level checkerboard corner points. For example, for windows Each pixel Calculate image gradient ,in These are the partial derivatives of the enhanced image I'(x,y) in the horizontal and vertical directions, respectively. A gradient structure matrix is ​​constructed based on the image gradient. and target vector When the gradient structure matrix When it is invertible, then according to the target vector and gradient structure matrix Correct the initial chessboard corner points to obtain the updated chessboard corner points. Additionally, it will calculate and update the coordinate corrections corresponding to the corner points of the chessboard. When coordinate correction amount If the number of iterations is less than the preset threshold or the current iteration count reaches the maximum iteration count, the corner points of the chessboard will be updated. As a subpixel-level chessboard corner point For example, the preset threshold τ is 0.01 pixels, and the maximum number of iterations is... The iteration is repeated 30 times. This is merely an example in this implementation and does not specify the exact conditions for stopping the iteration. Furthermore, the determination of the sub-pixel level checkerboard corner points is based on the gradient structure matrix. In the case of reversibility, if G is not reversible, the previous position will be maintained or the window will be reduced before recalculation to avoid calculating incorrect results.

[0028] S104 calibrates the parameters of the infrared camera to be calibrated based on the sub-pixel checkerboard corner points.

[0029] Optionally, parameter calibration of the infrared camera to be calibrated is performed based on the sub-pixel checkerboard corner points, including: obtaining the actual checkerboard corner points corresponding to the sub-pixel checkerboard corner points; solving for the specified parameters of the infrared camera to be calibrated based on the relationship between the sub-pixel checkerboard corner points and the actual checkerboard corner points, wherein the specified parameters include the infrared camera's intrinsic parameters, extrinsic parameters, and distortion parameters; and calibrating the infrared camera to be calibrated based on the specified parameters.

[0030] Specifically, this embodiment will not only acquire the aforementioned sub-pixel level checkerboard corner points, (Precise pixel coordinates on the photo), and also obtain the actual corner points of the chessboard. (The physical dimensions and three-dimensional spatial coordinates of the chessboard in reality), and obtain and This establishes a mapping bridge between the real world and the camera sensor. In this embodiment, based on the aforementioned relationship, data algorithms, such as the Zhang Zhengyou calibration method, are used to reverse-engineer the specified parameters of the infrared camera to be calibrated. These specified parameters include intrinsic parameters, extrinsic parameters, and distortion parameters of the infrared camera. Intrinsic parameters include the camera's focal length, principal point coordinates, and other imaging characteristics representing the camera itself. Extrinsic parameters include the camera's spatial position and rotation angle relative to the checkerboard pattern. Distortion parameters include optical defects in the lens itself, such as barrel or pincushion distortion common in infrared lenses. Of course, this embodiment is merely illustrative and does not limit the specific types of the specified parameters.

[0031] The technical solution of this invention enhances the recognizability of checkerboard boundaries and corners by performing enhancement processing on infrared images. It also quantitatively characterizes the difference in blurriness in different regions of the image by calculating the local blurriness of each initial corner. Based on this blurriness, the search window size for subpixel optimization is adaptively determined, avoiding the problems of failed optimization of blurred corners due to an excessively small window or the introduction of background noise due to an excessively large window. This achieves high-precision and high-stability subpixel corner localization in non-uniformly blurred infrared images, thereby significantly improving the robustness and reprojection accuracy of infrared camera calibration.

[0032] Example 2 Figure 2 This is a flowchart of another infrared camera calibration method provided by an embodiment of the present invention. Based on the above embodiment, after calibrating the parameters of the infrared camera to be calibrated according to the sub-pixel level checkerboard corner points, it further includes: detecting the calibration result of the infrared camera to be calibrated, generating a detection report, generating a calibration error message when it is determined from the detection report that the calibration result is incorrect, and visually displaying the calibration error message, such as... Figure 2 As shown, the method includes: S201, acquire the original chessboard image captured by the infrared camera to be calibrated, and detect the original chessboard image to obtain the initial chessboard corner points.

[0033] Optionally, the process involves acquiring the original checkerboard image captured by the infrared camera to be calibrated, and detecting the original checkerboard image to obtain initial checkerboard corner points. This includes: acquiring the original checkerboard image captured by the infrared camera to be calibrated; performing local contrast enhancement on the original checkerboard image to obtain an enhanced checkerboard image; and using a specified detection method to detect the enhanced checkerboard image to obtain initial checkerboard corner points. The specified detection method may include checkerboard corner point detection, coded checkerboard detection, or edge intersection detection.

[0034] Optionally, the original checkerboard image is enhanced by local contrast enhancement, including: dividing the original checkerboard image into multiple local sub-blocks and calculating the gray-level histogram of each local sub-block; cropping the gray-level counts exceeding the gray-level threshold in the gray-level histogram of each local sub-block and redistributing the cropping amount to each gray level to obtain the redistributed gray-level histogram; calculating the normalized cumulative distribution function for the redistributed gray-level histogram and establishing a local gray-level mapping; and eliminating the gray-level discontinuities between adjacent local sub-blocks through bilinear interpolation based on the local gray-level mapping to obtain the enhanced checkerboard image.

[0035] S202, calculate the local sharpness index of the corresponding analysis window for each initial chessboard corner point, and construct the local blur based on the local sharpness index.

[0036] Optionally, for each initial checkerboard corner point, a local sharpness index is calculated for the corresponding analysis window, and a local blur is constructed based on the local sharpness index. This includes: performing Laplacian calculation on the enhanced checkerboard image to obtain a response map; constructing a corresponding analysis window centered on each initial checkerboard corner point, and calculating a local sharpness index based on the response map within the analysis window, wherein the local sharpness index includes the Laplacian response mean and the Laplacian response variance; calculating the initial blur based on the Laplacian response variance, and normalizing the initial blur based on the minimum and maximum blur in the enhanced checkerboard image to obtain the local blur.

[0037] S203, determine the search window based on the local ambiguity, and iteratively optimize the initial chessboard corner points within the search window to obtain sub-pixel level chessboard corner points.

[0038] Optionally, a search window is determined based on local ambiguity, and the initial chessboard corner points are iteratively optimized within the search window to obtain sub-pixel level chessboard corner points. This includes: calculating an adaptive search radius for each initial chessboard corner point based on local ambiguity and pre-set minimum and maximum search radii, where the search radius is positively correlated with local ambiguity; using the initial chessboard corner points as the center and the range defined by the search radius as the search window, calculating the image gradient of each pixel within the search window, and constructing a gradient structure matrix and a target vector based on the image gradient; correcting the initial chessboard corner points based on the gradient structure matrix and the target vector to obtain updated chessboard corner points; calculating the coordinate correction amount corresponding to the updated chessboard corner points, and when the coordinate correction amount is less than a preset threshold or the current iteration number reaches the maximum iteration number, the updated chessboard corner points are taken as sub-pixel level chessboard corner points.

[0039] S204 calibrates the parameters of the infrared camera to be calibrated based on the sub-pixel checkerboard corner points.

[0040] Optionally, parameter calibration of the infrared camera to be calibrated is performed based on the sub-pixel checkerboard corner points, including: obtaining the actual checkerboard corner points corresponding to the sub-pixel checkerboard corner points; solving for the specified parameters of the infrared camera to be calibrated based on the relationship between the sub-pixel checkerboard corner points and the actual checkerboard corner points, wherein the specified parameters include the infrared camera's intrinsic parameters, extrinsic parameters, and distortion parameters; and calibrating the infrared camera to be calibrated based on the specified parameters.

[0041] S205 checks the calibration results of the infrared camera to be calibrated, generates a test report, and generates a calibration error message when it is determined from the test report that the calibration results are incorrect, and then displays the calibration error message visually.

[0042] In this implementation, after calibrating the infrared camera to be calibrated, the calibration results are checked. This check involves three main parameters: reprojection error calculation, parameter rationality verification, and data integrity check. Reprojection error calculation involves substituting the previously calibrated "specified parameters" (intrinsic, extrinsic, and distortion) into the model, reprojecting the real-world checkerboard corner points back onto the image plane, and calculating the distance between the "theoretical projection point" and the "actually detected sub-pixel corner point." The average of this distance is the reprojection error. If the calculated reprojection error exceeds a set threshold, the calibration is considered a failure. Parameter rationality verification checks whether the calculated parameters conform to physical laws. For example, whether the focal length is positive, whether the distortion coefficient is within a reasonable range, and whether there are extreme numerical jumps. Data integrity check confirms whether the number of corner points used for calibration is sufficient and whether they cover the center and edge areas of the image, preventing calibration deviations caused by uneven sample distribution.

[0043] It should be noted that in this embodiment, a test report will be generated based on the above test results. The test report contains specific test parameters and corresponding test results. When it is determined from the test report that the calibration result is incorrect, a calibration error prompt will be generated and displayed visually, for example, as a pop-up window, to prompt the operator to handle it in a timely manner.

[0044] The technical solution of this invention enhances the recognizability of checkerboard boundaries and corners by performing enhancement processing on infrared images. It also quantitatively characterizes the difference in blurriness in different regions of the image by calculating the local blurriness of each initial corner. Based on this blurriness, the search window size for subpixel optimization is adaptively determined, avoiding the problems of failed optimization of blurred corners due to an excessively small window or the introduction of background noise due to an excessively large window. This achieves high-precision and high-stability subpixel corner localization in non-uniformly blurred infrared images, thereby significantly improving the robustness and reprojection accuracy of infrared camera calibration.

[0045] Example 3 Figure 3 This is a schematic diagram of the structure of an infrared camera calibration device provided in an embodiment of the present invention. Figure 3 As shown, the device includes: an initial chessboard corner point acquisition module 310, a local ambiguity construction module 320, a chessboard corner point sub-pixel level optimization module 330, and a parameter calibration module 340.

[0046] The initial chessboard corner point acquisition module 310 is used to acquire the original chessboard image captured by the infrared camera to be calibrated, and to detect the original chessboard image to acquire the initial chessboard corner points. The local ambiguity construction module 320 is used to calculate the local sharpness index of the corresponding analysis window for each initial chessboard corner point, and construct the local ambiguity based on the local sharpness index; The checkerboard corner point subpixel-level optimization module 330 is used to determine the search window based on the local ambiguity, and to iteratively optimize the initial checkerboard corner points within the search window to obtain subpixel-level checkerboard corner points. The parameter calibration module 340 is used to calibrate the parameters of the infrared camera to be calibrated based on the sub-pixel level checkerboard corner points.

[0047] Optionally, the initial checkerboard corner point acquisition module 310 is used to acquire the original checkerboard image captured by the infrared camera to be calibrated, and to perform local contrast enhancement on the original checkerboard image to obtain an enhanced checkerboard image. The enhanced checkerboard image is detected using a specified detection method to obtain the initial checkerboard corner points. The specified detection method includes checkerboard corner point detection, coded checkerboard detection, or edge intersection point detection.

[0048] Optionally, the initial chessboard corner point acquisition module includes an image enhancement unit, which is used to divide the original chessboard image into multiple local sub-blocks and calculate the grayscale histogram of each local sub-block respectively. For each local sub-block, the grayscale counts exceeding the grayscale threshold in the grayscale histogram are cropped, and the cropping amount is redistributed to each grayscale level to obtain the redistributed grayscale histogram; A normalized cumulative distribution function is calculated for the redistributed gray-level histogram, and a local gray-level mapping is established. Based on the local gray-level mapping, bilinear interpolation is used to eliminate gray-level discontinuities between adjacent local sub-blocks to obtain an enhanced checkerboard image.

[0049] Optionally, a local ambiguity construction module 320 is used to perform Laplacian calculations on the enhanced checkerboard image to obtain a response map; An analysis window is constructed with each initial chessboard corner as the center, and the local sharpness index is calculated based on the response map within the analysis window. The local sharpness index includes the Laplace response mean and the Laplace response variance. The initial ambiguity is calculated based on the Laplace response variance, and the initial ambiguity is normalized based on the minimum and maximum ambiguities in the enhanced checkerboard image to obtain the local ambiguity.

[0050] Optionally, a checkerboard corner point subpixel-level optimization module 330 is used to calculate an adaptive search radius for each initial checkerboard corner point based on the local ambiguity and the preset minimum and maximum search radii, wherein the size of the search radius is positively correlated with the local ambiguity. Using the initial chessboard corner points as the center and the range defined by the search radius as the search window, the image gradient of each pixel is calculated within the search window, and the gradient structure matrix and target vector are constructed based on the image gradient. The initial chessboard corner points are corrected based on the gradient structure matrix and the target vector to obtain the updated chessboard corner points; Calculate the coordinate correction amount corresponding to the updated chessboard corner point. When the coordinate correction amount is less than the preset threshold or the current iteration number reaches the maximum iteration number, the updated chessboard corner point is treated as a sub-pixel level chessboard corner point.

[0051] Optionally, the parameter calibration module 340 is used to obtain the actual chessboard corner points corresponding to the sub-pixel level chessboard corner points; The specified parameters of the infrared camera to be calibrated are solved based on the relationship between the sub-pixel level checkerboard corner points and the actual checkerboard corner points. The specified parameters include the infrared camera intrinsic parameters, infrared camera extrinsic parameters, and distortion parameters. Perform parameter calibration on the infrared camera to be calibrated according to the specified parameters.

[0052] Optionally, the device also includes a calibration result detection module, which is used to detect the calibration results of the infrared camera to be calibrated and generate a detection report; When the calibration result is determined to be incorrect based on the test report, a calibration error message is generated and displayed visually.

[0053] The infrared camera calibration device provided in this embodiment of the invention can execute the infrared camera calibration method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0054] Example 4 Figure 4 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0055] The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0056] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0057] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other electronic devices through computer networks such as the Internet and / or various telecommunications networks.

[0058] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the calibration method for an infrared camera.

[0059] That is, acquire the enhanced checkerboard image captured by the infrared camera to be calibrated, and detect the corner points of the initial checkerboard in the enhanced checkerboard image; For each initial chessboard corner point, calculate the local sharpness index of the corresponding analysis window, and construct the local blur based on the local sharpness index; The search window is determined based on the local ambiguity, and the initial chessboard corner points are iteratively optimized within the search window to obtain sub-pixel level chessboard corner points. The parameters of the infrared camera to be calibrated are determined based on the sub-pixel level checkerboard corner points.

[0060] In some embodiments, the infrared camera calibration method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the infrared camera calibration method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the infrared camera calibration method by any other suitable means (e.g., by means of firmware).

[0061] Various embodiments of the apparatuses and techniques described above herein can be implemented in digital electronic circuit devices, integrated circuit devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), device-on-a-chip (SoC) devices, complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable device including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage device, at least one input device, and at least one output device, and transmitting data and instructions to the storage device, the at least one input device, and the at least one output device.

[0062] The computer program used to implement the calibration method for the infrared camera of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer or a special-purpose computer, such that when executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program can be executed entirely on the machine, partially on the machine, or as a standalone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0063] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution apparatus, device, or electronic device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage electronics, magnetic storage electronics, or any suitable combination thereof.

[0064] To provide interaction with a user, the devices and techniques described herein can be implemented on an electronic device having: a display device (e.g., a touchscreen) for displaying information to the user; and buttons through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0065] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0066] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A calibration method for an infrared camera, characterized in that, The method includes: Acquire the original chessboard image captured by the infrared camera to be calibrated, and detect the original chessboard image to obtain the initial chessboard corner points; For each initial chessboard corner point, calculate the local sharpness index of the corresponding analysis window, and construct the local blur based on the local sharpness index; The search window is determined based on the local ambiguity, and the initial chessboard corner points are iteratively optimized within the search window to obtain sub-pixel level chessboard corner points; The parameters of the infrared camera to be calibrated are determined based on the sub-pixel level checkerboard corner points.

2. The method according to claim 1, characterized in that, The process of acquiring the original checkerboard image captured by the infrared camera to be calibrated, and detecting the initial checkerboard corner points from the original checkerboard image, includes: Acquire the original chessboard image captured by the infrared camera to be calibrated, and perform local contrast enhancement on the original chessboard image to obtain an enhanced chessboard image. The enhanced checkerboard image is detected using a specified detection method to obtain the initial checkerboard corner points, wherein the specified detection method includes checkerboard corner point detection, coded checkerboard detection, or edge intersection point detection.

3. The method according to claim 2, characterized in that, The step of performing local contrast enhancement on the original chessboard image to obtain the enhanced chessboard image includes: The original chessboard image is divided into multiple local sub-blocks, and the grayscale histogram of each local sub-block is calculated. For each local sub-block, the grayscale counts exceeding the grayscale threshold in the grayscale histogram are cropped, and the cropping amount is redistributed to each grayscale level to obtain the redistributed grayscale histogram; A normalized cumulative distribution function is calculated for the redistributed grayscale histogram, and a local grayscale mapping is established. Based on the local grayscale mapping, bilinear interpolation is used to eliminate grayscale discontinuities between adjacent local sub-blocks to obtain the enhanced checkerboard image.

4. The method according to claim 1, characterized in that, The calculation of the local sharpness index for each initial chessboard corner point corresponding to the analysis window, and the construction of local blur based on the local sharpness index, includes: The enhanced checkerboard image is subjected to Laplacian calculation to obtain the response map; An analysis window is constructed centered on each of the initial chessboard corner points, and the local sharpness index is calculated based on the response map within the analysis window. The local sharpness index includes the Laplace response mean and the Laplace response variance. The initial ambiguity is calculated based on the Laplace response variance, and the initial ambiguity is normalized based on the minimum and maximum ambiguities in the enhanced checkerboard image to obtain the local ambiguity.

5. The method according to claim 4, characterized in that, The step of determining a search window based on the local ambiguity and iteratively optimizing the initial chessboard corner points within the search window to obtain sub-pixel level chessboard corner points includes: For each initial chessboard corner point, an adaptive search radius is calculated based on the local ambiguity and the preset minimum and maximum search radii, wherein the size of the search radius is positively correlated with the local ambiguity; The search window is defined by the range of the initial chessboard corner points and the search radius. The image gradient of each pixel is calculated within the search window, and a gradient structure matrix and a target vector are constructed based on the image gradient. The initial chessboard corner points are corrected based on the gradient structure matrix and the target vector to obtain the updated chessboard corner points; Calculate the coordinate correction amount corresponding to the updated chessboard corner point. When the coordinate correction amount is less than a preset threshold or the current iteration number reaches the maximum iteration number, the updated chessboard corner point is taken as the sub-pixel level chessboard corner point.

6. The method according to claim 1, characterized in that, The parameter calibration of the infrared camera to be calibrated based on the sub-pixel level checkerboard corner points includes: Obtain the actual chessboard corner point corresponding to the sub-pixel level chessboard corner point; The specified parameters of the infrared camera to be calibrated are solved based on the relationship between the sub-pixel level checkerboard corner points and the actual checkerboard corner points. The specified parameters include the infrared camera intrinsic parameters, the infrared camera extrinsic parameters, and the distortion parameters. The infrared camera to be calibrated is calibrated according to the specified parameters.

7. The method according to claim 1, characterized in that, After calibrating the parameters of the infrared camera to be calibrated based on the sub-pixel checkerboard corner points, the method further includes: The calibration results of the infrared camera to be calibrated are detected, and a detection report is generated; When the calibration result is determined to be incorrect based on the test report, a calibration error message is generated and the calibration error message is displayed visually.

8. A calibration device for an infrared camera, characterized in that, The device includes: The initial chessboard corner point acquisition module is used to acquire the original chessboard image captured by the infrared camera to be calibrated, and to detect the original chessboard image to acquire the initial chessboard corner points. The local ambiguity construction module is used to calculate the local sharpness index of the corresponding analysis window for each initial chessboard corner point, and construct the local ambiguity based on the local sharpness index; The chessboard corner point subpixel-level optimization module is used to determine a search window based on the local ambiguity, and to iteratively optimize the initial chessboard corner points within the search window to obtain subpixel-level chessboard corner points. The parameter calibration module is used to calibrate the parameters of the infrared camera to be calibrated based on the sub-pixel level checkerboard corner points.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images