Dual-light fusion method and device and thermal imager

Abstract

The invention provides a dual-light fusion method, a dual-light fusion device and a thermal imager. The method comprises the following steps: segmenting a dual-light fusion object distance range according to dual-light fusion pixel deviation between a thermal imaging image and a visible light image under different object distances to obtain at least two reference object distance ranges; further determining an optimal zoom coefficient corresponding to the reference object distance range by using zoom coefficients corresponding to different object distances in each reference object distance range, and determining a dual-light fusion calibration parameter corresponding to the reference object distance range according to the optimal zoom coefficient; according to the technical scheme of the invention, different optimal zoom coefficients and dual-light fusion calibration parameters are configured for different reference object distance ranges, so that the thermal imaging image and the visible light image which are acquired under any object distance within the reference object distance range can be subjected to dual-light fusion according to the dual-light fusion calibration parameters corresponding to the reference object distance range. The dual-light fusion pixel deviation occurring in the fusion process under each object distance is effectively reduced, and the dual-light fusion effect is improved.

Description

Technical Field

[0001] This application relates to the field of dual - light fusion technology, and particularly to a dual - light fusion method, apparatus, and thermal imager. Background Art

[0002] In some thermal imaging devices such as thermal imagers, visible - light images and infrared images are usually used for dual - light fusion. Since when focusing on different object distances, the focal lengths and field - of - view angles of the lens for collecting visible - light images and the lens for collecting infrared images will also change. For example, when the object is at a relatively far distance, the field - of - view angle of the lens is small and the focal length is long; while at a short distance, the field - of - view angle is large and the focal length is short. If a unified scaling factor is used at different object distances, it may cause a large deviation in dual - light fusion pixels in the dual - light fusion image, affecting the fusion effect. Summary of the Invention

[0003] In view of this, this application provides a dual - light fusion method, apparatus, and thermal imager to improve the dual - light fusion effect.

[0004] The technical solutions provided by this application are as follows:

[0005] According to an embodiment of the first aspect of this application, a dual - light fusion method is provided. This method is applied to a thermal imager and includes:

[0006] Segment the dual - light fusion object - distance range according to the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image at different object distances, to obtain at least two reference object - distance ranges;

[0007] For each reference object - distance range, based on the thermal imaging image and the visible - light image collected by the thermal imager at different object distances within this reference object - distance range, determine the scaling factor corresponding to different object distances within this reference object - distance range;

[0008] Use the scaling factors corresponding to different object distances within this reference object - distance range to determine the optimal scaling factor corresponding to this reference object - distance range, and determine the dual - light fusion calibration parameters corresponding to this reference object - distance range according to the optimal scaling factor; wherein, the dual - light fusion calibration parameters corresponding to any reference object - distance range are used to perform dual - light fusion on the thermal imaging image and the visible - light image collected by the thermal imager at any object distance within this reference object - distance range.

[0009] Optionally, the segmenting the dual - light fusion object - distance range according to the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image at different object distances, to obtain at least two reference object - distance ranges includes:

[0010] Perform dual - light fusion processing on the thermal imaging images and visible - light images collected at different object distances within the dual - light fusion object distance range based on a preset scaling factor, and obtain the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image after dual - light fusion processing at each object distance.

[0011] Divide the dual - light fusion object distance range according to the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image at each object distance to obtain at least two reference object distance ranges. Among them, the difference between the dual - light fusion pixel deviation at the farthest object distance and the dual - light fusion pixel deviation at the nearest object distance within each reference object distance range is less than a preset threshold, so that each object distance within the reference object distance range corresponds to the same optimal scaling factor; the preset threshold is used to represent the maximum change amount of the allowable dual - light fusion pixel deviation within the reference object distance range.

[0012] Optionally, the method of using the scaling factors corresponding to different object distances within the reference object distance range to determine the optimal scaling factor corresponding to the reference object distance range includes:

[0013] For the scaling factor corresponding to each object distance within the reference object distance range, perform dual - light fusion processing on the thermal imaging images and visible - light images collected at different object distances within the reference object distance range by using this scaling factor, and obtain the error information corresponding to the scaling factor; the error information includes: the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image after dual - light fusion processing for the thermal imaging images and visible - light images collected at each object distance.

[0014] Determine the optimal scaling factor corresponding to the reference object distance range according to the error information corresponding to each scaling factor.

[0015] Optionally, the method of determining the optimal scaling factor corresponding to the reference object distance range according to the error information corresponding to each scaling factor includes:

[0016] Perform a specified operation on each dual - light fusion pixel deviation in the error information corresponding to each scaling factor to obtain the error result corresponding to each scaling factor.

[0017] According to the error result corresponding to each scaling factor, determine the optimal scaling factor corresponding to the reference object distance range from the various scaling factors, and the error result corresponding to the optimal scaling factor meets the set dual - light fusion pixel deviation requirement.

[0018] Optionally, the dual - light fusion calibration parameters include the optimal scaling factor corresponding to the reference object distance range and the offset coefficients. The offset coefficients include the horizontal offset coefficient, the vertical offset coefficient, the calibrated horizontal offset amount, and the calibrated vertical offset amount; the method of determining the dual - light fusion calibration parameters corresponding to the reference object distance range according to the optimal scaling factor includes:

[0019] Scale the thermal imaging images and / or visible light images collected by the thermal imager at different object distances within the reference object distance range according to the optimal scaling factor corresponding to the reference object distance range, and determine the offset between the thermal imaging images and the visible light images after scaling at each object distance; the offset includes a horizontal offset and a vertical offset;

[0020] Fit the horizontal offsets between the thermal imaging images and the visible light images at different object distances to obtain the horizontal offset coefficient corresponding to the reference object distance range, and the horizontal offset coefficient is used to indicate the relationship between the object distance and the horizontal offset within the reference object distance range;

[0021] Fit the vertical offsets between the thermal imaging images and the visible light images at different object distances to obtain the vertical offset coefficient corresponding to the reference object distance range, and the vertical offset coefficient is used to indicate the relationship between the object distance and the vertical offset within the reference object distance range;

[0022] Determine the calibrated horizontal offset and the calibrated vertical offset corresponding to the reference object distance range according to the optimal scaling factor corresponding to the reference object distance range and the object distance corresponding to the optimal scaling factor;

[0023] Use the optimal scaling factor, the horizontal offset coefficient, the vertical offset coefficient, the calibrated horizontal offset, and the calibrated vertical offset as the dual - light fusion calibration parameters corresponding to the reference object distance range.

[0024] According to an embodiment of the second aspect of the present application, a dual - light fusion method is provided. The method is applied to a thermal imager, and the method includes:

[0025] Determine the target object distance range to which the reference distance belongs from the obtained multiple reference object distance ranges according to the reference distance between the target object and the thermal imager; wherein, the multiple reference object distance ranges are obtained by segmenting the dual - light fusion object distance range according to the dual - light fusion pixel deviation between the thermal imaging image and the visible light image at different object distances, different reference object distance ranges correspond to different dual - light fusion calibration parameters, the dual - light fusion calibration parameters are determined according to the optimal scaling factor corresponding to the reference object distance range, and the optimal scaling factor corresponding to the reference object distance range is determined according to the scaling factors corresponding to different object distances within the reference object distance range;

[0026] Perform dual - light fusion on the infrared image and the visible light image of the target object collected by the thermal imager according to the reference distance, the optimal scaling factor corresponding to the target object distance range, and the dual - light fusion calibration parameters.

[0027] Optionally, the reference distance between the target object and the thermal imager is determined by the following method:

[0028] Determine a reference distance between the target object and the thermal imager according to the focusing condition when the thermal imaging component and / or visible light component included in the thermal imager performs image acquisition on the target object;

[0029] Alternatively, the thermal imager further includes a ranging component, and determine the reference distance between the target object and the thermal imager according to the ranging component.

[0030] According to an embodiment of the third aspect of the present application, there is provided a dual - light fusion device, which is applied to a thermal imager, and the device includes:

[0031] A segmentation unit, configured to segment the dual - light fusion object distance range according to the dual - light fusion pixel deviation between the thermal imaging image and the visible light image at different object distances, to obtain at least two reference object distance ranges;

[0032] A coefficient determination unit, configured to, for each reference object distance range, determine the scaling coefficients corresponding to different object distances within the reference object distance range based on the thermal imaging image and the visible light image collected by the thermal imager at different object distances within the reference object distance range;

[0033] A calibration unit, configured to use the scaling coefficients corresponding to different object distances within the reference object distance range to determine the optimal scaling coefficient corresponding to the reference object distance range, and determine the dual - light fusion calibration parameters corresponding to the reference object distance range according to the optimal scaling coefficient; wherein, the dual - light fusion calibration parameters corresponding to any reference object distance range are used to perform dual - light fusion on the thermal imaging image and the visible light image collected by the thermal imager at any object distance within the reference object distance range.

[0034] According to an embodiment of the fourth aspect of the present application, there is provided a dual - light fusion device, which is applied to a thermal imager, and the device includes:

[0035] An object distance range determination unit, configured to determine the target object distance range to which the reference distance belongs from the obtained multiple reference object distance ranges according to the reference distance between the target object and the thermal imager; wherein, the multiple reference object distance ranges are obtained by segmenting the dual - light fusion object distance range according to the dual - light fusion pixel deviation between the thermal imaging image and the visible light image at different object distances, different reference object distance ranges correspond to different dual - light fusion calibration parameters, the dual - light fusion calibration parameters are determined according to the optimal scaling coefficient corresponding to the reference object distance range, and the optimal scaling coefficient corresponding to the reference object distance range is determined according to the scaling coefficients corresponding to different object distances within the reference object distance range;

[0036] A fusion unit, configured to perform dual - light fusion on the infrared image and the visible light image of the target object collected by the thermal imager according to the reference distance, the optimal scaling coefficient corresponding to the target object distance range, and the dual - light fusion calibration parameters.

[0037] According to an embodiment of the fifth aspect of the present application, a thermal imager is provided, and the thermal imager includes:

[0038] An image collector, including a thermal imaging component and a visible light component, where the thermal imaging component is used to collect infrared images at various object distances, and the visible light component is used to collect visible light images at various object distances;

[0039] A memory, which is used to store infrared images, visible light images, the best scaling factor corresponding to each reference object distance range, and dual-light fusion calibration parameters;

[0040] An image display device, which is used to display the infrared image, the visible light image, and the fused image obtained after dual-light fusion processing;

[0041] A processor, which is used to execute the method described in the first aspect or the second aspect.

[0042] As can be seen from the above technical solutions, the present application segments the dual-light fusion object distance range according to the dual-light fusion pixel deviation between the thermal imaging image and the visible light image at different object distances, obtains at least two reference object distance ranges, further determines the best scaling factor corresponding to the reference object distance range by using the scaling factors corresponding to different object distances within each reference object distance range, and determines the dual-light fusion calibration parameters corresponding to the reference object distance range according to the best scaling factor, so as to perform dual-light fusion on the thermal imaging image and the visible light image collected at any object distance within the reference object distance range through the dual-light fusion calibration parameters corresponding to the reference object distance range. Through the above solution, by configuring different best scaling factors and dual-light fusion calibration parameters for different reference object distance ranges, the dual-light fusion pixel deviation occurring in the fusion process at each object distance is effectively reduced, and the effect of dual-light fusion is improved. Description of the Drawings

[0043] The drawings here are incorporated into the description and form a part of the description, showing embodiments consistent with the present application, and are used together with the description to explain the principles of the present application.

[0044] Figure 1 It is a schematic diagram of a dual-light fusion scenario provided by an embodiment of the present application;

[0045] Figure 2 It is a flow block diagram of a dual-light fusion method provided by an embodiment of the present application;

[0046] Figure 3 It is another flow block diagram of a dual-light fusion method provided by an embodiment of the present application;

[0047] Figure 4 It is a schematic diagram of the structure of a thermal imager provided by an embodiment of the present application;

[0048] Figure 5 Another schematic diagram of the structure of the thermal imager provided by the embodiment of the present application;

[0049] Figure 6 A structural diagram of a dual - light fusion device provided by the embodiment of the present application;

[0050] Figure 7 Another structural diagram of the dual - light fusion device provided by the embodiment of the present application. Detailed implementation manners

[0051] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application and make the above - mentioned objects, features, and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0052] In some thermal imaging devices such as thermal imagers, in order to improve the integrity, clarity, and detection accuracy of image information, two types of images, visible - light images and infrared images, can be used for dual - light fusion to obtain richer image features. During the dual - light fusion process, the visible - light image and the infrared image are usually unified to the same size through a scaling factor for the convenience of image fusion. Currently, thermal imagers are usually pre - configured with a fixed scaling factor at the factory, that is, regardless of the distance between the detected target object and the thermal imager, the fixed scaling factor is used for dual - light fusion.

[0053] However, at different object distances, the focal lengths and field - of - view angles of the lens for collecting visible - light images and the lens for collecting infrared images will both change.

[0054] For example, when the object is at a relatively far distance, the field - of - view angle of the lens is small and the focal length is long; while at a relatively close distance, the field - of - view angle is large and the focal length is short. If the same scaling factor is used at different object distances, it may cause pixel deviation in the dual - light fusion of the obtained image, affecting the fusion effect.

[0055] In this embodiment, the dual - light fusion pixel deviation refers to the pixel - level difference generated in the fused image when fusing images collected by two different sensors (such as a visible - light sensor and a thermal imaging sensor), which is used to characterize the effect of dual - light fusion. The smaller the dual - light fusion pixel deviation, the better the effect of dual - light fusion.

[0056] For some devices such as handheld thermal imagers, there are observation requirements at different distances. The object distance span of dual - light fusion is large, for example, 0.2m - 40m. During the lens focusing process, both the lens field of view angle and the corresponding field - of - view distortion degree of fusion are changing. At this time, using the same scaling factor at different object distances will result in a large pixel deviation in dual - light fusion due to the mismatch of the scaling factor, deteriorating the effect of the dual - light fusion image. Especially in the close - range observation scenario with a small object distance, such as the scenario where the object distance is 0.2m - 1m, the change degree of the field - of - view angle and the field - of - view distortion degree is large. Using a fixed scaling factor may result in a large pixel deviation in dual - light fusion.

[0057] Please refer to Figure 1 , Figure 1 which is a schematic diagram of the dual - light fusion scenario provided by the embodiment of the present application.

[0058] As Figure 1 shown, f1 and f2 are the focal lengths of the thermal imaging lens and the visible - light lens respectively. The circular and triangular icons are two objects at different object distances. The distance between the object corresponding to the circular icon and the thermal imaging lens is D1, and the distance between the object corresponding to the triangular icon and the thermal imaging lens is D2.

[0059] Taking the example that both of the above - mentioned two objects are imaged at the center of the thermal imaging image plane, due to the existence of the pupil distance between the thermal imaging lens and the visible - light lens (that is, the distance between the center of the thermal imaging lens and the center of the visible - light lens), the images of the objects at different object distances are at the same position on the thermal imaging image plane. However, they are at different field - of - view positions on the visible - light image plane (refer to Figure 1 the positions of the objects corresponding to the circular icon and the triangular icon marked on the visible - light image plane in

[0060] It can be seen that when the object distance changes, the corresponding fields of view in the visible - light lens and the thermal imaging lens will also change. In order to fuse the visible - light image and the thermal imaging image together, first, the problem of their spatial alignment needs to be solved.

[0061] When aligning the images, if the influence of the object distance on the field - of - view distortion of the images is not considered, it will lead to the failure to correctly adjust the scale difference between the two images. At this time, it is crucial to determine the appropriate scaling factor at this object distance.

[0062] Based on this, the present application provides a dual - light fusion method to improve the effect of dual - light fusion.

[0063] Please refer to Figure 2 , Figure 2 which is a flow block diagram of the dual - light fusion method provided by the embodiment of the present application.

[0064] As Figure 2 shown, the method may include the following steps:

[0065] Step 201: Segment the dual - light fusion object distance range according to the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image at different object distances, to obtain at least two reference object distance ranges.

[0066] In this embodiment, the dual - light fusion object distance range refers to the object distance range that the thermal imager supports for dual - light fusion. This range can be configured at the factory of the thermal imager, or obtained through automated fusion or manual fusion actual tests. This application does not limit this.

[0067] In this embodiment, the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image is a spatial alignment error caused by factors such as not considering the physical differences of the imaging devices and the inconsistent fields of view, and is caused by directly performing dual - light fusion processing with a fixed scaling factor.

[0068] As an embodiment, the specific method for segmenting the dual - light fusion object distance range according to the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image at different object distances to obtain at least two reference object distance ranges may include:

[0069] Perform dual - light fusion processing on the thermal imaging image and the visible - light image collected at different object distances within the dual - light fusion object distance range based on a preset scaling factor, and obtain the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image after dual - light fusion processing at each object distance;

[0070] Divide the dual - light fusion object distance range according to the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image at each object distance, to obtain at least two reference object distance ranges. Among them, the difference between the dual - light fusion pixel deviation at the farthest object distance and the dual - light fusion pixel deviation at the nearest object distance within each reference object distance range is less than a preset threshold, so that each object distance within this reference object distance range corresponds to the same optimal scaling factor; the preset threshold is used to characterize the maximum change amount of the allowable dual - light fusion pixel deviation within this reference object distance range.

[0071] In this embodiment, the dual - light fusion pixel deviation at each object distance within the dual - light fusion object distance range can be measured based on a preset scaling factor (this scaling factor can be an empirical value), that is, perform dual - light fusion processing on the thermal imaging image and the visible - light image collected at multiple object distance points within the dual - light fusion object distance range according to this preset scaling factor, and determine the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image collected at each object distance point. Divide the dual - light fusion object distance range according to the dual - light fusion pixel deviation at each object distance to obtain multiple reference object distance ranges, so that the difference between the dual - light fusion pixel deviation of the farthest object distance point and the nearest object distance point included within each divided reference object distance range is within a preset range.

[0072] Among them, if the difference is within a preset range, for example, less than a certain preset threshold, it indicates that within this object distance range, the dual - light fusion pixel deviation is relatively close and the change is small, that is, the division of this reference object distance range is appropriate and the same scaling coefficient can be used; if the difference is not within the preset range, for example, not less than a certain preset threshold, it indicates that the change of the dual - light fusion pixel deviation within this object distance range is large, that is, the division of this reference object distance range is inappropriate and the same scaling coefficient cannot be used. At this time, the reference object distance range needs to be adjusted until the difference between the dual - light fusion pixel deviations of the farthest object distance point and the nearest object distance point included in each divided reference object distance range is less than the preset threshold.

[0073] After the division of the reference object distance range is completed, for each reference object distance range, the difference between the dual - light fusion pixel deviations at each object distance included in it is within the maximum allowable change of the dual - light fusion pixel deviation within this reference object distance range. Using the same optimal scaling coefficient can ensure the fusion consistency of all object distances within this range.

[0074] In this embodiment, the method for selecting different object distances can be equidistant sampling, that is, an object distance point is collected at the same interval, or the object distances can be obtained according to a preset rule. For example, the frequency of collecting object distance points is inversely proportional to the object distance, that is, the smaller the object distance, the smaller the collection interval of the object distance point and the higher the collection frequency, that is, more object distance points are collected at close range and fewer object distance points are collected at long range.

[0075] So far, the description of step 201 ends, and then step 202 is executed.

[0076] Step 202: For each reference object distance range, based on the thermal imaging images and visible light images collected by the thermal imager at different object distances within this reference object distance range, determine the scaling coefficients corresponding to different object distances within this reference object distance range.

[0077] In this embodiment, after determining multiple reference object distance ranges according to step 201, for each reference object distance range, the scaling coefficients corresponding to different object distances within this reference object distance range can be determined.

[0078] Similarly to the above, in this embodiment, the method for selecting different object distances can be equidistant sampling or the object distances can be obtained according to a preset rule. For example, the frequency of collecting object distance points is inversely proportional to the object distance, that is, the smaller the object distance, the smaller the collection interval of the object distance point and the higher the collection frequency, that is, more object distance points are collected at close range and fewer object distance points are collected at long range.

[0079] Among them, for each object distance, the process of determining the scaling coefficient corresponding to this object distance according to the thermal imaging image and visible light image collected at this object distance is a common method in the related art and will not be elaborated here.

[0080] Thus, the description of step 202 ends, and step 203 is executed below.

[0081] Step 203: Use the scaling factors corresponding to different object distances within the reference object distance range to determine the optimal scaling factor corresponding to the reference object distance range, and determine the dual-light fusion calibration parameters corresponding to the reference object distance range according to the optimal scaling factor.

[0082] Among them, the dual-light fusion calibration parameters corresponding to any reference object distance range are used to perform dual-light fusion on the thermal imaging image and visible light image collected by the thermal imager at any object distance within the reference object distance range.

[0083] In this embodiment, after determining the scaling factors corresponding to different object distances within each reference object distance range through step 202, for each reference object distance range, the optimal scaling factor corresponding to the reference object distance range can be determined from the scaling factors corresponding to different object distances within the reference object distance range.

[0084] Specifically, the method for using the scaling factors corresponding to different object distances within the reference object distance range to determine the optimal scaling factor corresponding to the reference object distance range may include:

[0085] For the scaling factor corresponding to each object distance within the reference object distance range, use the scaling factor to perform dual-light fusion processing on the thermal imaging image and visible light image collected by the thermal imager at different object distances within the reference object distance range, and obtain the error information corresponding to the scaling factor; the error information includes: the dual-light fusion pixel deviation between the thermal imaging image and the visible light image after the thermal imaging image and the visible light image collected at each object distance are subjected to dual-light fusion processing.

[0086] Based on the error information corresponding to each scaling factor, determine the optimal scaling factor corresponding to the reference object distance range.

[0087] In this embodiment, for each object distance within the same reference object distance range, the scaling factor corresponding to the object distance can be applied to each object distance within the reference object distance range, and the visible light image and the thermal imaging image are fused according to the scaling factor, and the dual-light fusion pixel deviation of the visible light and the thermal imaging at the image edge at multiple object distance points based on the scaling factor is statistically calculated, that is, the error information corresponding to the scaling factor.

[0088] Further, the specific method for determining the optimal scaling factor corresponding to the reference object distance range based on the error information corresponding to each scaling factor may include:

[0089] Perform a specified operation on each dual-light fusion pixel deviation in the error information corresponding to each scaling factor to obtain an error result corresponding to each scaling factor;

[0090] According to the error results corresponding to each scaling factor, determine the optimal scaling factor corresponding to the reference object distance range from the various scaling factors, and the error result corresponding to the optimal scaling factor meets the set dual - light fusion pixel deviation requirement.

[0091] In this embodiment, the specified operation on the dual - light fusion pixel deviations in the error information corresponding to each scaling factor can be calculating the average value, that is, the error result is the average value of the dual - light fusion pixel deviations; it can also be determining the magnitude relationship between it and a specified value (such as taking the difference from the specified value), that is, the error result is the set of differences between the dual - light fusion pixel deviations and the specified value. This application does not limit this.

[0092] As an embodiment, taking the specified operation as calculating the average value as an example, the method for determining the optimal scaling factor corresponding to the reference object distance range from the various scaling factors according to the error results corresponding to each scaling factor can be:

[0093] Take the scaling factor corresponding to the error result with the smallest average value of the dual - light fusion pixel deviations as the optimal scaling factor corresponding to the reference object distance range.

[0094] At this time, the set dual - light fusion pixel deviation requirement is to select the scaling factor corresponding to the error result with the smallest average value of the dual - light fusion pixel deviations as the optimal scaling factor.

[0095] As an embodiment, taking the specified operation as determining the magnitude relationship between the dual - light fusion pixel deviation and the specified value as an example, the method for determining the optimal scaling factor corresponding to the reference object distance range from the various scaling factors according to the error results corresponding to each scaling factor can be:

[0096] Take the scaling factor corresponding to the error result where the dual - light fusion pixel deviations are all less than the specified value as the optimal scaling factor corresponding to the reference object distance range.

[0097] At this time, the set dual - light fusion pixel deviation requirement is to select the scaling factor corresponding to the error result where all the dual - light fusion pixel deviations are less than the specified value (that is, the differences between all the dual - light fusion pixel deviations and the specified value are all negative) as the optimal scaling factor.

[0098] In this embodiment, in the case where the specified operation is to determine the magnitude relationship between the dual - light fusion pixel deviation and the specified value, the above - mentioned specified value can be determined based on the resolution of the thermal imaging component in the thermal imager for collecting the thermal imaging image.

[0099] As an embodiment, the specified value can be positively correlated with the size of the resolution of the thermal imaging component, that is, the higher the resolution of the thermal imaging component, the larger the specified value; the lower the resolution of the thermal imaging component, the smaller the specified value.

[0100] Exemplarily, N times the resolution of the thermal imaging component can be determined as the specified value, where N is greater than 0 and less than 1. For example, taking a resolution of 640*512 as an example, the specified value can be set to 6 pixels, which is approximately 1% of 640.

[0101] In this embodiment, the dual - light fusion pixel deviation can be the pixel - level difference generated in the overall image obtained after fusing the thermal imaging image and the visible - light image, or the pixel deviation values respectively determined for the central region and the edge region in the fused image.

[0102] Since the requirements for the fusion pixel deviation at the center of the image are usually higher, when setting the above - mentioned specified value, different specified values can be set for the central region and the edge region of the image. For example, taking a resolution of 640*512 as an example, it is required that the pixel deviation in the central region is less than 3 pixels (as the specified value corresponding to the central region), and the edge is less than 6 pixels (as the specified value corresponding to the edge region), which is approximately 0.5% and 1% of 640.

[0103] As an embodiment, after determining the optimal scaling factor corresponding to each reference object - distance range, the dual - light fusion pixel deviation of each object - distance within the reference object - distance range can be calculated according to the optimal scaling factor to verify whether the selection of the reference object - distance range is appropriate. For example, based on the optimal scaling factor, the dual - light fusion pixel deviation of each object - distance within the reference object - distance range is determined. If the dual - light fusion pixel deviation corresponding to at least one object - distance is greater than the deviation threshold, the reference object - distance range is re - divided.

[0104] The method for re - dividing the reference object - distance range can include:

[0105] Dividing at least one object - distance whose dual - light fusion pixel deviation is greater than the deviation threshold into other reference object - distance ranges adjacent to the reference object - distance range; or,

[0106] Dividing the reference object - distance range into a first reference object - distance range and a second reference object - distance range, where the first reference object - distance range includes object - distances with dual - light fusion pixel deviation less than or equal to the deviation threshold, and the second reference object - distance range includes object - distances with dual - light fusion pixel deviation greater than the deviation threshold. Further, the optimal scaling factor corresponding to the original reference object - distance range is used as the optimal scaling factor corresponding to the first reference object - distance range, and a corresponding optimal scaling factor is re - determined for the second reference object - distance range.

[0107] In this embodiment, the size of the above - mentioned deviation threshold can be selected according to actual needs. For example, the size of the deviation threshold can be not greater than the size of the above - mentioned specified value, and the present application does not limit this.

[0108] After verifying the optimal scaling factor, the dual - light fusion calibration parameters corresponding to the reference object distance range can be further determined according to the optimal scaling factor.

[0109] Specifically, the dual - light fusion calibration parameters may include the optimal scaling factor corresponding to the reference object distance range and the offset coefficients. The offset coefficients include the horizontal offset coefficient, the vertical offset coefficient, the calibrated horizontal offset amount, and the calibrated vertical offset amount. The method for determining the dual - light fusion calibration parameters corresponding to the reference object distance range according to the optimal scaling factor may include:

[0110] Perform scaling processing on the thermal imaging image and / or visible light image collected by the thermal imager at different object distances within the reference object distance range based on the optimal scaling factor corresponding to the reference object distance range, and determine the offset amount between the thermal imaging image and the visible light image after scaling processing at each object distance. The offset amount includes the horizontal offset amount and the vertical offset amount.

[0111] Fit the horizontal offset amounts between the thermal imaging image and the visible light image at different object distances to obtain the horizontal offset coefficient corresponding to the reference object distance range. The horizontal offset coefficient is used to indicate the relationship between the object distance and the horizontal offset amount within the reference object distance range.

[0112] Fit the vertical offset amounts between the thermal imaging image and the visible light image at different object distances to obtain the vertical offset coefficient corresponding to the reference object distance range. The vertical offset coefficient is used to indicate the relationship between the object distance and the vertical offset amount within the reference object distance range.

[0113] Determine the calibrated horizontal offset amount and the calibrated vertical offset amount corresponding to the reference object distance range according to the optimal scaling factor corresponding to the reference object distance range and the object distance corresponding to the optimal scaling factor.

[0114] Take the optimal scaling factor, the horizontal offset coefficient, the vertical offset coefficient, the calibrated horizontal offset amount, and the calibrated vertical offset amount as the dual - light fusion calibration parameters corresponding to the reference object distance range.

[0115] In this embodiment, the scaling factor is used to indicate the proportional relationship between the infrared image and the visible light image of the target object, and the offset coefficient is used to indicate the relationship between the offset amount between the infrared image and the visible light image of the target object and the object distance.

[0116] In this embodiment, for each reference object distance range, based on the optimal scaling factor corresponding to the reference object distance range, the horizontal and vertical offsets at different object distances can be confirmed through automated fusion or manual fusion methods, the first horizontal offset coefficient and the first vertical offset coefficient are confirmed by fitting, and according to the object distance corresponding to the object distance point for determining the optimal scaling factor (denoted as the calibration object distance), the calibration horizontal offset and the calibration vertical offset corresponding to this reference object distance range are determined (i.e., the horizontal offset and the vertical offset corresponding to this calibration object distance).

[0117] Further, the optimal scaling factor, the horizontal offset coefficient, the vertical offset coefficient, the calibration horizontal offset, and the calibration vertical offset can be used as the dual - light fusion calibration parameters corresponding to the reference object distance range, so as to perform dual - light fusion on the thermal imaging image and the visible - light image collected by the thermal imager at any object distance within this reference object distance range through the dual - light fusion calibration parameters corresponding to any reference object distance range.

[0118] In this embodiment, factors such as the dual - light fusion object distance range, the dual - pupil distance, the change in the field of view angle during the lens focusing process, and the change in the visible - light field - of - view distortion corresponding to thermal imaging at different object distances can be combined to segment the dual - light fusion object distance. Taking two segments as an example, the first object distance range and the second object distance range;

[0119] Within the first object distance range, multiple object distance points are selected, the optimal scaling factor is confirmed as the first scaling factor, the horizontal and vertical offsets at different object distances are confirmed through automated fusion or manual fusion methods, and the first horizontal offset coefficient and the first vertical offset coefficient are confirmed by fitting. And according to the object distance corresponding to the object distance point for determining the first scaling factor (the first calibration object distance), the first calibration horizontal offset and the second calibration vertical offset corresponding to the first calibration object distance are determined.

[0120] Similarly, within the second object distance range, the second scaling factor, the second horizontal offset coefficient, the vertical offset coefficient, the second calibration object distance, the second calibration horizontal offset, and the second calibration vertical offset are confirmed.

[0121] Thus, the description of step 203 ends.

[0122] Thus, the description of Figure 2 ends.

[0123] According to the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image at different object distances, the present application segments the dual - light fusion object - distance range to obtain at least two reference object - distance ranges. Further, the optimal scaling factor corresponding to each reference object - distance range is determined by using the scaling factors corresponding to different object distances within each reference object - distance range, and the dual - light fusion calibration parameters corresponding to the reference object - distance range are determined according to the optimal scaling factor. In order to perform dual - light fusion on the thermal imaging image and the visible - light image collected at any object distance within the reference object - distance range through the dual - light fusion calibration parameters corresponding to the reference object - distance range, through the above - mentioned solution, by configuring different optimal scaling factors and dual - light fusion calibration parameters for different reference object - distance ranges, the dual - light fusion pixel deviation occurring in the fusion process at each object distance is effectively reduced, and the effect of dual - light fusion is improved.

[0124] In this embodiment, Figure 2 The dual - light fusion method shown is actually a method for calibrating the dual - light fusion parameters of a thermal imager. After only one calibration, the calibrated parameters and the optimal scaling factor can be directly used for calculating the dual - light fusion parameters subsequently. For thermal imagers of the same model, there is no need to perform the calibration process every time.

[0125] Please refer to Figure 3 , Figure 3 which is another flow chart of the dual - light fusion method provided by the embodiment of the present application.

[0126] The dual - light fusion method provided in this embodiment is actually a process of applying the calibrated thermal imager after completing the Figure 2 calibration of the thermal imager in the method shown.

[0127] As Figure 3 shown, the method may include the following steps:

[0128] Step 301, determine the target object - distance range to which the reference distance belongs from the obtained multiple reference object - distance ranges according to the reference distance between the target object and the thermal imager.

[0129] Among them, the multiple reference object - distance ranges are obtained by segmenting the dual - light fusion object - distance range according to the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image at different object distances. Different reference object - distance ranges correspond to different dual - light fusion calibration parameters. The dual - light fusion calibration parameters are determined according to the optimal scaling factor corresponding to the reference object - distance range, and the optimal scaling factor corresponding to the reference object - distance range is determined according to the scaling factors corresponding to different object distances within the reference object - distance range.

[0130] In this embodiment, it can be pre - passed through Figure 2The method shown divides the object distance range supported by the thermal imager for dual - light fusion into multiple reference object distance ranges, and different optimal zoom factors and dual - light fusion calibration parameters are configured for different reference object distance ranges.

[0131] As an embodiment, the reference distance between the target object and the thermal imager can be determined by the following method:

[0132] Determine the reference distance between the target object and the thermal imager according to the focusing condition of the thermal imaging component and / or visible light component included in the thermal imager when collecting an image of the target object;

[0133] Alternatively, the thermal imager further includes a ranging component, and the reference distance between the target object and the thermal imager is determined according to the ranging component.

[0134] In this embodiment, the method for determining the reference distance will be described based on the subsequent structure of the thermal imager and will not be elaborated here.

[0135] Thus, the description of step 301 ends, and step 302 is executed below.

[0136] Step 302, perform dual - light fusion on the infrared image and visible - light image of the target object collected by the thermal imager according to the reference distance, the optimal zoom factor corresponding to the target object distance range, and the dual - light fusion calibration parameters.

[0137] As an embodiment, the specific method for performing dual - light fusion on the infrared image and visible - light image of the target object collected by the thermal imager according to the reference distance, the optimal zoom factor corresponding to the target object distance range, and the dual - light fusion calibration parameters includes:

[0138] Determine the target horizontal offset corresponding to the reference distance according to the reference distance, the horizontal offset coefficient corresponding to the target object distance range, and the calibrated horizontal offset;

[0139] Determine the target vertical offset corresponding to the reference distance according to the reference distance, the vertical offset coefficient corresponding to the target object distance range, and the calibrated vertical offset;

[0140] Perform dual - light fusion on the infrared image and visible - light image of the target object collected by the thermal imager according to the reference distance, the optimal zoom factor corresponding to the target object distance range, the target horizontal offset, and the target vertical offset.

[0141] As an embodiment, taking the relational expression between the object distance and the horizontal offset as as an example, where y is the horizontal offset, k is the horizontal offset coefficient, x is the object distance, and b is the calibrated horizontal offset.

[0142] Substituting the obtained x (reference distance), k (horizontal offset coefficient), and b (calibrated horizontal offset) into the relational expression, the target horizontal offset corresponding to the reference distance can be obtained.

[0143] It should be noted that the relational expression here is only an exemplary description, and the form of the relational expression is not limited in this application.

[0144] Similarly, according to the reference distance, the vertical offset coefficient corresponding to the target object distance range, and the calibrated vertical offset configured for the target object distance range, the target vertical offset corresponding to the reference distance can be determined.

[0145] In this embodiment, the process of performing dual - light fusion processing on the infrared image and visible - light image of the target object is a common method in the related art and will not be elaborated here.

[0146] Thus, the description of step 302 ends.

[0147] Thus, the description of Figure 3 ends.

[0148] In this embodiment, during the process of image acquisition of the target object by the thermal imager, first, according to the reference distance between the target object and the thermal imager, the target object distance range to which the reference distance belongs can be determined from the obtained multiple reference object distance ranges. Further, according to the reference distance, the best scaling coefficient and dual - light fusion calibration parameters corresponding to the target object distance range, the infrared image and visible - light image of the target object are fused. By segmenting the object distance to adapt the best scaling coefficient and dual - light fusion calibration parameters, in the case of a large span of the fusion object distance range, the dual - light fusion pixel deviation caused by the change of the field - of - view angle and the distortion change of the fusion field - of - view during the focusing process can be eliminated or reduced, improving the fusion accuracy.

[0149] Next, a specific embodiment is used to describe the dual - light fusion method proposed in this application.

[0150] According to the working requirements of the product and the object distance range, such as 0.2m - 40m, the object distance is divided into two reference object distance ranges: a short - distance range (0.2m - 1m) and a long - distance range (1m - 40m), and the scaling coefficient and offset coefficient corresponding to each reference object distance range are determined. At this time, the change of the object distance will cause the change of the lens field - of - view angle and distortion, so calibration needs to be performed according to different object distance ranges respectively.

[0151] Calibration for the short - distance range (0.2m - 1m):

[0152] In the short-distance range, select several representative object distances (such as 0.2m, 0.3m, 0.5m, 1m), and determine the optimal scaling factor corresponding to this short-distance range from the scaling factors corresponding to these object distances. Further, for each object distance, measure and confirm the offsets in the horizontal and vertical directions based on this optimal scaling factor.

[0153] Use the horizontal and vertical offsets of the selected object distances for fitting to determine the horizontal offset coefficient and vertical offset coefficient corresponding to this short-distance range, as well as the calibrated horizontal offset value and calibrated vertical offset value at the calibrated object distance corresponding to the optimal scaling factor.

[0154] Calibration of the long-distance range (1m - 40m):

[0155] Similar to the short-distance range, select several object distances (such as 1m, 2m, 5m, 10m, 20m, 40m) within the long-distance range for calibration.

[0156] Determine the optimal scaling factor corresponding to this long-distance range from the scaling factors corresponding to these object distances. Further, for each object distance, measure and confirm the offsets in the horizontal and vertical directions based on this optimal scaling factor.

[0157] Use the horizontal and vertical offsets of the selected object distances for fitting to determine the horizontal offset coefficient and vertical offset coefficient corresponding to this long-distance range, as well as the calibrated horizontal offset value and calibrated vertical offset value at the calibrated object distance corresponding to the optimal scaling factor.

[0158] Through the above process, obtain the optimal scaling factors and offset coefficients corresponding to the two object distance ranges. During subsequent applications, appropriate calibration parameters can be automatically selected according to the object distance for image fusion.

[0159] In this embodiment, the current object distance can be measured in real time through a laser ranging sensor configured in the thermal imager or through an autofocus system configured in the thermal imaging lens or visible light lens of the thermal imager, that is, the current object distance can be obtained in real time through the laser ranging module, or the current focusing distance of the lens can be obtained through the focusing system, so as to obtain accurate object distance information.

[0160] After obtaining the current object distance, the system will select the corresponding optimal scaling factor according to the preset reference object distance range (such as short-distance range, long-distance range, etc.). For example, if the current object distance is within the range of 0.2m - 1m, the optimal scaling factor corresponding to the short-distance range is applied; if the object distance is within the range of 1m - 40m, the optimal scaling factor corresponding to the long-distance range is applied.

[0161] According to the calibration data for each reference object distance range, select appropriate horizontal and vertical offset coefficients, as well as the calibrated horizontal offset value and calibrated vertical offset value corresponding to the calibrated object distance of the optimal scaling coefficient, and calculate the dual-light offset amount (i.e., the target horizontal offset value and the target vertical offset value) at the current object distance.

[0162] According to the calculated dual-light fusion parameters (including the optimal scaling coefficient, horizontal offset coefficient, vertical offset coefficient, target horizontal offset value, and target vertical offset value), adjust the position, size, and field of view of the thermal imaging image and the visible light image, and fuse the thermal imaging image and the visible light image. According to actual requirements, different image fusion algorithms (such as weighted average, maximum fusion, minimum fusion, etc.) can be used to achieve the final dual-light fusion processing.

[0163] When the object distance changes, the system can recalculate the dual-light fusion parameters corresponding to the object distance in real time and automatically adjust the dual-light fusion process to ensure a good dual-light fusion effect.

[0164] Through the above solution, the problem of large deterioration of dual-light fusion pixels caused by changes in object distance can be effectively solved, ensuring accurate registration and fusion of images at different object distances, and improving the quality of the dual-light fusion image. Especially in the application of handheld products, due to the large span of object distances, the system can adjust the scaling coefficient and offset coefficient in real time according to the change of object distance to ensure that the image fusion effect is always maintained at a high level.

[0165] In addition, since the method proposed in this application is applied to a thermal imager, the present application also proposes the specific structure of the thermal imager.

[0166] Please refer to Figure 4 , Figure 4 which is a schematic diagram of the structure of a thermal imager provided by an embodiment of the present application.

[0167] As Figure 4 shown, the thermal imager includes:

[0168] An image collector, including a thermal imaging component and a visible light component, where the thermal imaging component is used to collect infrared images at each object distance, and the visible light component is used to collect visible light images at each object distance;

[0169] Specifically, the thermal imaging component may include a thermal imaging detector and a thermal imaging lens for collecting infrared images at each object distance;

[0170] The visible light component may include a visible light detector and a visible light lens for collecting the visible light images at each object distance;

[0171] A memory for storing infrared images, visible light images, as well as the optimal scaling coefficient and dual-light fusion calibration parameters corresponding to each reference object distance range;

[0172] An image display device for displaying the infrared image, visible light image, and the fused image obtained after dual - light fusion processing;

[0173] A processor for executing Figure 2 or Figure 3 the method shown.

[0174] In this embodiment, the thermal imager further includes:

[0175] A ranging module for detecting the reference distance between the target object and the thermal imager.

[0176] In this embodiment, since the thermal imager includes a ranging module, the distance between the target object and the thermal imager can be directly detected by the ranging module.

[0177] Please refer to Figure 5 , Figure 5 which is another schematic structural diagram of the thermal imager provided by the embodiment of the present application.

[0178] As Figure 5 shown, the thermal imager includes:

[0179] An image collector including a thermal imaging component and a visible light component. The thermal imaging component is used to collect infrared images at various object distances, and the visible light component is used to collect visible light images at various object distances;

[0180] Specifically, the thermal imaging component may include a thermal imaging detector and a thermal imaging lens for collecting infrared images at various object distances; in this embodiment, the thermal imaging component further includes an autofocus module for obtaining the focusing information of the thermal imaging component, such as focal length, sensor size, field of view angle, etc., to further calculate the reference distance between the target object and the thermal imager based on these focusing information.

[0181] The visible light component may include a visible light detector and a visible light lens for collecting the visible light images at the various object distances.

[0182] A memory for storing infrared images, visible light images, and the scaling coefficient and offset coefficient corresponding to each reference object distance range;

[0183] An image display device for displaying the infrared image, visible light image, and the target image obtained after dual - light fusion processing;

[0184] A processor for executing Figure 2 or Figure 3 the method shown.

[0185] In this embodiment, the thermal imager does not include a ranging module. At this time, the object distance can be input by the user himself, or the reference distance between the target object and the thermal imager can be calculated according to the focusing situation of the thermal imaging component and / or the visible light component when collecting images of the target object (that is, the focusing information obtained by the autofocus module, such as the current focusing distance of the lens, etc.). The present application does not limit this.

[0186] So far, the description of the thermal imager shown in Figure 4 and Figure 5 is completed.

[0187] Please refer to Figure 6 , Figure 6 which is a structural diagram of a dual - light fusion device proposed in an embodiment of the present application. This device is applied to a thermal imager. As Figure 6 shown, the device may include a segmentation unit 601, a coefficient determination unit 602, and a calibration unit 603. Specifically, the device includes:

[0188] The segmentation unit 601 is configured to segment the dual - light fusion object distance range according to the dual - light fusion pixel deviation between the thermal imaging image and the visible light image at different object distances, so as to obtain at least two reference object distance ranges;

[0189] The coefficient determination unit 602 is configured to, for each reference object distance range, based on the thermal imaging images and visible light images collected by the thermal imager at different object distances within this reference object distance range, determine the scaling coefficients corresponding to different object distances within this reference object distance range;

[0190] The calibration unit 603 is configured to use the scaling coefficients corresponding to different object distances within this reference object distance range to determine the optimal scaling coefficient corresponding to this reference object distance range, and determine the dual - light fusion calibration parameters corresponding to this reference object distance range according to the optimal scaling coefficient; wherein, the dual - light fusion calibration parameters corresponding to any reference object distance range are used to perform dual - light fusion on the thermal imaging image and the visible light image collected by the thermal imager at any object distance within this reference object distance range.

[0191] Optionally, the segmentation unit 601 is specifically configured to:

[0192] Perform dual - light fusion processing on the thermal imaging images and visible light images collected at different object distances within the dual - light fusion object distance range based on a preset scaling coefficient, and obtain the dual - light fusion pixel deviation between the thermal imaging image and the visible light image after dual - light fusion processing at each object distance;

[0193] Divide the dual - light fusion object distance range according to the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image at each object distance, to obtain at least two reference object distance ranges. Among them, the dual - light fusion pixel deviation at each object distance within each reference object distance range is less than a preset threshold, and the preset threshold is used to characterize the maximum change amount of the allowable dual - light fusion pixel deviation within this reference object distance range.

[0194] Optionally, the calibration unit 603 is specifically configured to:

[0195] For the scaling factor corresponding to each object distance within the reference object distance range, use this scaling factor to perform dual - light fusion processing on the thermal imaging image and the visible - light image collected by the thermal imager at different object distances within this reference object distance range, and obtain the error information corresponding to the scaling factor; the error information includes: the dual - light fusion pixel deviation between the thermal imaging image and the visible - light image after the dual - light fusion processing of the thermal imaging image and the visible - light image collected at each object distance.

[0196] Determine the optimal scaling factor corresponding to this reference object distance range according to the error information corresponding to each scaling factor.

[0197] Optionally, the calibration unit 603 is specifically configured to:

[0198] Perform a specified operation on each dual - light fusion pixel deviation in the error information corresponding to each scaling factor to obtain the error result corresponding to each scaling factor.

[0199] Determine the optimal scaling factor corresponding to this reference object distance range from the various scaling factors according to the error result corresponding to each scaling factor, and the error result corresponding to the optimal scaling factor meets the set dual - light fusion pixel deviation requirement.

[0200] Optionally, the dual - light fusion calibration parameters include the optimal scaling factor corresponding to the reference object distance range and the offset coefficients. The offset coefficients include a horizontal offset coefficient, a vertical offset coefficient, a calibrated horizontal offset amount, and a calibrated vertical offset amount; the calibration unit 603 is specifically configured to:

[0201] Based on the optimal scaling factor corresponding to this reference object distance range, perform scaling processing on the thermal imaging image and / or the visible - light image collected by the thermal imager at different object distances within this reference object distance range, and determine the offset amount between the thermal imaging image and the visible - light image after the scaling processing at each object distance; the offset amount includes a horizontal offset amount and a vertical offset amount.

[0202] Fit the horizontal offset amounts between the thermal imaging image and the visible - light image at different object distances to obtain the horizontal offset coefficient corresponding to this reference object distance range, and the horizontal offset coefficient is used to indicate the relationship between the object distance and the horizontal offset amount within this reference object distance range.

[0203] Fit the vertical offset between the thermal imaging image and the visible light image at different object distances to obtain the vertical offset coefficient corresponding to the reference object distance range, where the vertical offset coefficient is used to indicate the relationship between the object distance and the vertical offset within the reference object distance range;

[0204] Determine the calibrated horizontal offset and the calibrated vertical offset corresponding to the reference object distance range according to the optimal scaling coefficient corresponding to the reference object distance range and the object distance corresponding to the optimal scaling coefficient;

[0205] Take the optimal scaling coefficient, the horizontal offset coefficient, the vertical offset coefficient, the calibrated horizontal offset, and the calibrated vertical offset as the dual - light fusion calibration parameters corresponding to the reference object distance range.

[0206] So far, the description of the dual - light fusion device in Figure 6 is completed.

[0207] Please refer to Figure 7 , Figure 7 which is another structural diagram of the dual - light fusion device proposed in the embodiment of this application. This device is applied to a thermal imager. As Figure 7 shown, the device may include an object distance range determination unit 701 and a fusion unit 702. Specifically, the device includes:

[0208] The object distance range determination unit 701 is configured to determine the target object distance range to which the reference distance belongs from the obtained multiple reference object distance ranges according to the reference distance between the target object and the thermal imager; where the multiple reference object distance ranges are obtained by segmenting the dual - light fusion object distance range according to the dual - light fusion pixel deviation between the thermal imaging image and the visible light image at different object distances. Different reference object distance ranges correspond to different dual - light fusion calibration parameters, and the dual - light fusion calibration parameters are determined according to the optimal scaling coefficient corresponding to the reference object distance range, and the optimal scaling coefficient corresponding to the reference object distance range is determined according to the scaling coefficients corresponding to different object distances within the reference object distance range;

[0209] The fusion unit 702 is configured to perform dual - light fusion on the infrared image and the visible light image of the target object collected by the thermal imager according to the reference distance, the optimal scaling coefficient corresponding to the target object distance range, and the dual - light fusion calibration parameters.

[0210] Optionally, the object distance range determination unit 701 is specifically configured to:

[0211] Determine the reference distance between the target object and the thermal imager according to the focusing condition of the thermal imaging component and / or the visible light component included in the thermal imager when collecting an image of the target object;

[0212] Alternatively, the thermal imager further includes a ranging component, and a reference distance between the target object and the thermal imager is determined according to the ranging component.

[0213] Thus far, the description of the dual - light fusion device in Figure 7 is completed.

[0214] Correspondingly, an embodiment of the present application further provides a computer - readable storage medium. A number of computer instructions are stored on the computer - readable storage medium. When the computer instructions are executed, the methods disclosed in the above examples of the present application can be implemented.

[0215] Exemplarily, the above - mentioned computer - readable storage medium can be any electronic, magnetic, optical or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, the computer - readable storage medium can be: RAM (Random Access Memory), volatile memory, non - volatile memory, flash memory, storage drives (such as hard disk drives), solid - state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or a combination thereof.

[0216] The above are only the preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of protection of the present application.

Claims

1. A dual light fusion method, characterized in that: The method is applied to a thermal imager, and the method comprises: According to the dual-light fusion pixel deviation between the thermal imaging image and the visible light image at different object distances, the dual-light fusion object distance range is segmented to obtain at least two reference object distance ranges; For each reference object distance range, based on the thermal imaging images and visible light images acquired by the thermal imager at different object distances within the reference object distance range, determining the scaling factors corresponding to the different object distances within the reference object distance range; The optimal scaling factor corresponding to the reference object distance range is determined by utilizing the scaling factors corresponding to different object distances within the reference object distance range, and the dual-light fusion calibration parameters corresponding to the reference object distance range are determined according to the optimal scaling factor; wherein the dual-light fusion calibration parameters corresponding to any reference object distance range are used to perform dual-light fusion on the thermal imaging image and the visible light image acquired by the thermal imager at any object distance within the reference object distance range.

2. The method according to claim 1, characterized in that The dual-light fusion object distance range is segmented according to the dual-light fusion pixel deviation between the thermal imaging image and the visible light image at different object distances to obtain at least two reference object distance ranges, including: Based on a preset scaling factor, dual-light fusion processing is performed on the thermal imaging image and the visible light image collected at different object distances within the dual-light fusion object distance range, and a dual-light fusion pixel deviation between the thermal imaging image and the visible light image after dual-light fusion processing at each object distance is obtained; The dual-light fusion object distance range is divided according to the dual-light fusion pixel deviation between the thermal imaging image and the visible light image at each object distance to obtain at least two reference object distance ranges, wherein the difference between the dual-light fusion pixel deviation at the farthest object distance and the dual-light fusion pixel deviation at the closest object distance in each reference object distance range is less than a preset threshold, so that each object distance in the reference object distance range corresponds to the same optimal scaling factor; the preset threshold is used to characterize the maximum change in the dual-light fusion pixel deviation allowed within the reference object distance range.

3. The method according to claim 1, characterized in that Determining the optimal scaling factor corresponding to the reference object distance range by using scaling factors corresponding to different object distances within the reference object distance range includes: For each object distance within the reference object distance range, a scaling factor corresponding to each object distance is used to perform dual-light fusion processing on the thermal imaging image and the visible light image acquired by the thermal imager at different object distances within the reference object distance range, and error information corresponding to the scaling factor is obtained; the error information includes: a dual-light fusion pixel deviation between the thermal imaging image and the visible light image acquired at each object distance after the dual-light fusion processing; The optimal scaling factor corresponding to the reference object distance range is determined according to the error information corresponding to each scaling factor.

4. The method according to claim 3, characterized in that Determining the optimal scaling factor corresponding to the reference object distance range according to the error information corresponding to each scaling factor includes: Performing a specified operation on each dual-light fusion pixel deviation in the error information corresponding to each scaling factor to obtain an error result corresponding to each scaling factor; According to the error result corresponding to each scaling factor, the optimal scaling factor corresponding to the reference object distance range is determined from the scaling factors, and the error result corresponding to the optimal scaling factor meets the set dual-light fusion pixel deviation requirement.

5. The method according to claim 1, characterized in that: The dual-light fusion calibration parameters include an optimal scaling factor and an offset factor corresponding to the reference object distance range, and the offset factor includes a horizontal offset factor, a vertical offset factor, a calibrated horizontal offset, and a calibrated vertical offset; The step of determining the dual-light fusion calibration parameters corresponding to the reference object distance range according to the optimal scaling factor includes: Scaling the thermal imaging image and / or visible light image acquired by the thermal imager at different object distances within the reference object distance range based on the optimal scaling factor corresponding to the reference object distance range, and determining the offset between the thermal imaging image and the visible light image after scaling at each object distance; the offset includes a horizontal offset and a vertical offset; Fitting the horizontal offset between the thermal imaging image and the visible light image at different object distances to obtain a horizontal offset coefficient corresponding to the reference object distance range, wherein the horizontal offset coefficient is used to indicate the relationship between the object distance and the horizontal offset within the reference object distance range; Fitting the vertical offset between the thermal imaging image and the visible light image at different object distances to obtain a vertical offset coefficient corresponding to the reference object distance range, wherein the vertical offset coefficient is used to indicate the relationship between the object distance and the vertical offset within the reference object distance range; Determine a calibrated horizontal offset and a calibrated vertical offset corresponding to the reference object distance range according to an optimal scaling factor corresponding to the reference object distance range and an object distance corresponding to the optimal scaling factor; The optimal scaling factor, the horizontal offset factor, the vertical offset factor, the calibrated horizontal offset, and the calibrated vertical offset are used as dual-light fusion calibration parameters corresponding to the reference object distance range.

6. A dual light fusion method, characterized in that: The method is applied to a thermal imager, and the method comprises: According to a reference distance between the target object and the thermal imager, a target object distance range to which the reference distance belongs is determined from a plurality of reference object distance ranges that have been obtained; wherein the plurality of reference object distance ranges are obtained by segmenting the dual-light fusion object distance range according to dual-light fusion pixel deviations between the thermal imaging image and the visible light image at different object distances, different reference object distance ranges correspond to different dual-light fusion calibration parameters, the dual-light fusion calibration parameters are determined according to an optimal scaling factor corresponding to the reference object distance range, and the optimal scaling factor corresponding to the reference object distance range is determined according to scaling factors corresponding to different object distances within the reference object distance range; According to the reference distance, the optimal scaling factor corresponding to the target object distance range, and the dual-light fusion calibration parameters, dual-light fusion is performed on the infrared image and the visible light image of the target object collected by the thermal imager.

7. The method according to claim 6, characterized in that The reference distance between the target object and the thermal imager is determined by the following method: Determine a reference distance between the target object and the thermal imager according to a focusing condition of a thermal imaging component and / or a visible light component included in the thermal imager when capturing an image of the target object; Alternatively, the thermal imager further includes a distance measuring component, and a reference distance between the target object and the thermal imager is determined according to the distance measuring component.

8. A dual-light fusion device, characterized in that: The device is applied to a thermal imager and comprises: A segmentation unit, used to segment the dual-light fusion object distance range according to the dual-light fusion pixel deviation between the thermal imaging image and the visible light image at different object distances, to obtain at least two reference object distance ranges; a coefficient determination unit, for determining, for each reference object distance range, scaling coefficients corresponding to different object distances within the reference object distance range based on thermal imaging images and visible light images acquired by the thermal imager at different object distances within the reference object distance range; A calibration unit is used to determine the optimal scaling factor corresponding to the reference object distance range by using the scaling factors corresponding to different object distances within the reference object distance range, and determine the dual-light fusion calibration parameters corresponding to the reference object distance range according to the optimal scaling factor; wherein the dual-light fusion calibration parameters corresponding to any reference object distance range are used to perform dual-light fusion on the thermal imaging image and the visible light image acquired by the thermal imager at any object distance within the reference object distance range.

9. A dual-light fusion device, characterized in that: The device is applied to a thermal imager and comprises: an object distance range determination unit, for determining, according to a reference distance between a target object and the thermal imager, a target object distance range to which the reference distance belongs from a plurality of reference object distance ranges that have been obtained; wherein the plurality of reference object distance ranges are obtained by segmenting the dual-light fusion object distance range according to dual-light fusion pixel deviations between a thermal imaging image and a visible light image at different object distances, different reference object distance ranges correspond to different dual-light fusion calibration parameters, the dual-light fusion calibration parameters are determined according to an optimal scaling factor corresponding to the reference object distance range, and the optimal scaling factor corresponding to the reference object distance range is determined according to scaling factors corresponding to different object distances within the reference object distance range; A fusion unit is used to perform dual-light fusion on the infrared image and the visible light image of the target object collected by the thermal imager according to the reference distance, the optimal zoom factor corresponding to the target object distance range, and the dual-light fusion calibration parameter.

10. A thermal imager, characterized in that: The thermal imager comprises: An image collector, comprising a thermal imaging component and a visible light component, wherein the thermal imaging component is used to collect infrared images at various object distances, and the visible light component is used to collect visible light images at various object distances; A memory, used to store infrared images, visible light images, and optimal zoom factors corresponding to each reference object distance range and dual-light fusion calibration parameters; An image display device, used for displaying the infrared image, the visible light image and the fused image obtained after dual-light fusion processing; A processor, configured to execute the method according to any one of claims 1 to 7.

Images