Optical Axis Calibration Method and Device for an Image Acquisition Device

Abstract

The present application discloses an optical axis calibration method and device for an image acquisition device, which uses an acquired image as a reference image and crops other images according to the reference image to make the central images of the cropped images consistent with the central image of the reference image, so as to realize more convenient and accurate optical axis calibration of the image acquisition device. The operation is simple and does not require manual intervention in optical components, thereby improving the calibration efficiency and saving labor costs. An optical axis calibration method for an image acquisition device provided by the present application includes: acquiring images simultaneously collected by each of the sensors, and determining whether the difference between the central images of the respective images is within a preset range; when the difference is not within the preset range, using one of the images as a reference image and cropping the other images according to the reference image so that the difference between the central image of the cropped image and the central image of the reference image is within the preset range.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular, to an optical axis calibration method and device for an image acquisition device. Background Art

[0002] The optical axis refers to the axis of symmetry of an optical system, which is an axis passing through the center of the lens and perpendicular to the lens plane, and can also be understood as the center point of the imaging screen of a camera. A thermal imaging camera generally has two image sensors (sensors for acquiring images). One sensor is responsible for acquiring visible light images, and the other sensor is responsible for acquiring thermal imaging images. Under normal circumstances, the center position of the visible light image screen is consistent with the center position of the thermal imaging image screen, that is, the images (i.e., the center images) at the center positions of the two screens are basically the same. However, sometimes during the assembly of a thermal imaging camera, due to the assembly of multiple components, there are inevitably assembly errors, resulting in deviations in the installation positions of the two sensors, causing the center images of the two-channel screens to be inconsistent, and the imaging effect not reaching the expected effect. Generally, by physical means such as disassembling the camera and reinstalling the sensor, or adjusting the position of the sensor, the position of the optical axis can be adjusted to solve the problem of inconsistent center images of the two-channel screens. However, this approach not only requires manual intervention in optical components, increasing labor costs, but also the physical space of the camera device is limited, the disassembly of the camera is difficult, and manually adjusting the position of the sensor is too complex. This approach is not only time-consuming and laborious, but also has low calibration efficiency. Summary of the Invention

[0003] The embodiments of this application provide an optical axis calibration method and device for an image acquisition device, which uses an acquired image as a reference image and crops other images according to the reference image, so that the center images of the cropped images are consistent with the center image of the reference image, realizing more convenient and accurate optical axis calibration for the image acquisition device, with simple operation and no need for manual intervention in optical components, thereby improving the calibration efficiency and saving labor costs.

[0004] An optical axis calibration method for an image acquisition device provided by the embodiments of this application includes:

[0005] Obtain the images simultaneously acquired by each sensor, and determine whether the difference between the center images of the respective images is within a preset range;

[0006] When the difference is not within the preset range, use one of the images as a reference image, and crop the other images according to the reference image, so that the difference between the center image of the cropped image and the center image of the reference image is within the preset range.

[0007] By this method, the images simultaneously collected by each of the sensors are obtained, and it is determined whether the difference between the central images of the respective images is within a preset range; when the difference is not within the preset range, one of the images is used as a reference image, and based on the reference image, the other images are cropped so that the difference between the central images of the cropped images and the central image of the reference image is within the preset range, realizing more convenient and accurate optical axis calibration for the image acquisition device, with simple operation, thereby improving the calibration efficiency and saving labor costs.

[0008] In some embodiments, determining whether the difference between the central images of the respective images is within a preset range includes:

[0009] Mark the central position of each of the images and output a user interface, on which the images with the marks are displayed; and buttons for the user to select whether optical axis calibration is required;

[0010] When an instruction to select the button for optical axis calibration is received through the user interface, it is determined that the difference is not within the preset range.

[0011] By this method, it is realized to determine whether optical axis calibration is required by comparing the central images of the collected images.

[0012] In some embodiments, the mark for each of the images is a first crosshair, and the intersection point of the first crosshair is located at the center of the image;

[0013] On the other images displayed on the user interface, cropping mark lines are also displayed, and the cropping mark lines include a cropping area frame and a second crosshair, and the intersection point of the second crosshair is the center point of the area frame;

[0014] The user interface also includes up, down, left, and right adjustment buttons for optical axis calibration. When the user clicks any one of the adjustment buttons, the cropping mark lines move on the other images in the corresponding direction;

[0015] The cropping of the other images includes: cropping the other images according to the position of the cropping area frame of the cropping mark lines.

[0016] By this method, it is realized that after aligning the central positions of the other images with the central position of the reference image by moving the position of the cropping area frame, the other images are cropped according to the cropping area frame.

[0017] In some embodiments, on the other images displayed on the user interface, a moving range frame is also displayed, and the moving range frame is used to mark the moving range of the center point of the second crosshair.

[0018] In some embodiments, the position of the moving range box is determined according to the center point position, width, and height of the other image. The center point of the moving range box is the center point of the other image. The height of the moving range box accounts for a preset ratio of the height of the other image, and the width of the moving range box accounts for a preset ratio of the width of the other image.

[0019] By this method, the center point of the moving range box is determined according to the center point of the other image, and the size of the moving range box is determined according to the width and height of the other image.

[0020] In some embodiments, cropping the other image according to the position of the cropping area box along the cropping mark line specifically includes:

[0021] Converting the coordinates of the center point of the second crosshair on the user interface into the screen coordinates on the other image, and determining the cropping start point coordinates of the other image based on the screen coordinates;

[0022] According to the cropping start point coordinates, determining whether the cropping area exceeds the boundary of the other image; if so, adjusting the position of the cropping area box to within the boundary of the other image; otherwise, cropping the other image according to the cropping start point and the width and height of the cropping area.

[0023] By this method, coordinate conversion is performed on the center point of the second crosshair, the cropping start point coordinates are determined using the converted center point coordinates, and the other image is cropped according to the cropping start point.

[0024] In some embodiments, the reference image is a thermal imaging image, and the other image is a visible light image;

[0025] When the magnification of the lens of the image acquisition device for acquiring visible light images changes, cropping the other image according to the position of the cropping area box along the cropping mark line further includes:

[0026] Obtaining the current magnification of the lens and the visible light image at the current magnification;

[0027] Converting the coordinates of the center point of the second crosshair on the user interface into coordinates at the current magnification; and converting the coordinates at the current magnification into the screen coordinates on the visible light image at the current magnification, and determining the cropping start point coordinates of the visible light image at the current magnification based on the screen coordinates on the visible light image at the current magnification;

[0028] According to the cropping starting point coordinates of the visible light image at the current magnification, determine whether the cropping area exceeds the boundary of the visible light image at the current magnification; if so, adjust the position of the cropping area frame to within the boundary of the visible light image at the current magnification; otherwise, crop the visible light image at the current magnification according to the cropping starting point coordinates of the visible light image at the current magnification and the width and height of the cropping area.

[0029] Through this method, the visible light image collected by the zoom lens is cropped to achieve optical axis calibration.

[0030] Another embodiment of the present application provides an optical axis calibration device for an image acquisition device, which includes a memory and a processor. Among them, the memory is used to store program instructions, and the processor is used to call the program instructions stored in the memory and execute any of the above methods according to the obtained program.

[0031] In addition, according to an embodiment, for example, a computer program product for a computer is provided, which includes software code portions that, when the product runs on the computer, are used to execute the steps of the method defined above. The computer program product may include a computer-readable medium on which the software code portions are stored. In addition, the computer program product may be directly loaded into the internal memory of the computer and / or sent via the network through at least one of an upload process, a download process, and a push process.

[0032] Another embodiment of the present application provides a computer-readable storage medium, and the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to cause the computer to execute any of the above methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for description in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0034] Figure 1 A visible light image screen and a thermal imaging image screen collected by a thermal imaging camera provided for an embodiment of the present application;

[0035] Figure 2 A schematic overall flow chart of an optical axis calibration method for an image acquisition device provided for an embodiment of the present application;

[0036] Figure 3 A schematic diagram of an electronic optical axis calibration operation interface provided for an embodiment of the present application;

[0037] Figure 4 A visible light image screen and a thermal imaging image screen displayed after optical axis calibration provided by an embodiment of the present application;

[0038] Figure 5a A schematic diagram of a cropping starting point not within the boundary of an image provided by an embodiment of the present application;

[0039] Figure 5b A schematic diagram of moving the cropping starting point to within the boundary of the image provided by an embodiment of the present application;

[0040] Figure 6a A schematic diagram of the lower right point of a cropping area not within the boundary of the image provided by an embodiment of the present application;

[0041] Figure 6b A schematic diagram of moving the lower right point to within the boundary of the image provided by an embodiment of the present application;

[0042] Figure 7 A specific process schematic diagram of an optical axis calibration method for a fixed - focus camera (the size of the visible light image screen collected by the camera remains unchanged) provided by an embodiment of the present application;

[0043] Figure 8 A specific process schematic diagram of an optical axis calibration method for a zoom camera (the size of the visible light image screen collected by the camera can be changed) provided by an embodiment of the present application;

[0044] Figure 9 A structural schematic diagram of an optical axis calibration device for an image acquisition device provided by an embodiment of the present application. Specific embodiments

[0045] Next, in combination with the accompanying drawings in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0046] The embodiments of the present application provide an optical axis calibration method and device for an image acquisition device, which use an acquired image as a reference image and crop other images according to the reference image, so that the central images of the cropped images are consistent with the central image of the reference image, realizing more convenient and accurate optical axis calibration of the image acquisition device. The operation is simple and does not require manual intervention in optical components, thereby improving the calibration efficiency and saving labor costs.

[0047] Among them, the method and the apparatus are based on the same inventive concept. Since the principles for solving problems by the method and the apparatus are similar, the implementation of the apparatus and the method can be referred to each other, and the repeated parts will not be described again.

[0048] In the description of the embodiments of the present application, the terms "first", "second", etc. (if any) in the specification, claims and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments described here can be implemented in an order different from that shown or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0049] The following examples and embodiments should only be understood as illustrative examples. Although the present specification may mention "one", "a" or "some" examples or embodiments in several places, this does not mean that each such mention is related to the same example or embodiment, nor does it mean that the feature only applies to a single example or embodiment. The individual features of different embodiments can also be combined to provide other embodiments. In addition, terms such as "comprising" and "including" should be understood not to limit the described embodiments to only the features that have been mentioned; such examples and embodiments may also include features, structures, units, modules, etc. that have not been specifically mentioned.

[0050] The following will describe each embodiment of the present application in detail with reference to the drawings of the specification. It should be noted that the display order of the embodiments of the present application only represents the sequence of the embodiments, and does not represent the superiority or inferiority of the technical solutions provided by the embodiments.

[0051] It should be noted that in the technical solution provided by the embodiments of the present application, the example of calibrating the optical axes of a network camera with two optical axes, such as a thermal imaging camera, is used for illustration, but it is not limited thereto.

[0052] The following will explain some terms appearing in the text:

[0053] 1. In the embodiments of the present application, the term "thermal imaging camera" refers to a camera that displays by receiving infrared rays emitted by an object. Generally, it has two image sensors (sensors for collecting images). One sensor is responsible for collecting visible light images, and the other sensor is responsible for collecting thermal imaging images.

[0054] 2. In the embodiments of the present application, the term "optical axis" refers to the axis of symmetry of an optical system, which is an axis passing through the center of a lens and perpendicular to the lens plane, and can also be understood as the center point of the imaging screen of a camera. A dual optical axis means that a camera has two optical axes, that is, two center points of the imaging screens. For example, in a thermal imaging camera, one center point is for the visible light image screen, and the other center point is for the thermal imaging image screen.

[0055] 3. In the embodiments of the present application, the term "optical axis calibration" refers to adjusting the position of the optical axis of a camera so that the center position of the visible light image screen is the same as the center position of the thermal imaging image screen, that is, the images at the center positions of the two screens, namely the center images, are the same.

[0056] 4. In the embodiments of the present application, the term "fixed focus" means that the focal length of the visible light lens of a camera is fixed and cannot perform a zoom operation on the visible light image screen, and the scene range seen through the camera lens always remains unchanged; "zoom" means that the focal length of the visible light lens of a camera can be changed within a certain range and can perform a zoom (i.e., magnification) operation on the visible light image screen, and the scene range seen through the camera lens can change, and the lens can see distant scenes.

[0057] 5. In the embodiments of the present application, the term "enable" refers to the operation button for controlling the optical axis calibration function of a camera. Selecting "on" means turning on the optical axis calibration function of the camera, and the camera device processor trims the visible light image; selecting "off" means turning off the optical axis calibration function of the camera; "enable change" means that the state of the optical axis calibration function of the camera has changed, for example, from on to off, or from off to on.

[0058] A thermal imaging camera generally has two sensors, one responsible for collecting visible light images and the other responsible for collecting thermal imaging images. See Figure 1 (The left side is the visible light image screen, and the right side is the thermal imaging image screen). Normally, the center image of the visible light image screen collected by the thermal imaging camera is the same as the center image of the thermal imaging image screen collected. However, sometimes during the assembly of the thermal imaging camera, due to the assembly of multiple components, assembly errors inevitably occur, resulting in a deviation in the installation positions of the two sensors, causing the center images of the two image screens collected through the visible light channel and the thermal imaging channel to be inconsistent, thus making the imaging effect of the thermal imaging camera unable to reach the expected effect.

[0059] To solve the above problems, physical means are generally adopted, such as disassembling the camera and reinstalling the sensor, adjusting the positions of the two sensors, so that the central images of the image frames collected by the thermal imaging camera through these two channels are consistent. However, this method is greatly affected by the difficulty of disassembling the camera, not only time-consuming and laborious, but also unable to improve the calibration efficiency and save labor costs. Therefore, to overcome the above problems, the embodiments of the present application propose an electronic optical axis calibration method, which does not require disassembling the camera. By software such as an optical axis calibration device, the central image of the visible light image frame collected by the camera is adjusted, so that the central image of the visible light image frame can be made consistent with the central image of the thermal imaging image frame, realizing optical axis calibration.

[0060] Generally, the visible light image frame collected by the thermal imaging camera is larger than the thermal imaging image frame collected. Electronic optical axis calibration is to find the central point position on the visible light image frame that is the same as the central point position of the thermal imaging image frame, and use this central point position as the cropping central point to crop the visible light image, and display the cropped visible light image, so as to achieve the purpose that the central image of the cropped visible light image frame is consistent with the central image of the thermal imaging image frame, thereby achieving the purpose of optical axis calibration. Therefore, the basic principle of electronic optical axis calibration is to ensure that the optical axis of the visible light channel is consistent with the optical axis of the thermal imaging channel by cropping the visible light image.

[0061] See Figure 2 , the optical axis calibration method for an image acquisition device provided by the embodiments of the present application includes:

[0062] Step S101, obtain the images collected by each of the sensors simultaneously, and determine whether the difference between the central images of the respective images is within a preset range;

[0063] Among them, the image acquisition device is, for example, the above-mentioned thermal imaging camera; the image is, for example, the visible light image collected by the visible light sensor of the above-mentioned thermal imaging camera and the thermal imaging image collected by the thermal imaging detector;

[0064] That is to say, by comparing the images (i.e., the central images) at the central positions of the collected images, it is determined whether the central images of the respective images are substantially the same;

[0065] Determine whether the difference between the central images of each of the said images is within a preset range. For example, image recognition technology can be used to determine the images at the central positions of each image, and then compare whether the images at the central positions of each image are approximately the same. For example, compare whether the difference between the images in a preset range around the central points of each image is less than a preset value. That is to say, it can be automatically determined whether optical axis calibration is required through an image recognition algorithm; it is also possible to determine the central positions of each image through web image display technology, and then output the marks at the central positions to the corresponding images, so that the user can compare whether the images at the central positions of each image are approximately the same and whether the optical axis needs to be adjusted, that is, perform optical axis calibration according to the user's instructions. The embodiments of the present application do not impose any restrictions on which of the above methods is specifically used.

[0066] Step S102, when the difference is not within the preset range, use one of the images as a reference image, and crop the other images according to the reference image so that the difference between the central images of the cropped images and the central image of the reference image is within the preset range;

[0067] Among them, the reference image is, for example, a thermal imaging image collected by a thermal imaging lens;

[0068] That is to say, if the central images of each image are not the same, use one of the images (such as a thermal imaging image) as a reference image to crop the other images (such as visible light images) so that the central images of the cropped other images are approximately the same as the central image of the reference image;

[0069] Through step S102, it is realized to use one collected image as a reference image to crop the other images, so that the central images of the cropped images are consistent with the central image of the reference image, thereby realizing more convenient and accurate optical axis calibration of the image acquisition device, improving the calibration efficiency and saving labor costs.

[0070] To determine whether optical axis calibration of the image acquisition device is required, in some embodiments, determining whether the difference between the central images of each of the said images is within a preset range includes:

[0071] Mark the central position of each of the said images and output a user interface, and display the images with the marks on the user interface; and a button for the user to select whether optical axis calibration is required;

[0072] When receiving an instruction from the user to select the button for optical axis calibration through the user interface, determine that the difference is not within the preset range;

[0073] Among them, for the user interface, for example, refer to Figure 3, on the left is the visible light image collected by the visible light lens of the camera, and on the right is the thermal imaging image collected by the thermal imaging lens of the camera; the markings, for example Figure 3 The first black cross line and the second black cross line shown in Figure 3 , where the first black cross line represents the current center position of the visible light image screen, and the second black cross line represents the current center position of the thermal imaging image screen; the button, for example Figure 3 The enable shown in Figure 3 . Clicking to enable means that the user selects to perform optical axis calibration. Then click OK, and the user interface will send the instruction of optical axis calibration to the camera device processor for processing;

[0074] Among them, if the difference is not within the preset range, that is, it is judged whether the images near the intersection points of the two black cross lines in Figure 3 Figure 3 (i.e., the central images) are roughly the same, and even whether the images at these two intersection points are the same can be judged. Ideally, the images at the intersection points are the same, but a certain difference is also acceptable. The embodiments of the present application do not limit the prerequisite conditions for performing optical axis calibration (the specific value range of the preset range).

[0075] To realize cropping other images on the operation interface, in some embodiments, the marking of each said image is a first cross line, and the intersection point of the first cross line is located at the center of the image;

[0076] On the other images displayed on the user interface, cropping marking lines are also displayed. The cropping marking lines include a cropping area frame and a second cross line, and the intersection point of the second cross line is the center point of the area frame;

[0077] The user interface also includes adjustment buttons for the four directions of up, down, left, and right for optical axis calibration. When the user clicks any one of the adjustment buttons, the cropping marking lines move on the other images in the corresponding direction;

[0078] The cropping of other images includes: cropping the other images according to the position of the cropping area frame of the cropping marking lines;

[0079] Among them, the first cross line, for example Figure 3 The first black cross line and the second black cross line shown in Figure 3 ; the cropping marking lines, for example Figure 3 The white cross line and the white large frame in the visible light image shown in Figure 3 ; the cropping area frame, for example Figure 3 The white large frame shown in Figure 3 ; the second cross line, for example Figure 3 The white cross line shown in Figure 3 . The intersection point of the white cross line is the center point of the cropping area frame (i.e., the white large frame); the adjustment buttons for the four directions of up, down, left, and right, for example Figure 3The four up, down, left, and right arrows for electronic optical axis calibration shown in [Figure], when any one of the arrows is clicked, the cropping mark line moves on the left visible light image in the direction indicated by the arrow; the position of the cropping area frame, for example Figure 3 the border of the large white frame shown in [Figure], crops the left visible light image according to this border to obtain the cropped visible light image, see Figure 4 .

[0080] In some embodiments, on the other image displayed on the user interface, a movement range frame is also displayed, and the movement range frame is used to mark the movement range of the center point of the second crosshair;

[0081] wherein, the movement range frame, for example Figure 3 the small white frame shown in [Figure].

[0082] That is to say, the center point of the second crosshair moves within the movement range frame and cannot move outside the movement range frame. For example Figure 3 the intersection point of the white crosshair shown in [Figure] can only move within the small white frame.

[0083] To implement determining the movement range of the center point of the second crosshair according to the other image, in some embodiments, the position of the movement range frame is determined according to the center point position, width, and height of the other image. The center point of the movement range frame is the center point of the other image. The height of the movement range frame accounts for a preset ratio of the height of the other image, and the width of the movement range frame accounts for a preset ratio of the width of the other image;

[0084] Among them, the preset ratio of the height and the preset ratio of the width can be the same, for example, both can be 10% or other values, or they can be different, for example, one is 10% and the other is other values, which are set according to needs;

[0085] For example Figure 3 the intersection point of the first black crosshair shown in [Figure] (i.e., the center point of the current visible light image) is the center point of the small white frame (movement range frame). The width of the small white frame accounts for 10% of the width of the visible light image, and the height of the small white frame accounts for 10% of the height of the visible light image.

[0086] To implement determining the cropping starting point coordinates using the center point of the second crosshair and cropping the other image according to the cropping starting point, in some embodiments, cropping the other image according to the position of the cropping area frame of the cropping mark line specifically includes:

[0087] Convert the coordinates of the center point of the second crosshair on the user interface into the screen coordinates on the other image (for example, use Equation 1 in Embodiment 1 for conversion), and determine the coordinates of the starting point of cropping of the other image based on the screen coordinates (for example, use Equation 2 in Embodiment 1 to determine);

[0088] According to the coordinates of the starting point of cropping, determine whether the cropping area exceeds the boundary of the other image; if so, adjust the position of the cropping area frame to within the boundary of the other image; otherwise, crop the other image according to the starting point of cropping and the width and height of the cropping area;

[0089] Among them, the starting point of cropping, for example Figure 3 the upper left point among the 4 vertices of the large white frame shown in Figure 3 whether the large white frame shown in

[0090] exceeds the boundary of the visible light image; if it does, the large white frame needs to be adjusted to within the boundary of the visible light image; According to the coordinates of the starting point of cropping, determine whether the cropping area exceeds the boundary of the other image. For example, it can be determined whether the starting point of cropping is within the boundary of the other image; it can also be to determine the coordinates of the lower right point of the cropping area frame according to the coordinates of the starting point of cropping, and then determine whether the lower right point is within the boundary of the other image; that is to say, as long as one of the starting point of cropping and the lower right point is not within the boundary of the other image, it can be considered that the cropping area exceeds the boundary of the other image;

[0091] For example, the starting point of cropping is not within the boundary of the other image, see Figure 5a , the black frame represents the area where the other image is located, the gray frame represents the cropping area frame, assuming that the coordinates of the starting point of cropping are (5, -5), for example, translate the gray frame downward by 5 until the ordinate is 0 (that is, adjust the abscissa of the starting point of cropping to within the range of [0, (SW - sw)], and the ordinate to within the range of [0, (CH - ch)], where SW represents the width of the image, SH represents the height of the image, cw represents the width of the cropping area, and ch represents the height of the cropping area), so that the cropping area is within the boundary of the other image, see Figure 5b ;

[0092] For example, the lower right point is not within the boundary of the other image, see Figure 6a, the black frame indicates the area where the other image is located, and the gray frame indicates the cropping area. Assume the width of the image is 15 cm and the height is 10 cm, and the coordinates of the lower right point are (20, 15). For example, translate the gray frame 5 to the left (the difference between the abscissa of the lower right point and the width of the image), and then translate the gray frame 5 upward (the difference between the ordinate of the lower right point and the height of the image). It is also possible to first translate the gray frame 5 upward and then 5 to the left (that is, adjust the abscissa of the lower right point within the range of [cw, SW] and the ordinate within the range of [ch, CH]), so that the cropping area is within the boundary of the other image. See Figure 6b , without limitation;

[0093] Among them, for the coordinates of the lower right point, the abscissa is the sum of the abscissa of the cropping starting point and the width of the cropping area, and the ordinate is the sum of the ordinate of the cropping starting point and the height of the cropping area; the width of the cropping area, for example, accounts for 90% of the width of the other image; the height of the cropping area, for example, accounts for 90% of the height of the other image; where 90% represents an empirical value.

[0094] To implement cropping of the visible light image collected by the zoom lens, in some embodiments, the reference image is a thermal imaging image, and the other image is a visible light image;

[0095] When the magnification of the lens of the image acquisition device for collecting visible light images changes, cropping the other image according to the position of the cropping area frame of the cropping mark line further includes:

[0096] Obtain the current magnification of the lens and the visible light image at the current magnification;

[0097] Convert the coordinates of the center point of the second crosshair on the user interface to the coordinates at the current magnification (for example, use formula three in Embodiment 2 for conversion); and convert the coordinates at the current magnification to the screen coordinates on the visible light image at the current magnification (for example, use formula one in Embodiment 1 for conversion), and determine the cropping starting point coordinates of the visible light image at the current magnification based on the screen coordinates on the visible light image at the current magnification (for example, use formula two in Embodiment 1 to determine);

[0098] According to the cropping starting point coordinates of the visible light image at the current magnification, determine whether the cropping area exceeds the boundary of the visible light image at the current magnification; if so, adjust the position of the cropping area frame to within the boundary of the visible light image at the current magnification; otherwise, crop the visible light image at the current magnification according to the cropping starting point coordinates of the visible light image at the current magnification and the width and height of the cropping area.

[0099] Several embodiments of specific method flows are given below.

[0100] Since many visible light lenses of thermal imaging cameras are zoom lenses, zoom operations will be performed during shooting. For the same object being photographed, there are differences in the visible light image frames collected in real time at different magnifications, that is, the central images of the frames are different. When calibrating the optical axis of a camera with a zoom visible light lens, real-time calibration during zooming is required; while a camera with a fixed-focus visible light lens does not have a zoom operation during shooting, and the visible light images collected in real time are the same, so real-time calibration during zooming is not required. Therefore, there are differences in the specific calibration processes for the optical axis calibration of these two different devices: the camera with a fixed-focus visible light lens and the camera with a zoom visible light lens.

[0101] Embodiments for implementing optical axis calibration in combination with the operation instructions of the user on the user interface are given below.

[0102] Embodiment 1:

[0103] See Figure 7 , for the above-mentioned thermal imaging camera with a fixed-focus visible light lens, the specific steps of an optical axis calibration method for a fixed-focus thermal imaging camera provided by an embodiment of the present application include:

[0104] Step S201: Obtain the visible light image and the thermal imaging image collected by the camera, and use the thermal imaging image to determine the coordinates of the center point for cropping the visible light image;

[0105] Among them, the visible light image is collected by the visible light sensor of the camera; the thermal imaging image is collected by the thermal imaging detector of the camera;

[0106] To determine the coordinates of the center point for cropping the visible light image using the thermal imaging image, for example, web image display technology can be used to determine the center position of the thermal imaging image, and then find the same center position in the visible light image according to this center position, and then determine its center point based on the image at this center position. This center point is the center point for cropping the visible light image. Finally, determine its coordinates (i.e., the coordinates of the center point for cropping) according to this center point for cropping. There is no limitation here;

[0107] Step S202: Set calibration parameters, for example, including enabling (i.e., enabling optical axis calibration) and setting the coordinates of the center point for cropping the visible light image;

[0108] The enabling is generally in the default enabled state. When the central image of the cropped visible light image obtained through subsequent steps is different from the central image of the thermal imaging image (i.e., the optical axis calibration effect is not ideal), the enabling state can be first changed to disabled and then enabled;

[0109] Step S203: Convert the coordinates of the center point of the visible light image cropping according to the following Formula 1, that is, convert the coordinates of the center point of the cropping at the display resolution of the image display terminal (such as 8192*8192) into the coordinates at the current resolution of the visible light image;

[0110] x = m * SW / 8192, y = n * SH / 8192 Formula 1

[0111] Among them, the current resolution is the resolution of the visible light image screen collected by the camera obtained in Step S201. Through this resolution, the coordinates of the center point of the cropping at its resolution can be calculated;

[0112] x and y respectively represent the abscissa and ordinate of the center point of the visible light image cropping at the current resolution; m and n respectively represent the abscissa and ordinate of the center point of the visible light image cropping at a resolution of 8192*8192, that is to say, m and n can also represent the abscissa and ordinate of the center point of the visible light image cropping at other resolutions, and only need to replace 8192 in Formula 1 with the corresponding resolution; SW and SH respectively represent the current width and current height of the visible light image, and are obtained by acquiring the resolution currently used by the camera to collect the visible light image through the communication interface provided by the camera device;

[0113] Step S204: Calculate the starting point of the visible light image cropping according to the following Formula 2, determine the coordinates of the lower right point of the cropping area according to the coordinates of the starting point of the cropping, and then determine whether the starting point of the cropping and / or the lower right point exceed the boundary of the visible light image screen, that is, determine whether the cropping area is out of bounds. If it is not out of bounds, perform Step S205; if it is out of bounds, first adjust the starting point of the cropping and / or the lower right point to within the boundary of the visible light image screen (that is, adjust the cropping area to within the boundary of the visible light image screen), and then perform Step S205;

[0114] sx = x – cw / 2, sy = y – ch / 2 Formula 2

[0115] Among them, sx and sy respectively represent the abscissa and ordinate of the starting point of the cropping; x and y respectively represent the abscissa and ordinate of the center point of the cropping at the current resolution; cw and ch respectively represent the width and height of the cropping area, and can be obtained, for example, by converting according to 90% (empirical value) of the current resolution;

[0116] For example, based on the center point of the cropping, shift the x coordinate of the center point of the cropping to the left by half of the width of the cropping area to obtain the sx coordinate of the starting point of the cropping, and shift the y coordinate of the center point of the cropping upward by half of the height of the cropping area to obtain the sy coordinate of the starting point of the cropping;

[0117] Then the abscissa rx of the lower right point is rx = sx + cw, and the ordinate ry is ry = sy + ch;

[0118] That is to say, based on the cropping starting point, the abscissa of the lower right point is obtained by shifting the abscissa of the cropping starting point to the right by the width of the cropping area, and the ordinate of the lower right point is obtained by shifting the ordinate of the cropping starting point downward by the height of the cropping area;

[0119] If sx and / or sy are negative values, then the cropping starting point is not within the boundary of the visible light image frame. It is necessary to move the cropping starting point. For example, adjusting the coordinate with a negative value to 0 can make the cropping starting point fall within the boundary of the visible light image frame, thereby adjusting the position of the cropping area frame to within the boundary of the visible light image frame;

[0120] If rx is greater than SW and / or ry is greater than SH, then the lower right point is not within the boundary of the visible light image frame. It is necessary to move the lower right point. For example, adjusting rx to SW and ry to SH can make the lower right point fall within the boundary of the visible light image frame, thereby adjusting the position of the cropping area frame to within the boundary of the visible light image frame;

[0121] Step S205: Set the cropping parameters and send a cropping instruction to the camera processor, where the cropping instruction includes the cropping parameters; the cropping parameters include: the cropping starting point coordinates, the width and height of the cropping area;

[0122] Step S206: The camera device processor crops the visible light image according to the above cropping instruction and displays the cropped visible light image; among them, the central image of the cropped visible light image is consistent with the central image of the thermal imaging image obtained in step S201.

[0123] Embodiment 2:

[0124] See Figure 8 For the above thermal imaging camera with a zoom lens as the visible light lens, the specific steps of an optical axis calibration method for a zoom thermal imaging camera provided by an embodiment of the present application include:

[0125] Step S301: Obtain the visible light image and the thermal imaging image collected by the camera, and use the thermal imaging image to determine the coordinates of the cropping center point of the visible light image;

[0126] Step S302: Set the initialization calibration parameters, for example, include: setting the coordinates of the cropping center point of the visible light image, the cropping starting point coordinates, the resolution of the visible light image frame, the real-time magnification value, the magnification value during optical axis calibration (the magnification value corresponding to the enabled state), the initial value of the current enable value;

[0127] First, according to the above formula 1, convert the coordinates of the center point of the visible light image cropping at the display resolution of the image display terminal (e.g., 8192*8192) into the coordinates at the current resolution of the visible light image, and then calculate the coordinates of the cropping starting point according to formula 2 for initialization settings; the resolution of the visible light image screen is obtained through the communication interface provided by the camera device; the current enable value, e.g., 1 or 0, where 1 indicates enabling and 0 indicates disabling;

[0128] Step S303: Obtain the real-time magnification value of the visible light lens of the camera through the communication interface provided by the camera device;

[0129] Step S304: Receive the enable value and the coordinates of the center point of the visible light image cropping sent by the camera device, and synchronously update the initialization calibration parameters set in step S302 in real time;

[0130] When the visible light lens performs a zoom operation, the visible light image screen collected at different magnification values is different, so the coordinates of the center point of the visible light image cropping are also different, and the coordinates of the cropping starting point calculated subsequently are also different;

[0131] Step S305: Determine whether the enable state has changed by comparing the currently received enable value with the previously received enable value (i.e., the user's previous operation instruction on the enable button). If the enable value has not changed (i.e., the current enable value is the same as the previous enable value), it means that the enable has been in the on or off state all the time, and go to step S306; if the enable value has changed (i.e., the current enable value is different from the previous enable value), then go to step S307;

[0132] It should be noted that if there is no previous enable value, i.e., the first time for optical axis calibration, the previous enable value is defaulted to 0;

[0133] Step S308: Enter the sleep state, and after the preset sleep duration, re-perform step S303;

[0134] Step S307: Determine whether the enable is in the off state (i.e., the optical axis calibration function is in the off state). If the enable is off, go to step S312; if the enable is on, go to step S310;

[0135] Step S308: Enter the sleep state, and after the preset sleep duration, re-perform step S303;

[0136] Step S309: Compare the real-time magnification value obtained in step S303 with the real-time magnification value obtained last time to determine whether the visible light lens has zoomed. If the visible light lens has zoomed, go to step S307; if the visible light lens has not zoomed, go to step S303;

[0137] Step S310: First, convert the coordinates of the cropping center point at the magnification value during optical axis calibration into coordinates at the current real-time magnification value according to the following formula 3, and then convert the coordinates at the current real-time magnification value into coordinates at the current resolution of the visible light image screen according to the above formula 1;

[0138] m’ = m + (m - 4096) * zoom1 / zoom2 Formula 3

[0139] n’ = n + (n - 4096) * zoom1 / zoom2

[0140] where, zoom1 represents the current real-time magnification value of the visible light lens of the camera; zoom2 represents the magnification value during optical axis calibration setting; m and n respectively represent the abscissa and ordinate of the cropping center point at the magnification value during calibration setting; m’ and n’ respectively represent the abscissa and ordinate of the cropping center point at the current real-time magnification value;

[0141] m, n, m’, n’ are coordinates at the display resolution of the image display terminal, for example;

[0142] Step S311: Use the converted coordinates of the cropping center point obtained in step S310 to calculate the cropping start point coordinates according to the above formula 2;

[0143] Step S312: Set the cropping parameters and send a cropping instruction to the camera processor, where the cropping instruction includes the cropping parameters; the cropping parameters include: the cropping start point coordinates, the width and height of the cropping area;

[0144] Step S313: The camera device processor crops the visible light image according to the cropping instruction to obtain a cropped visible light image for display; where the central image of the cropped visible light image is consistent with the central image of the thermal imaging image obtained in step S301.

[0145] Each time the visible light lens of the camera zooms, the visible light image screen collected by the visible light lens changes, the coordinates of the visible light image cropping center point change accordingly, and then the cropping start point coordinates change. It is necessary to reset the cropping start point coordinates in the cropping parameters once. The camera device processor re-crops the visible light image using the new cropping parameters until the visible light lens stops zooming.

[0146] The following introduces the device or apparatus provided in the embodiments of the present application. For the explanations or examples of the same or corresponding technical features as those described in the above method, they will not be repeated hereinafter.

[0147] Refer to Figure 9 , an optical axis calibration device for an image acquisition device provided in the embodiments of the present application. The device can be a module in the image acquisition device for executing the above method, or an independent device outside the image acquisition device for executing the above method. The device includes:

[0148] A processor 600, configured to read a program in a memory 620 and execute the following processes:

[0149] Acquire the images simultaneously collected by each of the sensors, and determine whether the difference between the central images of the respective images is within a preset range;

[0150] When the difference is not within the preset range, use one of the images as a reference image, and crop the other images according to the reference image so that the difference between the central images of the cropped images and the central image of the reference image is within the preset range.

[0151] In some embodiments, determining whether the difference between the central images of the respective images is within a preset range includes:

[0152] Mark the central position of each of the images, and output a user interface, and display the images with the marks on the user interface; and a button for the user to select whether to perform optical axis calibration;

[0153] When an instruction to select the button for performing optical axis calibration is received through the user interface, it is determined that the difference is not within the preset range.

[0154] In some embodiments, the mark for each of the images is a first crosshair, and the intersection point of the first crosshair is located at the center of the image;

[0155] On the other images displayed on the user interface, crop marking lines are further displayed. The crop marking lines include a crop region frame and a second crosshair, and the intersection point of the second crosshair is the center point of the region frame;

[0156] The user interface further includes up, down, left, and right adjustment buttons for optical axis calibration. When the user clicks any one of the adjustment buttons, the crop marking lines move on the other images in the corresponding direction;

[0157] The cropping of the other images includes: cropping the other images according to the position of the crop region frame of the crop marking lines.

[0158] In some embodiments, a movement range box is further displayed on the other image displayed on the user interface, and the movement range box is used to mark the movement range of the center point of the second crosshair.

[0159] In some embodiments, the position of the movement range box is determined according to the center point position, width, and height of the other image. The center point of the movement range box is the center point of the other image. The height of the movement range box accounts for a preset ratio of the height of the other image, and the width of the movement range box accounts for a preset ratio of the width of the other image.

[0160] In some embodiments, cropping the other image according to the position of the cropping area box of the cropping marking line specifically includes:

[0161] Converting the coordinates of the center point of the second crosshair on the user interface into screen coordinates on the other image, and determining the cropping start point coordinates of the other image based on the screen coordinates;

[0162] Judging whether the cropping area exceeds the boundary of the other image according to the cropping start point coordinates; if so, adjusting the position of the cropping area box to within the boundary of the other image; otherwise, cropping the other image according to the cropping start point and the width and height of the cropping area.

[0163] In some embodiments, the reference image is a thermal imaging image, and the other image is a visible light image;

[0164] When the magnification of the lens for collecting visible light images of the image acquisition device changes, cropping the other image according to the position of the cropping area box of the cropping marking line further includes:

[0165] Obtaining the current magnification of the lens and the visible light image at the current magnification;

[0166] Converting the coordinates of the center point of the second crosshair on the user interface into coordinates at the current magnification; and converting the coordinates at the current magnification into screen coordinates on the visible light image at the current magnification, and determining the cropping start point coordinates of the visible light image at the current magnification based on the screen coordinates on the visible light image at the current magnification;

[0167] According to the cropping starting point coordinates of the visible light image at the current magnification, determine whether the cropping area exceeds the boundary of the visible light image at the current magnification; if so, adjust the position of the cropping area frame to within the boundary of the visible light image at the current magnification; otherwise, crop the visible light image at the current magnification according to the cropping starting point coordinates of the visible light image at the current magnification and the width and height of the cropping area.

[0168] In some embodiments, the optical axis calibration device of the image acquisition device provided by the embodiments of the present application further includes a transceiver 610, configured to receive and send data under the control of the processor 600.

[0169] Among them, in Figure 9 , the bus architecture may include any number of interconnected buses and bridges, specifically various circuits of one or more processors represented by the processor 600 and the memory represented by the memory 620 are linked together. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art, and thus will not be further described herein. The bus interface provides an interface. The transceiver 610 may be multiple elements, that is, including a transmitter and a receiver, and provides a unit for communicating with various other devices on the transmission medium.

[0170] In some embodiments, the optical axis calibration device of the image acquisition device provided by the embodiments of the present application further includes a user interface 630, and the user interface 630 may be an interface capable of externally or internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, etc.

[0171] The processor 600 is responsible for managing the bus architecture and general processing, and the memory 620 may store data used by the processor 600 when performing operations.

[0172] In some embodiments, the processor 600 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a CPLD (Complex Programmable Logic Device).

[0173] Embodiments of the present application provide a computing device, which may specifically be a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (PDA), etc. The computing device may include a central processing unit (CPU), a memory, input / output devices, etc. The input devices may include a keyboard, a mouse, a touch screen, etc., and the output devices may include a display device, such as a liquid crystal display (LCD), a cathode ray tube (CRT), etc.

[0174] The memory may include a read-only memory (ROM) and a random access memory (RAM), and provide program instructions and data stored in the memory to the processor. In the embodiments of the present application, the memory may be used to store the programs of any of the methods provided by the embodiments of the present application.

[0175] By invoking the program instructions stored in the memory, the processor is configured to execute any of the methods provided by the embodiments of the present application according to the obtained program instructions.

[0176] Embodiments of the present application also provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes any of the methods in the above embodiments. The program product may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0177] Embodiments of the present application provide a computer-readable storage medium for storing computer program instructions used for the device provided by the embodiments of the present application, which includes a program for executing any of the methods provided by the embodiments of the present application. The computer-readable storage medium may be a non-transitory computer-readable medium.

[0178] The computer-readable storage medium may be any available medium or data storage device accessible by a computer, including but not limited to magnetic memory (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical memory (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor memory (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drives (SSD)), etc.

[0179] Those skilled in the art should understand that the embodiments of the present application may be provided as a method, a system, or a computer program product. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0180] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0181] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0182] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0183] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these modifications and variations.

Claims

1. A method for calibrating the optical axis of an image acquisition device, the image acquisition device including a plurality of sensors for acquiring images, characterized in that, The method includes: Obtaining the images simultaneously collected by each of the sensors, and determining whether the difference between the central images of the respective images is within a preset range; When the difference is not within the preset range, using one of the images as a reference image, and cropping the other images according to the reference image so that the difference between the central images of the cropped images and the central image of the reference image is within the preset range; Wherein, the reference image is a thermal imaging image, and the other images are visible light images.

2. The method according to claim 1, wherein Determining whether the difference between the central images of the respective images is within a preset range includes: Marking the central position of each of the images, and outputting a user interface to display the images with the marks thereon; and buttons for the user to select whether optical axis calibration is required; When an instruction to select the button for optical axis calibration is received through the user interface, it is determined that the difference is not within the preset range.

3. The method according to claim 2, wherein The mark of each of the images is a first crosshair, and the intersection point of the first crosshair is located at the center of the image; On the other images displayed on the user interface, cropping mark lines are further displayed, and the cropping mark lines include a cropping area frame and a second crosshair, and the intersection point of the second crosshair is the center point of the area frame; The user interface further includes adjustment buttons for the up, down, left, and right directions of optical axis calibration. When the user clicks any one of the adjustment buttons, the cropping mark lines move on the other images in the corresponding direction; Cropping the other images includes: Cropping the other images according to the position of the cropping area frame of the cropping mark lines.

4. The method according to claim 3, wherein On the other images displayed on the user interface, a moving range frame is further displayed, and the moving range frame is used to mark the moving range of the center point of the second crosshair.

5. The method according to claim 4, characterized in that The position of the moving range frame is determined according to the center point position, width, and height of the other image. The center point of the moving range frame is the center point of the other image. The height of the moving range frame accounts for a preset proportion of the height of the other image, and the width of the moving range frame accounts for a preset proportion of the width of the other image.

6. The method according to claim 3, wherein Cropping the other images according to the position of the cropping area frame of the cropping mark lines specifically includes: Converting the coordinates of the center point of the second crosshair on the user interface into the screen coordinates on the other image, and determining the cropping starting point coordinates of the other image based on the screen coordinates; Judging whether the cropping area exceeds the boundary of the other image according to the cropping starting point coordinates; if so, adjusting the position of the cropping area frame to within the boundary of the other image; otherwise, cropping the other image according to the cropping starting point and the width and height of the cropping area.

7. The method according to claim 6, wherein When the magnification of the lens for collecting visible light images of the image acquisition device changes, cropping the other images according to the position of the cropping area frame of the cropping mark lines further includes: Obtaining the current magnification of the lens and the visible light image under the current magnification; Convert the coordinates of the center point of the second crosshair on the user interface into coordinates at the current magnification; and convert the coordinates at the current magnification into screen coordinates on the visible light image at the current magnification, and determine the cropping starting point coordinates of the visible light image at the current magnification based on the screen coordinates on the visible light image at the current magnification; According to the cropping starting point coordinates of the visible light image at the current magnification, determine whether the cropping area exceeds the boundary of the visible light image at the current magnification; if so, adjust the position of the cropping area frame to within the boundary of the visible light image at the current magnification; otherwise, crop the visible light image at the current magnification according to the cropping starting point coordinates of the visible light image at the current magnification and the width and height of the cropping area.

8. An optical axis calibration device, characterized in that, Comprising: A memory for storing program instructions; A processor for calling the program instructions stored in the memory and executing the method according to any one of claims 1 to 7 according to the obtained program.

9. A computer program product for a computer, characterized in that, Comprising a software code portion that, when the product is running on the computer, is used to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing the computer to execute the method according to any one of claims 1 to 7.

Images