A visible-infrared camera image fast registration method for offshore drilling platforms

Abstract

The application discloses a visible-infrared camera image fast registration method for offshore drilling platforms, and belongs to the technical field of image processing. The method first performs stereo calibration on infrared and visible cameras, respectively obtains the internal parameter matrix and distortion coefficient of the two cameras, and a translation vector and a rotation transformation matrix describing the relative pose of the two cameras, and stores them. When imaging the sea surface, the algorithm uses the above parameters to perform stereo correction on the images formed by the two cameras, and obtains the images after polar line alignment. The camera pitch angle and the height information from the sea surface are obtained from the gimbal, the translation amount is calculated row by row, the images after polar line alignment are registered, and the registered sea level images are obtained. The algorithm can quickly realize the registration of sea surface images and meet the demand of real-time video processing.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for rapid registration of visible-infrared camera images for offshore drilling platforms. Background Technology

[0002] Infrared images are less affected by lighting or weather changes, while visible light images typically have high resolution and rich texture. Combining information from both types of images can effectively improve the accuracy of target detection and recognition. The prerequisite for comprehensively utilizing this information is image registration to obtain the geometric correspondence between infrared and visible light images. Existing registration algorithms based on feature extraction and matching, such as Harris corner detection, SIFT, or SURF, still face many difficulties in solving the multi-mode image matching problem between infrared and visible light cameras. On the one hand, infrared and visible images represent two different physical phenomena with completely different appearances, resulting in few common matchable features. On the other hand, infrared images have lower resolution, leading to significant scale differences with visible light images. Furthermore, infrared images are heavily affected by noise, making it difficult to extract reliable features. Finally, the slow matching speed makes them unsuitable for real-time video processing.

[0003] For infrared and visible light cameras with fixed relative positions, image matching can be performed through calibration using known geometric relationships. However, this method has inherent errors. Distance from object Inversely proportional, specifically

[0004]

[0005] in, Focal length For pixel size, This represents the baseline length. If there are significant differences between objects in the image and their distances are unknown, it is difficult to achieve a small matching error across the entire image using a single method.

[0006] When deployed on an offshore drilling platform for sea surface observation, the scenario is unique. The target distance can be calculated using trigonometric relationships by measuring the camera's height above the sea surface and the gimbal's pitch angle.

[0007] Accordingly, those skilled in the art are dedicated to developing a rapid registration method for visible-infrared cameras used on offshore drilling platforms. Summary of the Invention

[0008] In view of the above-mentioned deficiencies of the prior art, the present invention provides a method for rapid registration of visible-infrared camera images for offshore drilling platforms, the technical point of which is that it includes the following steps:

[0009] Step 1: During design and assembly, ensure that the visible light camera and infrared camera are placed horizontally side by side so that their focal planes are parallel and on the same plane;

[0010] Step 2: Acquire images from the visible light camera and the infrared camera, perform stereo calibration, obtain calibration parameters, and save them as a calibration parameter file;

[0011] Step 3: Mount the visible light camera and the infrared camera on the gimbal, place the gimbal on the offshore drilling platform, and acquire the sea surface images from the visible light camera and the infrared camera.

[0012] Step 4: Use the calibration parameter file obtained in Step 2 and the sea surface image obtained in Step 3 to perform stereo calibration to obtain an image with epipolar alignment;

[0013] Step 5: Obtain the pitch angle and height above the sea surface information of the visible light camera and infrared camera on the gimbal mentioned in Step 3, calculate the translation amount line by line, register the epipolar aligned image, and obtain the registered sea surface image.

[0014] In some embodiments of the present invention, the calibration parameters in step two of the above-described method for rapid registration of visible-infrared camera images for offshore drilling platforms include intrinsic parameter matrix, distortion coefficients, and relative position.

[0015] In some embodiments of the present invention, the relative positions of the above-described method for rapid registration of visible-infrared camera images for offshore drilling platforms include translation vectors and rotation transformation matrices of the visible camera image and the infrared camera image.

[0016] In some embodiments of the present invention, step five of the above-described method for rapid registration of visible-infrared camera images for offshore drilling platforms specifically includes the following steps:

[0017] Step 5-1: Calculate the pixel offset of each row of the image captured by the infrared camera to obtain the remapping matrix of each pixel of the infrared camera.

[0018] Step 5-2: Using the remapping matrix obtained in step 5-1, remap the image captured by the infrared camera to obtain an infrared image registered onto the visible light camera image.

[0019] In some embodiments of the present invention, in step 5-1 of the above-described method for rapid registration of visible-infrared camera images for offshore drilling platforms, the first... row offset Calculate using the following formula: ,in, The height of the main point above the sea surface The platform's pitch angle, focal length of the visible light camera (unit: pixel), image with the principal point as the origin and unit: pixel axis coordinates.

[0020] Compared with the prior art, the present application has the following beneficial effects:

[0021] The present application is a visible-infrared camera image fast registration method for offshore drilling platforms, which is based on geometric relationship and solves the problem of image fast registration when imaging the sea surface on offshore drilling platforms, effectively improving the registration speed and making it applicable to real-time video processing. BRIEF DESCRIPTION OF DRAWINGS

[0022] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The illustrative embodiments of the present application and their descriptions serve to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 Flow chart of the present application, a visible-infrared camera image fast registration method for offshore drilling platforms;

[0024] Figure 2 Pitch angle of the visible light camera and the infrared camera on the pan-tilt head in step three and the height h from the sea surface information schematic diagram. DETAILED DESCRIPTION

[0025] To make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below in combination with specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0026] As Figure 1 shown, the present application is a visible-infrared camera image fast registration method for offshore drilling platforms, which includes the following steps:

[0027] Step one: during design and assembly, ensure that the visible light camera and the infrared camera are placed horizontally side by side so that their focal planes are parallel and on the same plane;

[0028] Step two: obtain the visible light camera image and the infrared camera image for stereo calibration, obtain the calibration parameters, and store them as a calibration parameter file; further, the calibration parameters include the intrinsic matrix, the distortion coefficient and the relative position. Further, the relative position of the fast registration method includes the translation vector and the rotation transformation matrix of the visible light camera image and the infrared camera image.

[0029] Step 2-1, using visible light camera and infrared camera, simultaneously imaging the checkerboard calibration board: the calibration board is placed in different poses and at different distances, and the calibration effect is recorded step by step.

[0030] Step 2-2, using calib_gui program in MATLAB calibration toolbox (Camera Calibration Toolbox for Matlab), using the images obtained in step 2-1, calibrating the two cameras respectively, obtaining the intrinsic matrix and distortion coefficient of the two cameras;

[0031] Step 2-3, using stereo_gui program in MATLAB calibration toolbox, and using the calibration parameters obtained in step 2-2, calibrating the two cameras, obtaining the rotation transformation matrix R and translation vector T describing the relative position of the two cameras, and the updated intrinsic matrix K1 and distortion coefficient k1, k2, p1, p2, p3 of the visible light camera, and the intrinsic matrix and distortion parameters of the infrared camera.

[0032] Step three, placing the visible light camera and infrared camera horizontally on the gimbal of the offshore drilling platform to obtain the sea surface image of the visible light camera and the sea surface image of the infrared camera;

[0033] Step four, using the calibration parameter file obtained in step two and the sea surface image obtained in step three to perform stereo rectification to obtain the epipolar aligned image;

[0034] Step 4-1, using the rotation transformation matrix, translation vector, intrinsic matrix and distortion coefficient of the visible light camera, and the intrinsic matrix and distortion coefficient of the infrared camera obtained in step 1-3, inputting into the stereoRectify function in OpenCV to perform stereo rectification on the two cameras;

[0035] Step 4-2, using the initUndistortRectifyMap function in OpenCV and the parameters obtained in step 4-1, performing stereo rectification on the visible light camera and the infrared camera respectively to obtain the remapping matrix;

[0036] Step 4-3, using the remap function in OpenCV and the remapping matrix obtained in step 4-2, remapping the images formed by the visible light camera and the infrared camera respectively to obtain the stereo rectified image.

[0037] Step five, as shown in Figure 2 the pitch angle of the visible light camera and the infrared camera on the gimbal in step three is obtained And the height h information from the sea, the translation of each row of calculation, registration of the image after the epipolar alignment, get the registration of the sea surface image.

[0038] In some embodiments of the present application, the above-mentioned step five of the visible-infrared camera image registration method for offshore drilling platform specifically includes the following steps:

[0039] Step 5-1, calculate the offset d of each row of pixels of the infrared camera image to obtain the remapping matrix of each pixel of the infrared camera;

[0040] Step 5-2, using the remap function in OpenCV and the remapping matrix obtained in step 5-1, remapping the infrared camera image to obtain the infrared image registered to the visible light camera image.

[0041] Further, in step 5-1 of the above-mentioned visible-infrared camera image registration method for offshore drilling platform, the offset d of the first row of the infrared camera image is calculated as follows: According to the following formula: Wherein, is the height of the principal point from the sea surface, is the platform pitch angle, is the focal length of the visible light camera (unit: pixel), is the image axis coordinate with the principal point as the origin and the unit of pixel.

[0042] The above only describes the embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the scope of the claims of the present application.​​

Claims

1. A visible-infrared camera image fast registration method for offshore drilling platforms, characterized in that, The method comprises the following steps: Step 1: during design and assembly, ensure that the visible light camera and the infrared camera are placed horizontally and side by side, with their focal planes parallel and on the same plane; Step 2: acquire visible light camera images and infrared camera images for stereo calibration, obtain calibration parameters, and store them as a calibration parameter file; Step 3: place the visible light camera and the infrared camera on a gimbal, place the gimbal on a sea drilling platform, and acquire sea surface images of the visible light camera and sea surface images of the infrared camera; Step 4: use the calibration parameter file obtained in Step 2 and the sea surface images obtained in Step 3 to perform stereo rectification, and obtain epipolar aligned images; Step 5: acquire the pitch angle and the height from the sea surface of the visible light camera and the infrared camera on the gimbal in Step 3, calculate the translation amount row by row, perform registration on the epipolar aligned images, and obtain registered sea surface images; The specific operation of Step 5 comprises the following steps: Step 5-1: calculate the pixel offset amount of each row of the infrared camera image to obtain a pixel remapping matrix of the infrared camera; Step 5-2: use the remapping matrix obtained in Step 5-1 to perform remapping on the infrared camera image to obtain an infrared image registered to the visible light camera image; In Step 5-1, the offset amount d of the vth row is calculated according to the following formula: where h is the height of the main point from the sea surface, and θ is the platform pitch angle, f y = f / dy is the focal length of the visible light camera in pixels, and v is the image y-axis coordinate in pixels with the principal point as the origin.

2. A method for fast registration of visible-infrared camera images for offshore drilling platforms according to claim 1, characterized in that, In Step 2, the calibration parameters include an intrinsic matrix, distortion coefficients, and a relative position.

3. A method for fast registration of visible-infrared camera images for offshore drilling platforms according to claim 2, characterized in that, The relative position includes a translation vector and a rotation transformation matrix of the visible light camera image and the infrared camera image.

Images