A method for stitching flat hole images taken by multiple cameras

Abstract

The invention provides a method for stitching flat hole images taken by multi-eye cameras, and relates to the technical field of image stitching. According to the parameters of cameras and the geometric relationship between adjacent cameras, the relationship between the overlap rate of images taken by adjacent cameras and the hole diameter is formulated; according to the hole diameter size of the flat hole to be photographed, the overlap rate is determined; the multi-eye camera is placed at the center of the flat hole for shooting, and an overlapping area is determined in two images taken by adjacent cameras; features are extracted in the determined overlapping area to complete image stitching; thereby, the feature extraction area is narrowed, the demand for computing power is reduced, and the problem of high computing power requirement caused by too large a feature search range when stitching flat hole images is solved. The invention is suitable for images shot at flat holes and image stitching.

Description

Technical Field

[0001] The invention relates to the technical field of image splicing, and in particular to a method for splicing flat hole images shot by multi-cameras. Background Art

[0002] Image stitching is a common processing method used to merge multiple images into a continuous scene. This technology is widely used in remote sensing imaging, medical imaging, computer vision and other fields. Before the main construction of the hydropower project, it is also necessary to use the survey flat tunnel to identify the geological conditions of the project to ensure the safety and feasibility of the project. Usually, people enter the flat tunnel manually, use image acquisition equipment to obtain the real scene data on the site, and then post-process the image data, such as building a three-dimensional model to display it. In the process of converting image data into a plane unfolded map, the efficiency and quality of image stitching are particularly important.

[0003] Traditional image stitching methods usually involve complex algorithms and a large amount of computing resources, resulting in slow processing speed. In order to improve the speed and quality of image stitching, researchers have proposed a variety of acceleration technologies. These technologies mainly include two types. One is to improve the algorithm and pre-process the input image to reduce the complexity of subsequent processing; the other is to increase the application of hardware and use hardware resources such as chips, multi-core processors and GPUs to achieve parallel computing in the image stitching process.

[0004] For example, the Chinese patent application number CN202011154641.4, entitled "Aerial Image Stitching Acceleration Method and System Based on Optical Flow Tracking and Frame Mapping", discloses the optimization of the point extraction and matching and image transformation mapping process, reducing the number of image mappings during stitching synthesis, thereby increasing the speed of stitching more continuous images. Although this method has been optimized in terms of algorithm, it can still only run on GPU workstations, and has extremely high requirements for computing power.

[0005] For example, the Chinese patent with application number CN202010535149.5 and invention name "An FPGA chip for panoramic stitching acceleration and panoramic image stitching method" adopts a combination of software and hardware to accelerate image stitching, but it is necessary to add an additional FPGA chip for panoramic stitching acceleration. Through the targeted design of circuit structure, the image stitching can be accelerated, but additional hardware needs to be added, which increases the user's usage cost.

[0006] The length of hydropower survey tunnels is 200m or more. At a frequency of 30cm / image, the overall resolution of the stitched images is extremely high. Therefore, no matter whether conventional feature point search algorithms such as SIFT or ORB or improved and accelerated algorithms are used, when stitching new images onto already stitched high-resolution images, it will inevitably lead to problems such as long feature point search time and low image stitching efficiency. Moreover, when the algorithm is applied to low-power portable devices with low computing power, the calculation time will be infinitely extended at best, or the device will crash at worst. Summary of the invention

[0007] The technical problem solved by the present invention is to provide a method for stitching flat-hole images taken by multiple cameras, so as to solve the problem that the feature search range is too large during the existing flat-hole image stitching, resulting in high computing power requirements.

[0008] The present invention solves the above technical problems by adopting a technical solution: a method for stitching flat hole images taken by multiple cameras, comprising the following steps:

[0009] S1. Obtain camera parameters of a multi-camera and a geometric relationship between adjacent cameras; the camera parameters of the multi-camera include a field of view angle, and the field of view angle of each camera is equal; the geometric relationship between adjacent cameras includes an angle between boundary lines of the fields of view of adjacent cameras;

[0010] S2. According to the parameters of the cameras and the geometric relationship between the adjacent cameras, the relationship between the overlap rate of the images taken by the adjacent cameras and the hole diameter is formulated;

[0011] S3, determining the overlap rate according to the diameter of the flat hole to be photographed;

[0012] S4, placing the multi-camera at the center of the flat hole to shoot, and determining the overlapping area in the two images taken by adjacent cameras;

[0013] S5. Extract features from the determined overlapping areas to complete image stitching.

[0014] Furthermore, in S2, the relationship between the overlap rate of the images taken by the proposed adjacent cameras and the hole diameter is: ,in, Indicates the overlap rate of images taken by adjacent cameras. is the angle between the boundary lines of the adjacent camera fields of view, is the camera’s field of view, and are the lengths of the two boundary lines of the camera’s field of view, and , It is the distance from the intersection of the boundary lines of the adjacent camera fields of view to the camera.

[0015] Furthermore, in S2, the relationship between the overlap rate of the images taken by the proposed adjacent cameras and the hole diameter is: ,in, Indicates the overlap rate of images taken by adjacent cameras. is the angle between the boundary lines of the adjacent camera fields of view, is the camera’s field of view, is the radius of the hole, It is the minimum radius of the flat hole that can be captured by a multi-camera.

[0016] Furthermore, in S2, the relationship between the overlap rate of the images taken by the proposed adjacent cameras and the hole diameter is: ,in, Indicates the overlap rate of images taken by adjacent cameras. is the angle between the boundary lines of the adjacent camera fields of view, is the camera’s field of view, is the radius of the hole, is the minimum radius of the flat hole that can be captured by the multi-camera. Indicates the distance from the camera to the center of the flat hole.

[0017] Furthermore, in S4, the overlapping area is a rectangle, and the length of the side corresponding to the field of view angle of the overlapping area is equal to the product of the length of the side corresponding to the field of view angle of the image captured by the camera and the overlap rate.

[0018] The beneficial effects of the present invention are as follows: the present invention provides a method for stitching flat-hole images taken by multi-eye cameras, which comprises the following steps: according to the parameters of the cameras and the geometric relationship between adjacent cameras, the relationship between the overlap rate of the images taken by adjacent cameras and the hole diameter is formulated; according to the hole diameter size of the flat hole to be photographed, the overlap rate is determined; the multi-eye camera is placed at the center of the flat hole for shooting, and the overlapping area is determined in the two images taken by the adjacent cameras; features are extracted in the determined overlapping area to complete the image stitching; thereby, the feature extraction area is narrowed, the demand for computing power is reduced, and the problem of high computing power requirements caused by an excessively large feature search range during flat-hole image stitching is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic flow chart of a method for stitching flat hole images taken by multiple cameras provided by the present invention;

[0020] Figure 2 It is a schematic diagram of the position relationship of the flat hole image taken by the multi-eye camera provided by the present invention;

[0021] Figure 3 It is a schematic diagram of images taken by adjacent cameras provided by the present invention before and after being stitched together;

[0022] in, and For two adjacent cameras, the camera The intersection of a field of view boundary line and the minimum radius flat hole that the multi-eye camera can capture the whole picture is , and the intersection point with the flat hole to be photographed is ,Camera The intersection of another field of view boundary line and the minimum radius flat hole that the multi-eye camera can capture the whole picture is , and the intersection point with the flat hole to be photographed is ,Camera The intersection of a field of view boundary line and the minimum radius flat hole that the multi-eye camera can capture the whole picture is coincides, and the intersection with the flat hole to be photographed is , represents the center of the flat hole, For Camera The images taken, For Camera Shooting images The field of view angle corresponds to the length of the side, For images The overlapping area of For Camera The images taken, For Camera Shooting images The field of view angle corresponds to the length of the side, For images overlapping area. DETAILED DESCRIPTION

[0023] In view of the problem that the feature search range is too large when stitching flat-hole images, resulting in high computing power requirements, the present invention provides a method for stitching flat-hole images taken by multiple cameras, such as Figure 1 As shown, the following steps are included:

[0024] S1. Obtain camera parameters of a multi-camera and a geometric relationship between adjacent cameras; the camera parameters of the multi-camera include a field of view angle, and the field of view angle of each camera is equal; the geometric relationship between adjacent cameras includes an angle between boundary lines of the fields of view of adjacent cameras;

[0025] S2. According to the parameters of the cameras and the geometric relationship between the adjacent cameras, the relationship between the overlap rate of the images taken by the adjacent cameras and the hole diameter is formulated;

[0026] S3, determining the overlap rate according to the diameter of the flat hole to be photographed;

[0027] S4, placing the multi-camera at the center of the flat hole to shoot, and determining the overlapping area in the two images taken by adjacent cameras;

[0028] S5. Extract features from the determined overlapping areas to complete image stitching.

[0029] In this way, image stitching is completed by extracting features in the overlapping area determined in the two images taken by adjacent cameras, which narrows the scope of feature extraction, reduces the demand for computing power, and solves the problem of high computing power requirements caused by excessive feature search range when stitching flat hole images.

[0030] Furthermore, in S2, the relationship between the overlap rate of the images taken by the proposed adjacent cameras and the hole diameter is: ,in, Indicates the overlap rate of images taken by adjacent cameras. is the angle between the boundary lines of the adjacent camera fields of view, is the camera’s field of view, and are the lengths of the two boundary lines of the camera’s field of view, and , It is the distance from the intersection of the boundary lines of the adjacent camera fields of view to the camera.

[0031] Furthermore, in S2, the relationship between the overlap rate of the images taken by the proposed adjacent cameras and the hole diameter is: ,in, Indicates the overlap rate of images taken by adjacent cameras. is the angle between the boundary lines of the adjacent camera fields of view, is the camera’s field of view, is the radius of the hole, It is the minimum radius of the flat hole that can be captured by a multi-camera.

[0032] Furthermore, in S2, the relationship between the overlap rate of the images taken by the proposed adjacent cameras and the hole diameter is: ,in, Indicates the overlap rate of images taken by adjacent cameras. is the angle between the boundary lines of the adjacent camera fields of view, is the camera’s field of view, is the radius of the hole, is the minimum radius of the flat hole that can be captured by the multi-camera. Indicates the distance from the camera to the center of the flat hole.

[0033] Furthermore, in S4, the overlapping area is a rectangle, and the length of the side corresponding to the field of view angle of the overlapping area is equal to the product of the length of the side corresponding to the field of view angle of the image captured by the camera and the overlap rate.

[0034] Example:

[0035] The present invention is further described by using a six-eye camera to shoot the flat hole to be shot and perform image stitching. Five cameras of the six-eye camera are on the five sides of the hexagonal prism, which are used to shoot the circumferential image of the flat hole and stitch it. There is also a front camera for shooting the flat hole face, which does not participate in the stitching. When shooting, the six-eye camera is placed in the center of the flat hole for shooting. Figure 2 As shown in the figure, the angles between the five cameras shooting the flat hole and the lines connecting the center of the flat hole are , taking the connection line of the two cameras at the bottom as axis, passing through the top camera and perpendicular to The axis line is obtained Axis, establish rectangular coordinate system , is the center of the flat hole, and the two adjacent cameras are respectively denoted as cameras and camera ,Camera The field of view is , denoted as ,Camera The intersection of a field of view boundary line and the minimum radius flat hole that the multi-eye camera can capture the whole picture is , and the intersection point with the flat hole to be photographed is ,Camera The intersection of another field of view boundary line and the minimum radius flat hole that the multi-eye camera can capture the whole picture is , and the intersection point with the flat hole to be photographed is ,Camera The intersection of a field of view boundary line and the minimum radius flat hole that the multi-eye camera can capture the whole picture is coincides, and the intersection with the flat hole to be photographed is , represents the center of the flat hole, and For Camera The lengths of the two field of view boundaries are ,Camera With camera The angle between the boundary lines of the field of view is , denoted as , the minimum radius of the flat hole that can be captured by the six-eye camera is recorded as ,and is a fixed value, which is determined by the camera parameters of the six-eye camera and the geometric relationship between adjacent cameras; the radius of the flat hole to be photographed is recorded as ,when Greater than When the camera and camera The captured images have overlapping parts, and the arc corresponding to the overlapping parts is ,Camera The image captured corresponds to the arc of the flat hole The length value of , then the arc length calculation formula can be obtained: , the arc corresponding to the overlapping part The length value of , , then the relationship between the overlap rate of the images taken by the proposed adjacent cameras and the hole diameter is: ,in is the overlap ratio, and For Camera The lengths of the two field of view boundaries are , It is the distance from the intersection of the boundary lines of the adjacent camera fields of view to the camera.

[0036] Since the length of the field of view boundary line of camera A is difficult to measure, and the distance from the intersection of the field of view boundary lines of adjacent cameras to the camera is difficult to measure, the above formula is approximated:

[0037] The first approximate value transformation calculation: due to Click to Distance of points Much smaller than ,and Less than ,but Approximately equal to , Approximately equal to , Approximately equal to , so the relationship between the overlap rate of the images taken by adjacent cameras and the hole diameter after the approximate value transformation is: , is the first approximate calculation value of the overlap ratio;

[0038] The second approximate value transformation calculation: Click to Distance of points Much smaller than , then the triangle Approximate to a line segment, the triangle Approximately a line segment, then , , so the relationship between the overlap rate of the images taken by adjacent cameras and the hole diameter after the approximate value transformation is: , is the second approximate value of the overlap ratio;

[0039] Using the six-eye camera Images and cameras The captured images are stitched together as shown below: Figure 3 As shown, For Camera The images taken, For Camera Shooting images The field of view angle corresponds to the length of the side, For images The overlapping area of For Camera The images taken, For Camera Shooting images The field of view angle corresponds to the length of the side, For images The overlapping area of ​​the camera The images taken In terms of image Overlapping area The length of the side corresponding to the field of view angle is equal to the length of the side corresponding to the field of view angle of the image captured by the camera The product of the overlap ratio, for the camera The images taken In terms of image Overlapping area The length of the side corresponding to the field of view angle is equal to the length of the side corresponding to the field of view angle of the image captured by the camera The overlapping area of ​​the images taken by adjacent cameras is located at the edge of the image corresponding to the boundary line of the intersecting field of view. The images taken and camera The images taken Extract features from the image and complete image stitching.

Claims

1. A method for stitching flat hole images taken by multiple cameras, characterized in that: The following steps are involved: S1. Obtaining camera parameters of a multi-camera and a geometric relationship between adjacent cameras; the camera parameters of the multi-camera include a field of view angle, and the field of view angle of each camera is equal; The geometric relationship between adjacent cameras includes the angle between the boundary lines of the fields of view of adjacent cameras; S2. According to the parameters of the cameras and the geometric relationship between the adjacent cameras, the relationship between the overlap rate of the images taken by the adjacent cameras and the hole diameter is proposed. The relationship between the overlap rate of the images taken by the adjacent cameras and the hole diameter is: Wherein, K represents the overlap rate of images taken by adjacent cameras, β is the angle between the boundary lines of the field of view of adjacent cameras, α is the field of view angle of the camera, AF and AE are the lengths of the two boundary lines of the field of view of the camera, and AF = AE, AC is the distance from the intersection of the boundary lines of the field of view of adjacent cameras to the camera; S3, determining the overlap rate according to the diameter of the flat hole to be photographed; S4, placing the multi-camera at the center of the flat hole to shoot, and determining the overlapping area in the two images taken by adjacent cameras; S5. Extract features from the determined overlapping areas to complete image stitching.

2. The method for stitching flat hole images taken by multiple cameras according to claim 1, characterized in that: In S4, the overlapping area is a rectangle, and the length of the side corresponding to the field of view angle of the overlapping area is equal to the product of the length of the side corresponding to the field of view angle of the image captured by the camera and the overlap rate.

Images