A method, device, medium and program product for simply and quickly implementing double-lens splicing

Abstract

The present application belongs to the technical field of monitoring camera image processing, and particularly relates to a method, device, medium and program product for simply and quickly realizing double-lens splicing. The method is designed for a double-lens camera with a certain inclination, all reference point positions are identified by shooting a correction plate, projection parameters are corrected through matching and fitting, seamless splicing is finally realized, and splicing parameters are obtained. In the generation process of the double-lens camera, splicing parameters are quickly calculated and solidified into the camera to facilitate users to obtain video splicing and playing effects.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology for surveillance cameras, specifically relating to a simple and quick method, device, medium, and program product for achieving dual-lens stitching. Background Technology

[0002] For surveillance cameras, a wider field of view would facilitate real-time monitoring. However, the field of view of wide-angle lenses is still limited, rarely exceeding 180°; while panoramic lenses such as fisheye lenses produce unpleasant distortion. To achieve a wide field of view, the common practice now is to stitch together images from multiple lenses (such as...). Figure 1 (As shown).

[0003] The typical process for creating image stitching now is as follows: Figure 2 However, for ordinary surveillance camera production lines, the production calibration environment is highly variable, and the lens installation of each device will produce some displacement and angular deviation. Therefore, when each device is measured individually, this method is not robust. Summary of the Invention

[0004] To quickly calculate and embed stitching parameters during the generation process of a dual-lens camera, enabling users to easily obtain the video stitching playback effect, this invention provides a simple and rapid method for achieving dual-lens stitching. This method uses a dual-lens camera designed with a certain tilt angle. By photographing a correction plate, the positions of all reference points are identified. Then, through matching and fitting, the projection parameters are corrected, ultimately achieving seamless stitching and obtaining the stitching parameters.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A simple and quick method for stitching together two lenses includes the following steps:

[0007] 1) Prepare a standard correction board, marking several lines and a sufficient number of reference points on it;

[0008] 2) The standard correction plate is placed at a certain distance to ensure that it appears completely in both lenses;

[0009] 3) Project a curved image onto the screen to obtain the picture. and ;

[0010] 4) The included angle of the optical axes of the two lenses, determined by the camera structure. Horizontal field of view of the lens ,estimate and Horizontal overlap width: ;

[0011] Image enhancement and gradient calculation are performed on the suspected overlapping areas and surrounding regions to obtain a contour map, followed by matching to achieve coarse alignment; Top left corner The position in the middle and overlapping areas The position in the middle

[0012] ;

[0013] and in The position in the middle

[0014] ;

[0015] 5) Based on the coarse alignment result, segment. and The overlapping region is obtained and ;in:

[0016] ;

[0017] ;

[0018] Adaptive image enhancement and red spot detection are performed to obtain... and Reference point positions on the correction plate;

[0019] 6) According to Reference point position as well as Reference point position To achieve secondary correction, two transformation matrices are used. and ,make and All converted to

[0020] in

[0021] and It is aimed at and The multiple fitting matrix;

[0022] 7) Introduction and Reproject the original image onto a curved surface to obtain a new one. and ;

[0023] 8) To and Image enhancement and gradient calculation are performed to obtain a contour map, and then... Secondary alignment is achieved by matching within a certain range nearby, resulting in... ;

[0024] 9) Cut off the black borders to get the final result image.

[0025] Preferably, the standard correction plate consists of several horizontal, vertical, or diagonal black lines, with red reference points set at the junctions of the horizontal, vertical, or diagonal black lines and / or the connections between the horizontal, vertical, and diagonal black lines.

[0026] Preferably, in step 3), the pixel horizontal coordinates and polar coordinate horizontal angles are linearly related, and the pixel vertical coordinates and polar coordinate pitch angles are linearly related; the image width and height are... Its pixel coordinates

[0027] or

[0028] Wide screen Horizontal and vertical field of view of the lens The relationship is linear, with the center of the image corresponding to a horizontal angle of 0°.

[0029]

[0030]

[0031] Based on the transformation relationship between single-shot images and spatial vectors, the position of the point in the original image is obtained. Finally, interpolation is used to achieve the initial curved projection of the original image, resulting in the image. and .

[0032] Preferably, the two transformation matrices in step 6) and The construction method is as follows:

[0033] Constructing vectors in

[0034]

[0035]

[0036] Constructing vectors in

[0037]

[0038]

[0039] exist

[0040] Right now

[0041] According to the least squares method: ,get

[0042] Fitted That is the left lens curved projection correction matrix—in drawing At a certain pixel, according to Its value was calculated by fitting the matrix. The position in the middle ;

[0043] Similarly, the left lens curved surface projection correction matrix can be obtained. .

[0044] Furthermore, this invention also discloses a method for real-time stitching in a media player. After stitching and correction are completed using the methods described in claims 1-4, a fitting matrix is ​​obtained. and and alignment position Configure all settings in the camera firmware; and carefully follow these steps:

[0045] 1) The camera writes the parameters, along with the fitting parameters required for the curved surface projection, into the video stream;

[0046] 2) The player extracts all parameters when parsing the video stream;

[0047] 3) The player synchronizes the video feed after decoding it;

[0048] 4) The player projects a pair of synchronized images onto a curved surface to obtain... and And then according to The process involves progressive stitching with bidirectional exposure compensation to eliminate seams and remove black borders.

[0049] Furthermore, the present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method.

[0050] Furthermore, the present invention also discloses a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the method.

[0051] Furthermore, the present invention also discloses a computer program product, including a computer program or instructions that, when executed by a processor, implement the method.

[0052] This invention, employing the aforementioned technical solution, utilizes a dual-lens camera designed at a specific tilt angle. By photographing a correction plate, it identifies the positions of all reference points, then corrects the projection parameters through matching and fitting, ultimately achieving seamless stitching and obtaining the stitching parameters. This invention rapidly calculates the stitching parameters during the dual-lens camera generation process and embeds them into the camera for convenient video stitching playback. Attached Figure Description

[0053] Figure 1 This is a rendering of the image stitching effect of the present invention.

[0054] Figure 2 Create a flowchart for existing technologies.

[0055] Figure 3 This is a schematic diagram of a standard correction plate for reference.

[0056] Figure 4 This is a schematic diagram illustrating the effect of dual-lens imaging on the correction plate.

[0057] Figure 5 This is a schematic diagram of the curved surface projection.

[0058] Figure 6 Initial curved projection of the original image and Schematic diagram.

[0059] Figure 7 This is a rough alignment result.

[0060] Figure 8 Enhanced image of overlapping area and reference point segmentation image.

[0061] Figure 9 Projecting the original image onto a secondary curved surface and Schematic diagram.

[0062] Figure 10 This is for stitching together the final result image. Detailed Implementation

[0063] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0064] a) Prepare a standard correction board, marked with several lines and sufficient reference points, such as... Figure 3 .

[0065] b) The standard version is placed at a certain distance to ensure it appears fully in both shots, such as... Figure 4 As shown.

[0066] c) Project a curved image onto the screen.

[0067] Standard curved projection is also a commonly used projection method in VR panoramas. The horizontal coordinates of a pixel have a linear correspondence with the horizontal angle in polar coordinates, and the vertical coordinates of a pixel have a linear correspondence with the pitch angle in polar coordinates. For example... Figure 5 As shown, the image width and height are Its pixel coordinates

[0068] or

[0069] Wide screen Horizontal and vertical field of view of the lens The relationship is linear, with the center of the image corresponding to a horizontal angle of 0°.

[0070]

[0071]

[0072] Based on the transformation relationship between single-shot images and spatial vectors, the position of the point in the original image is obtained. Finally, interpolation is used to achieve the initial curved projection of the original image, resulting in the image. and ,like Figure 6 As shown.

[0073] d) The included angle of the optical axes of the two lenses, determined by the camera structure. Horizontal field of view of the lens Rough estimate and Horizontal overlap width:

[0074] Image enhancement and gradient calculation are performed on the suspected overlapping areas and their vicinity to obtain a contour map, which is then matched to achieve coarse alignment. Top left corner The position in the middle and overlapping areas The position in the middle

[0075]

[0076] and in The position in the middle

[0077]

[0078] The result of coarse alignment is shown in the image. Figure 7 As shown.

[0079] e) Based on the coarse alignment result, segment... and The overlapping region is obtained and .in:

[0080]

[0081]

[0082] Adaptive image enhancement and red spot detection are performed to obtain... and The positions of the 21 red reference points on the correction plate. The enhanced effect diagram of the overlapping area and the reference point segmentation diagram are shown below. Figure 8 As shown.

[0083] f) According to Reference point position as well as Reference point position To achieve secondary correction.

[0084] To compensate for the installation errors of the two lenses, it is necessary to obtain two transformation matrices, one for the left and one for the right. and , can make and All converted to

[0085] in

[0086] and It is aimed at and The multiple fitting matrices are used, therefore the actual parameters of the reference points should be converted back to their original values. and At scale.

[0087] Constructing vectors in

[0088]

[0089]

[0090] Constructing vectors in

[0091]

[0092]

[0093] exist

[0094] Right now

[0095] According to the least squares method: ,get

[0096] Fitted That is the left lens curved projection correction matrix—in drawing At a certain pixel, according to Its value was calculated by fitting the matrix. The position in the middle .

[0097] Similarly, the left lens curved surface projection correction matrix can be obtained. .

[0098] g) Introduction and Reproject the original image onto a curved surface to obtain a new one. and Original image quadratic arc projection and like Figure 9 As shown.

[0099] h) To and Image enhancement and gradient calculation are performed to obtain a contour map, and then... Secondary alignment is achieved by matching within a certain range nearby, resulting in... .

[0100] i) Remove the black borders to obtain the final result image, and stitch the final result images together as shown. Figure 10 As shown.

[0101] The following are examples and extensions of the present invention.

[0102] The above method enables real-time stitching in the player. After stitching correction is completed, the fitting matrix is ​​obtained. and and alignment position All settings are then incorporated into the camera firmware.

[0103] 1) The camera writes this parameter, along with the fitting parameters required for the curved surface projection, into the video stream;

[0104] 2) The player extracts all parameters when parsing the video stream;

[0105] 3) The player synchronizes the video feed after decoding it;

[0106] 4) The player projects a pair of synchronized images onto a curved surface to obtain... and And then according to A progressive stitching process is performed directly, with bidirectional exposure compensation applied to eliminate stitching seams and remove black borders. The final result image is then displayed.

[0107] The foregoing description of embodiments of the present invention, through which those skilled in the art are able to implement or use the present invention, will be readily apparent to those skilled in the art. Various modifications to these embodiments will be readily apparent to those skilled in the art. The general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novelty disclosed herein.

Claims

1. A simple and quick method for stitching together two lenses, characterized in that, The method includes the following steps: 1) Prepare a standard correction board, marking several lines and a sufficient number of reference points on it; 2) The standard correction plate is placed at a certain distance to ensure that it appears completely in both lenses; 3) Project a curved image onto the screen to obtain the picture. and ; 4) The included angle of the optical axes of the two lenses, determined by the camera structure. Horizontal field of view of the lens ,estimate and Horizontal overlap width: ; Image enhancement and gradient calculation are performed on the suspected overlapping areas and surrounding regions to obtain a contour map, followed by matching to achieve coarse alignment; Top left corner The position in the middle and overlapping areas The position in the middle ； and in The position in the middle ； 5) Based on the coarse alignment result, segment. and The overlapping region is obtained and ;in: ； ； Adaptive image enhancement and red spot detection are performed to obtain... and Reference point positions on the correction plate; 6) According to Reference point position as well as Reference point position To achieve secondary correction, two transformation matrices are used. and ,make and All converted to ; in ; and It is aimed at and The multiple fitting matrix; 7) Introduction and Reproject the original image onto a curved surface to obtain a new one. and ; 8) To and Image enhancement and gradient calculation are performed to obtain a contour map, and then... Secondary alignment is achieved by matching within a certain range nearby, resulting in... ; 9) Cut off the black borders to get the final result image.

2. The method for simple and rapid dual-lens stitching according to claim 1, characterized in that, The standard correction board consists of several horizontal, vertical, or diagonal black lines, with red reference points placed at the points where the horizontal, vertical, or diagonal black lines meet and / or where they connect.

3. The method for simple and rapid dual-lens stitching according to claim 1, characterized in that, In step 3), the pixel's horizontal coordinates are linearly related to the polar coordinates of the horizontal angle, and the pixel's vertical coordinates are linearly related to the polar coordinates of the pitch angle; the image width and height are... ; Its pixel coordinates or ； Wide screen Horizontal and vertical field of view of the lens The relationship is linear, with the center of the image corresponding to a horizontal angle of 0°. ； ； Based on the transformation relationship between single-shot images and spatial vectors, the position of the point in the original image is obtained. Finally, interpolation is used to achieve the initial curved projection of the original image, resulting in the image. and .

4. The method for simple and rapid dual-lens stitching according to claim 1, characterized in that, The two transformation matrices in step 6) and The construction method is as follows: Constructing vectors in ； ； Constructing vectors in ； ； exist Right now ; According to the least squares method: ,get ; Fitted That is the left lens curved projection correction matrix—in drawing At a certain pixel, according to Its value was calculated by fitting the matrix. The position in the middle ; Similarly, the left lens curved surface projection correction matrix can be obtained. .

5. A method for real-time splicing of media players, characterized in that, After the method described in claims 1-4 completes the splicing and correction, a fitting matrix is ​​obtained. and and alignment position Configure all settings in the camera firmware; and carefully follow these steps: 1) The camera writes the parameters, along with the fitting parameters required for the curved surface projection, into the video stream; 2) The player extracts all parameters when parsing the video stream; 3) The player synchronizes the video feed after decoding it; 4) The player projects a pair of synchronized images onto a curved surface to obtain... and And then according to The process involves progressive stitching with bidirectional exposure compensation to eliminate seams and remove black borders.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1-5.

7. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the method described in any one of claims 1-5.

8. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the method described in any one of claims 1-5.

Images