Method for splicing images acquired by multiple cameras

Abstract

The invention discloses a multi-camera acquisition image splicing method, which comprises the following steps: arranging cameras, and determining the breadth information of a single camera; calibrating the camera through the calibration plate, and calculating camera parameters; calculating physical coordinates of the corresponding processing alignment pattern of each camera according to a preset camera interval; performing image processing on a machine processing plane based on the physical coordinates of the centers of the camera alignment patterns, determining the alignment pattern corresponding to each camera, and calculating an alignment change matrix; splicing transformation is carried out on images collected by the camera, splicing transformation images are determined, and a splicing transformation image set is established; carrying out image splicing on the splicing transformation images in the splicing transformation image set based on the line direction to obtain a line direction splicing image; and carrying out image splicing on the row direction spliced image based on the column direction, and determining a final spliced image. According to the processing breadth visual field information input by the user, splicing of the images collected by the multiple cameras is effectively completed, and the image splicing precision is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to a method for stitching images captured by multiple cameras. Background Art

[0002] In visual applications, for scenes with a large field of view, the field of view and resolution of a single camera are difficult to meet the requirements, and multiple cameras are usually required to complete image acquisition and stitching. At present, multi-camera image stitching technology has been widely used in industrial processing, virtual reality, autonomous driving and other fields, and has broad application prospects. However, the existing multi-camera image stitching methods still have some problems, such as unsatisfactory stitching effect, slow processing speed, and high requirements for image quality.

[0003] Therefore, the prior art has defects and is in urgent need of improvement. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to provide a method for stitching images captured by multiple cameras, which can effectively complete the stitching of images captured by multiple cameras according to the processing format field of view information input by the user, improve the image stitching effect, and is suitable for various types of cameras.

[0005] A first aspect of the present invention provides a method for stitching images captured by multiple cameras, comprising: Arrange cameras and determine format information of a single camera; the format information includes format width and format height; Calibrate the camera using a calibration plate to calculate camera parameters; the camera parameters include the camera's intrinsic parameters, extrinsic parameters, and correction coefficients; Calculate the physical coordinates of the processing alignment pattern corresponding to each camera according to a preset camera spacing; the preset camera spacing includes the camera arrangement spacing in the x direction and the camera arrangement spacing in the y direction; Performing image processing on a machine processing plane based on the physical coordinates of the center of the camera alignment pattern, determining the alignment pattern corresponding to each camera, and calculating an alignment change matrix based on the alignment pattern; Performing splicing transformation on the camera captured images according to the camera parameters and the alignment change matrix, determining a splicing transformation image, and establishing a splicing transformation image set; Performing image stitching on the stitching transformation images in the stitching transformation image set based on the row direction to obtain a row-direction stitching image; The row-direction stitched images are stitched based on the column direction to determine a final stitched image.

[0006] In this solution, determining the format information of a single camera includes: Obtain the machine processing width W and height H input by the user; According to the machine processing width W and height H, the number of cameras n in the x direction of the machine processing screen is analyzed. x , the number of cameras in the y direction n y , calculate the camera arrangement spacing d0 in the x direction and the camera arrangement spacing d1 in the y direction to determine the format width w and format height h of a single camera; ; .

[0007] In this solution, the camera is calibrated by a calibration board to calculate the camera parameters, including: Place the calibration plate on the machined plane at the exact center of the camera field of view, and construct a world coordinate three-dimensional point set P0 of the corner points of the calibration plate according to the physical size information of the calibration plate; Get the calibration plate image; Identify the calibration plate corner points of the calibration plate image and construct a two-dimensional pixel point set P1 of the image; Based on the preset camera model, the world coordinate three-dimensional point set P0 of the corner points of the calibration plate and the image two-dimensional pixel point set P1, the camera's intrinsic parameters, extrinsic parameters and correction coefficients are determined.

[0008] In this solution, the physical coordinates of the processing alignment pattern corresponding to each camera are calculated according to the preset camera spacing, including: The first r i Row, c i The camera corresponding to the column is determined as camera N rc ; Calculate the camera N rc The physical coordinates (x rc ,y rc ); ; ; Among them, W is the width of the machine processing, H is the height of the machine processing, and n is x is the number of cameras in the x direction of the machine processing screen, n y is the number of cameras in the y direction, d0 is the spacing between cameras in the x direction, d1 is the spacing between cameras in the y direction, r i and c i They are camera N rc The corresponding number of rows and columns.

[0009] In this solution, the step of calculating the alignment change matrix according to the alignment pattern includes: Get Camera N i A first distorted image I0 containing an alignment pattern is acquired; Performing distortion correction on the first distorted image I0 according to the intrinsic parameters, extrinsic parameters and correction coefficients of the camera to obtain a corrected image I1; Identifying pixel coordinates of a center of an alignment pattern from the rectified image I1; The pixel coordinates of the center of the alignment pattern and the corresponding physical coordinates are fitted to obtain an alignment transformation matrix M from pixel coordinates to physical coordinates.

[0010] In this solution, the steps of performing splicing transformation on the camera captured images according to the camera parameters and the alignment change matrix, determining the splicing transformation images, and establishing the splicing transformation image set include: Get Camera N i A second distorted image I2 is acquired; Performing distortion correction on the second distorted image I2 according to the intrinsic parameters, extrinsic parameters and correction coefficients of the camera to obtain a corrected image I3; Performing an alignment transformation on the corrected image I3 by using an alignment transformation matrix M to obtain an alignment transformation image I4; The aligned transformed image I4 is cropped according to the format information of a single camera to obtain a spliced ​​transformed image I5; The stitched transformed image I5 is added to the stitched transformed image set O.

[0011] In this solution, the image stitching of the stitching transformation images in the stitching transformation image set based on the row direction to obtain the row-direction stitching image includes: Obtain the rth i Row c i The stitched transformed image I corresponding to the camera column 5(ri-ci) ; The first r i The row-wise stitched image R i Initialized to r i The concatenated transformed image I of row 0 5(ri-0) ; Calculate the clipping boundary L r and clipping boundary L0; ; ; Where w is the width of a single camera, d0 is the distance between cameras in the x direction, and L r is the row-wise stitching image R i The cropping boundary on the right, L0 is the stitching transformation image I 5(ri-0) The left clipping boundary, w r is the row-wise stitching image R i The width of , e is the fusion constant; By the clipping boundary L r Stitch the image R in the row direction i Crop and stitch the row-wise image R according to the cropping result. i Make updates; By clipping the boundary L0 to the rth i Row c a The stitched transformed image I corresponding to the camera column 5(ri-ca) Perform cropping and determine the cropped image I 6(ri-ca) ; Among them, 0<c a ≤c i ; The spliced ​​transformed image I 5(ri-ca) is the row-wise stitching image R i Adjacent stitched images; The cropped row-wise stitched image R i and cropped image I 6(ri-ca) Perform image fusion and stitch the cropped row-wise image R according to the fused image i Make updates; The updated row-wise stitched image R i Continue image fusion with the next stitching transformation image.

[0012] In this solution, the clipping boundary L r Stitch the image R in the row direction i Cutting is performed by cutting the boundary L0 for the rth i Row c a The stitched transformed image I corresponding to the camera column 5(ri-ca) Perform cropping and determine the cropped image I 6(ri-ca) ,include: According to the clipping boundary L r Determine the first ROI box (0, 0, L r , h); Through the first ROI box (0, 0, L r , h) cropping the row-wise stitched image Ri; Determine the second ROI frame (L0, 0, w-L0, h) according to the cropping boundary L0; The stitching transformation image I is processed by the second ROI frame (L0, 0, w-L0, h) 5(ri-ca) to crop.

[0013] In this solution, the cropped row-wise stitched image R i and cropped image I 6(ri-ca) Perform image fusion, including: The cropped row-wise stitched image R i and cropped image I6(ri-ca) Perform analysis to determine overlapping visual field areas and non-overlapping visual field areas; Concatenate images R in row direction i The pixel gray value v0 and the cropped image I 6(ri-ca) The pixel gray value v1 calculates the pixel gray value v in the overlapping field of view; ; Among them, k is the pixel column where the overlapping area pixels are located, and w3 is the row-wise stitched image R i and cropped image I 6(ri-ca) The width of the overlapping area; The image of the overlapping field of view is updated according to the pixel gray value v.

[0014] The present invention discloses a method for stitching images collected by multiple cameras, the method comprising: arranging cameras, determining the format information of a single camera; calibrating the cameras through a calibration plate, calculating camera parameters; calculating the physical coordinates of a processing alignment pattern corresponding to each camera according to a preset camera spacing; performing image processing on a machine processing plane based on the physical coordinates of the center of the camera alignment pattern, determining the alignment pattern corresponding to each camera, and calculating an alignment change matrix; performing stitching transformation on the images collected by the cameras, determining a stitching transformation image, and establishing a stitching transformation image set; performing image stitching on the stitching transformation images in the stitching transformation image set based on the row direction to obtain a row direction stitching image; performing image stitching on the row direction stitching images based on the column direction to determine a final stitching image. The present invention effectively completes the stitching of images collected by multiple cameras according to the processing format field of view information input by the user, and improves the image stitching accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flowchart of a method for stitching images captured by multiple cameras provided by the present invention is shown; Figure 2 A flow chart of the camera parameter calculation method provided by the present invention is shown; Figure 3 A flow chart of the alignment change matrix calculation method provided by the present invention is shown. DETAILED DESCRIPTION

[0016] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0017] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0018] Figure 1 A flow chart of a method for stitching images captured by multiple cameras provided by the present invention is shown.

[0019] like Figure 1 As shown, the present invention discloses a method for stitching images collected by multiple cameras, comprising: S102, arranging cameras and determining format information of a single camera; the format information includes format width and format height; S104, calibrating the camera using a calibration plate to calculate camera parameters; the camera parameters include the camera's intrinsic parameters, extrinsic parameters, and correction coefficients; S106, calculating the physical coordinates of the processing alignment pattern corresponding to each camera according to a preset camera spacing; the preset camera spacing includes the camera arrangement spacing in the x direction and the camera arrangement spacing in the y direction; S108, performing image processing on a machine processing plane based on the physical coordinates of the center of the camera alignment pattern, determining the alignment pattern corresponding to each camera, and calculating an alignment change matrix according to the alignment pattern; S110, performing splicing transformation on the camera captured images according to the camera parameters and the alignment change matrix, determining a splicing transformation image, and establishing a splicing transformation image set; S112, stitching the stitching transformation images in the stitching transformation image set based on the row direction to obtain a row-direction stitching image; S114, stitching the row-direction stitched images based on the column direction to determine a final stitched image.

[0020] According to an embodiment of the present invention, in the process of arranging cameras, multiple cameras are arranged in rows and columns, and the spacing between different cameras is made roughly the same as much as possible, so that there is a certain overlap of fields of view between different adjacent cameras, and at the same time, the fields of view of all cameras are superimposed to include the machine processing format field of view preset by the user. The format information (including format width and format height) of each single camera is determined by combining the number of cameras in the x-direction and y-direction of the machine processing plane and the camera arrangement spacing. All cameras are calibrated using a calibration plate, and a world coordinate three-dimensional point set P0 of the corner points of the calibration plate is constructed according to the physical size information of the calibration plate, and an image two-dimensional pixel point P1 is constructed according to the identified corner points of the calibration plate image. The camera's intrinsic parameters, extrinsic parameters, and correction coefficients are calculated using P0 and P1. Calculate the center position of the processing alignment pattern corresponding to each camera (that is, the physical coordinates of the center of the camera alignment pattern), prevent processing materials such as white paper or wood from being placed on the machine processing plane, process the alignment pattern corresponding to each camera on the processing material according to the physical coordinates of the center of the camera alignment pattern, identify the pixel coordinates of the alignment pattern, fit them with the corresponding physical coordinates, and fit the alignment transformation matrix M from pixel coordinates to physical coordinates.

[0021] When the camera captures images, the camera captured images are spliced ​​and transformed according to the camera's internal parameters, external parameters, correction coefficients and alignment transformation matrix to establish a spliced ​​transformation image set O. First, the images are spliced ​​in the row direction. For each row, the spliced ​​transformation image of the 0th column is determined as the initial image of the row-direction spliced ​​image. The spliced ​​transformation images of adjacent columns are selected, and the two are cropped at the boundary before image fusion. The row-direction spliced ​​image is updated according to the fused image. The updated row-direction spliced ​​image is continued to be spliced ​​and fused with the spliced ​​transformation image of the next adjacent column until the final row-direction spliced ​​image is determined and added to the row-direction spliced ​​image set R. Finally, all r row-direction splicing result images are taken from the row-direction splicing image set R, and they are spliced ​​up and down into one image to obtain the final splicing result image.

[0022] According to an embodiment of the present invention, determining the format information of a single camera includes: Obtain the machine processing width W and height H input by the user; Analyze based on the machine processing width W and height H, combined with the number of cameras n in the x direction of the machine processing screen x , the number of cameras in the y direction n y , calculate the camera arrangement spacing d0 in the x direction and the camera arrangement spacing d1 in the y direction to determine the format width w and format height h of a single camera; ; .

[0023] It should be noted that the camera format information includes the camera format width w and format height h. The machine processing format width W and format height H input by the user are input into the camera format information calculation formula to determine the format width w and format height h of a single camera. Figure 2 A flow chart of the camera parameter calculation method provided by the present invention is shown.

[0024] like Figure 2 As shown, according to an embodiment of the present invention, the camera is calibrated by a calibration board to calculate the camera parameters, including: S202, placing the calibration plate on the machine processing plane at the center of the camera field of view, and constructing a world coordinate three-dimensional point set P0 of the corner points of the calibration plate according to the physical size information of the calibration plate; S204, obtaining a calibration plate image; S206, identifying the calibration plate corner points of the calibration plate image, and constructing a two-dimensional pixel point set P1 of the image; S208, analyzing based on the preset camera model, the world coordinate three-dimensional point set P0 of the corner points of the calibration plate, and the image two-dimensional pixel point set P1 to determine the camera's intrinsic parameters, extrinsic parameters, and correction coefficients.

[0025] It should be noted that the calibration plate can adopt a checkerboard structure, and the world coordinate system is defined at the first inner corner point in the upper left corner of the calibration plate. The Z axis is perpendicular to the plane of the calibration plate, and the coordinates of each inner corner point of the checkerboard are determined to determine the world coordinate three-dimensional point set P0. The two-dimensional pixel coordinates of the corner points of the calibration plate are extracted through the system's preset image processing algorithm, and the image two-dimensional pixel point set P1.

[0026] The preset camera model is preset by the system, which can be a pinhole imaging model, a mei model, etc. The camera's internal parameters, external parameters and correction coefficients can be calculated using Zhang Zhengyou calibration, Cai's two-step calibration and other solution methods.

[0027] According to an embodiment of the present invention, calculating the physical coordinates of the processing alignment pattern corresponding to each camera according to the preset camera spacing includes: The first r i Row, c i The camera corresponding to the column is determined as camera N rc ; Calculate camera N rc The physical coordinates (x rc ,y rc ); ; ; Among them, W is the width of the machine processing, H is the height of the machine processing, and n is xis the number of cameras in the x direction of the machine processing screen, n y is the number of cameras in the y direction, d0 is the spacing between cameras in the x direction, d1 is the spacing between cameras in the y direction, r i and c i They are camera N rc The corresponding number of rows and columns.

[0028] It should be noted that r can be determined based on the total number of cameras n i and c i The value range of r i =1,2…r;c i =1, 2…c; r is the total number of rows of cameras, c is the total number of columns of cameras, r×c=n.

[0029] Figure 3 A flow chart of the alignment change matrix calculation method provided by the present invention is shown.

[0030] like Figure 3 As shown, according to an embodiment of the present invention, calculating an alignment change matrix according to an alignment pattern includes: S302, obtaining camera N i A first distorted image I0 containing an alignment pattern is acquired; S304, performing distortion correction on the first distorted image I0 according to the intrinsic parameters, extrinsic parameters and correction coefficients of the camera to obtain a corrected image I1; S306, identifying the pixel coordinates of the center of the alignment pattern from the corrected image I1; S308, fitting the pixel coordinates of the center of the alignment pattern with the corresponding physical coordinates to obtain an alignment transformation matrix M from the pixel coordinates to the physical coordinates.

[0031] It should be noted that for camera N i The first distorted image I0 of the alignment pattern captured by the camera N is first distorted by using the intrinsic parameters, extrinsic parameters and correction coefficients of the camera that have been determined, and the pixel coordinates of the center of the alignment pattern in the corrected image I1 are fitted with the corresponding physical coordinates by a fitting method such as the least squares method to determine the alignment pattern of the camera N. i The alignment transformation matrix M identifies the pixel coordinates of the center of the alignment pattern.

[0032] According to an embodiment of the present invention, performing a splicing transformation on the camera captured images according to the camera parameters and the alignment change matrix, determining a splicing transformation image, and establishing a splicing transformation image set include: Get Camera N i A second distorted image I2 is acquired; Performing distortion correction on the second distorted image I2 according to the camera's internal parameters, external parameters, and correction coefficients to obtain a corrected image I3; Perform alignment transformation on the rectified image I3 through the alignment transformation matrix M to obtain an alignment transformation image I4; The aligned transformed image I4 is cropped according to the format information of a single camera to obtain a spliced ​​transformed image I5; The stitched transformed image I5 is added to the stitched transformed image set O.

[0033] It should be noted that first, the camera N i The captured second distorted image I2 is subjected to a stitching transformation. The stitching transformation includes internal distortion correction, alignment transformation, and cropping. The image is subjected to distortion correction through the camera's internal parameters, external parameters, and correction coefficients, and the image is subjected to an alignment transformation through the alignment transformation matrix M. Finally, the image is cropped according to the format information of a single camera to determine a stitching transformation image I5 obtained after the stitching transformation of the second distorted image I2. According to the above method, the images captured by each camera are stitched and transformed in turn, and the stitching transformation images obtained are added to the stitching transformation image set O.

[0034] According to an embodiment of the present invention, stitching is performed on the stitching transformation images in the stitching transformation image set based on the row direction to obtain a row-direction stitching image, including: Get the rth image from the spliced ​​transformed image set O i Row c i The stitched transformed image I corresponding to the camera column 5(ri-ci) ; The first r i The row-wise stitched image R i Initialized to r i The concatenated transformed image I of row 0 5(ri-0) ; Calculate the clipping boundary L r and clipping boundary L0; ; ; Where w is the width of a single camera, d0 is the distance between cameras in the x direction, and L r is the row-wise stitching image R i The cropping boundary on the right, L0 is the stitching transformation image I 5(ri-0) The left clipping boundary, w r is the row-wise stitching image R i The width of , e is the fusion constant; By clipping the boundary L r Stitch the image R in the row direction iCrop and stitch the image R in the row direction according to the cropping result i Make updates; By clipping the boundary L0 to the rth i Row c a The stitched transformed image I corresponding to the camera column 5(ri-ca) Perform cropping and determine the cropped image I 6(ri-ca) ; Among them, 0<c a ≤c i ; Splice transformation image I 5(ri-ca) is the row-wise stitching image R i Adjacent stitched images; The cropped row-wise stitched image R i and cropped image I 6(ri-ca) Perform image fusion and stitch the cropped row-wise image R according to the fused image i Make updates; The updated row-wise stitched image R i Continue image fusion with the next stitching transformation image.

[0035] It should be noted that the stitching transformation images corresponding to each row of cameras are stitched left and right according to the row direction (that is, a whole row of images are stitched into a row direction image), and a set of r row-direction stitching images is obtained. i The spliced ​​transformed image of row 0 is determined as the rth i The initial value of the row-wise stitched image is first compared with the rth i The stitching transformation image of the first row and the first column is bounded, and the cropped image is fused, and the fused image is used as the new row-wise stitching image, and the rth row and the first column are traversed. i The spliced ​​transformed images of the next row and column are fused until the rth row is traversed. i All rows and columns, get the rth i The final row-wise stitched image.

[0036] According to an embodiment of the present invention, by cutting the boundary L r Stitch the image R in the row direction i Cutting is performed by cutting the boundary L0 for the rth i Row c a The stitched transformed image I corresponding to the camera column 5(ri-ca) Perform cropping and determine the cropped image I 6(ri-ca) ,include: According to the clipping boundary L r Determine the first ROI box (0, 0, L r , h); Through the first ROI box (0, 0, L r, h) cropping the row-wise stitched image Ri; Determine the second ROI frame (L0, 0, w-L0, h) according to the cropping boundary L0; The second ROI frame (L0, 0, w-L0, h) is used to stitch the transformed image I 5(ri-ca) to crop.

[0037] It should be noted that the first ROI frame and the second ROI frame are both rectangular areas, and their ROI frame parameters are the horizontal coordinate, vertical coordinate, width and height of the upper left corner of the ROI frame from left to right. r , h) is an image with the coordinates of the upper left corner (0,0) and width L r , a rectangular area with height h.

[0038] According to an embodiment of the present invention, the cropped row-wise spliced ​​image R i and cropped image I 6(ri-ca) Perform image fusion, including: Splice the cropped image R in row direction i and cropped image I 6(ri-ca) Perform analysis to determine overlapping visual field areas and non-overlapping visual field areas; Concatenate images R in row direction i The pixel gray value v0 and the cropped image I 6(ri-ca) The pixel gray value v1 calculates the pixel gray value v in the overlapping field of view; ; Among them, k is the pixel column where the overlapping area pixels are located, and w3 is the row-wise stitched image R i and cropped image I 6(ri-ca) The width of the overlapping area; The image of the overlapping field of view is updated according to the pixel gray value v.

[0039] It should be noted that in the image fusion process, the overlapping field of view area where the row-direction stitched image and the cropped image overlap each other is first identified, and the pixel grayscale values ​​of the row-direction stitched image and the cropped image in the overlapping field of view area are input into the system's preset pixel grayscale value calculation formula to calculate the pixel grayscale values ​​of the overlapping field of view area, and the images in the overlapping field of view area are adjusted. The pixel grayscale values ​​of their respective images are retained in the non-overlapping areas, thereby completing image fusion.

[0040] The information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions. For example, "obtaining the machine processing width W and height H input by the user" and "obtaining the calibration plate image" involved in this disclosure are all obtained with full authorization.

[0041] The present invention discloses a method for stitching images captured by multiple cameras, the method comprising: arranging cameras, determining the format information of a single camera; calibrating the cameras through a calibration plate, and calculating camera parameters; calculating the physical coordinates of the processing alignment pattern corresponding to each camera according to a preset camera spacing; performing image processing on a machine processing plane based on the physical coordinates of the center of the camera alignment pattern, determining the alignment pattern corresponding to each camera, and calculating the alignment change matrix; performing stitching transformation on the images captured by the cameras, determining the stitching transformation image, and establishing a stitching transformation image set; performing image stitching on the stitching transformation images in the stitching transformation image set based on the row direction to obtain a row direction stitching image; performing image stitching on the row direction stitching images based on the column direction to determine the final stitching image. The present invention effectively completes the stitching of images captured by multiple cameras based on the processing format field of view information input by the user, thereby improving the accuracy of image stitching. In the several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0042] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0043] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0044] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0045] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

Claims

1. A method for stitching images collected by multiple cameras, characterized in that: include: Arrange cameras and determine format information of a single camera; the format information includes format width and format height; Calibrate the camera using a calibration plate to calculate camera parameters; the camera parameters include the camera's intrinsic parameters, extrinsic parameters, and correction coefficients; Calculate the physical coordinates of the processing alignment pattern corresponding to each camera according to a preset camera spacing; the preset camera spacing includes the camera arrangement spacing in the x direction and the camera arrangement spacing in the y direction; Performing image processing on a machine processing plane based on the physical coordinates of the center of the camera alignment pattern, determining the alignment pattern corresponding to each camera, and calculating an alignment change matrix based on the alignment pattern; Performing splicing transformation on the camera captured images according to the camera parameters and the alignment change matrix, determining a splicing transformation image, and establishing a splicing transformation image set; Performing image stitching on the stitching transformation images in the stitching transformation image set based on the row direction to obtain a row-direction stitching image; The row-direction stitched images are stitched based on the column direction to determine a final stitched image.

2. The multi-camera image stitching method according to claim 1, characterized in that: The determining of the format information of a single camera includes: Obtain the machine processing width W and height H input by the user; According to the machine processing width W and height H, the number of cameras n in the x direction of the machine processing screen is analyzed. x , the number of cameras in the y direction n y , calculate the camera arrangement spacing d0 in the x direction and the camera arrangement spacing d1 in the y direction to determine the format width w and format height h of a single camera; ； 。 3. The multi-camera image stitching method according to claim 1, characterized in that: The step of calibrating the camera by using the calibration board and calculating the camera parameters includes: Place the calibration plate on the machined plane at the exact center of the camera field of view, and construct a world coordinate three-dimensional point set P0 of the corner points of the calibration plate according to the physical size information of the calibration plate; Get the calibration plate image; Identify the calibration plate corner points of the calibration plate image and construct a two-dimensional pixel point set P1 of the image; Based on the preset camera model, the world coordinate three-dimensional point set P0 of the corner points of the calibration plate and the image two-dimensional pixel point set P1, the camera's intrinsic parameters, extrinsic parameters and correction coefficients are determined.

4. The method for stitching images collected by multiple cameras according to claim 1, characterized in that: The step of calculating the physical coordinates of the processing alignment pattern corresponding to each camera according to the preset camera spacing includes: The first r i Row, c i The camera corresponding to the column is determined as camera N rc ; Calculate the camera N rc The physical coordinates (x rc ,y rc ); ； ； Among them, W is the width of the machine processing, H is the height of the machine processing, and n is x is the number of cameras in the x direction of the machine processing screen, n y is the number of cameras in the y direction, d0 is the spacing between cameras in the x direction, d1 is the spacing between cameras in the y direction, r i and c i They are camera N rc The corresponding number of rows and columns.

5. The multi-camera image stitching method according to claim 1, characterized in that: The calculating the alignment change matrix according to the alignment pattern comprises: Get Camera N i A first distorted image I0 containing an alignment pattern is acquired; Performing distortion correction on the first distorted image I0 according to the intrinsic parameters, extrinsic parameters and correction coefficients of the camera to obtain a corrected image I1; Identifying pixel coordinates of a center of an alignment pattern from the rectified image I1; The pixel coordinates of the center of the alignment pattern and the corresponding physical coordinates are fitted to obtain an alignment transformation matrix M from pixel coordinates to physical coordinates.

6. The multi-camera image stitching method according to claim 1, characterized in that: The step of performing splicing transformation on the camera captured images according to the camera parameters and the alignment change matrix, determining a splicing transformation image, and establishing a splicing transformation image set includes: Get Camera N i A second distorted image I2 is acquired; Performing distortion correction on the second distorted image I2 according to the intrinsic parameters, extrinsic parameters and correction coefficients of the camera to obtain a corrected image I3; Performing an alignment transformation on the corrected image I3 by using an alignment transformation matrix M to obtain an alignment transformation image I4; The aligned transformed image I4 is cropped according to the format information of a single camera to obtain a spliced ​​transformed image I5; The stitched transformed image I5 is added to the stitched transformed image set O.

7. The multi-camera image stitching method according to claim 1, characterized in that: The step of performing image stitching on the stitching transformation images in the stitching transformation image set based on the row direction to obtain a row-direction stitching image includes: Obtain the rth i Row c i The stitched transformed image I corresponding to the camera column 5(ri-ci) ; The first i The row-wise stitched image R i Initialize to r i The concatenated transformed image I of row 0 5(ri-0) ; Calculate the clipping boundary L r and clipping boundary L0; ； ； Where w is the width of a single camera, d0 is the distance between cameras in the x direction, and L r is the row-wise stitching image R i The cropping boundary on the right, L0 is the stitching transformation image I 5(ri-0) The left clipping boundary, w r is the row-wise stitching image R i The width of , e is the fusion constant; By the clipping boundary L r Stitch the image R in the row direction i Crop and stitch the row-wise image R according to the cropping result. i Make updates; By clipping the boundary L0 to the rth i Row c a The stitched transformed image I corresponding to the camera column 5(ri-ca) Perform cropping and determine the cropped image I 6(ri-ca) ; Among them, 0<c a ≤c i ; The spliced ​​transformed image I 5(ri-ca) is the row-wise stitching image R i Adjacent stitched images; The cropped row-wise stitched image R i and cropped image I 6(ri-ca) Perform image fusion and stitch the cropped row-wise image R according to the fused image i Make updates; The updated row-wise stitched image R i Continue image fusion with the next stitching transformation image.

8. The method for stitching images collected by multiple cameras according to claim 7, characterized in that: The clipping boundary L r Stitch the image R in the row direction i Cutting is performed by cutting the boundary L0 for the rth i Row c a The stitched transformed image I corresponding to the camera column 5(ri-ca) Perform cropping and determine the cropped image I 6(ri-ca) ,include: According to the clipping boundary L r Determine the first ROI box (0, 0, L r , h); Through the first ROI box (0, 0, L r , h) cropping the row-wise stitched image Ri; Determine the second ROI frame (L0, 0, w-L0, h) according to the cropping boundary L0; The stitching transformation image I is processed by the second ROI frame (L0, 0, w-L0, h) 5(ri-ca) to crop.

9. The method for stitching images collected by multiple cameras according to claim 7, characterized in that: The cropped row-wise stitched image R i and cropped image I 6(ri-ca) Perform image fusion, including: The cropped row-wise stitched image R i and cropped image I 6(ri-ca) Perform analysis to determine overlapping visual field areas and non-overlapping visual field areas; Concatenate images R in row direction i The pixel gray value v0 and the cropped image I 6(ri-ca) The pixel gray value v1 calculates the pixel gray value v in the overlapping field of view; ； Among them, k is the pixel column where the overlapping area pixels are located, and w3 is the row-wise stitched image R i and cropped image I 6(ri-ca) The width of the overlapping area; The image of the overlapping field of view is updated according to the pixel gray value v.

Images