Multi-camera image splicing method and system

Abstract

The invention belongs to the technical field of image stitching, and discloses a multi-camera image stitching method and system. The method comprises the following steps: collecting data of a plurality of real-time images shot by a multi-camera device, and carrying out image preprocessing; collecting real-time image edge information of each piece of preprocessed real-time image data, and extracting a plurality of real-time feature points; performing feature point matching on the plurality of real-time feature points by using a pre-trained feature point matching model; performing image registration on all the preprocessed real-time image data according to a plurality of feature point matching results; carrying out image fusion on the real-time overlapping region of the real-time image data after all the images are registered by using an improved multi-band fusion algorithm; and according to the real-time splicing parameters, carrying out image splicing on the real-time image data after all the images are fused to obtain real-time holographic image data. The problems of poor visual effect, low accuracy, low processing speed, poor splicing quality and poor fusion effect in the prior art are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image stitching, and in particular relates to a multi-camera image stitching method and system. Background Art

[0002] Multi-camera image stitching is a task that combines images from multiple different perspectives into a panoramic image. With the development of computer science and digital signal processing technology, digital image processing technology has gradually matured, enabling images acquired from multiple perspectives to be effectively processed and fused through algorithms, providing technical support for multi-camera image stitching.

[0003] The existing multi-camera image stitching technology has the following defects:

[0004] 1) Existing multi-camera image stitching technologies often have difficulty in accurately aligning the edges of images captured by multiple cameras, resulting in obvious seams or misalignment in the stitched images, affecting the visual effect and the accuracy of image expression;

[0005] 2) The existing multi-camera image stitching methods have high computational complexity in the steps of feature point extraction, matching and image registration, resulting in slow processing speed and difficulty in meeting the requirements of real-time stitching, especially in applications with high-resolution images and multi-camera systems;

[0006] 3) Existing multi-camera image stitching methods are easily disturbed under complex and changing environmental conditions, such as lighting changes, dynamic scenes, occlusions, etc., resulting in reduced stitching quality;

[0007] 4) In the process of image fusion, the existing multi-camera image stitching methods often cannot process the pixel values ​​of the overlapping areas well, resulting in halo, blur or unnatural transition effects in the fused image, and poor fusion effect. Summary of the invention

[0008] In order to solve the problems of poor visual effect, low accuracy, slow processing speed, poor stitching quality and poor fusion effect in the prior art, the present invention aims to provide a multi-camera image stitching method and system.

[0009] The technical solution adopted by the present invention is:

[0010] A multi-camera image stitching method comprises the following steps:

[0011] Collecting a number of real-time image data captured by multiple camera devices, and performing image preprocessing on the number of real-time image data to obtain a number of preprocessed real-time image data;

[0012] Collecting real-time image edge information of each pre-processed real-time image data, and extracting a number of real-time feature points of each pre-processed real-time image data according to the real-time image edge information;

[0013] Using a pre-trained feature point matching model, feature point matching is performed on a number of real-time feature points of all pre-processed real-time image data to obtain a number of feature point matching results;

[0014] According to the matching results of several feature points, image registration is performed on all pre-processed real-time image data to obtain several real-time image data after image registration;

[0015] Using an improved multi-band fusion algorithm, image fusion is performed on the real-time overlapping areas of all real-time image data after image registration to obtain real-time image data after several images are fused;

[0016] According to the real-time stitching parameters, all images are fused and the real-time image data is stitched to obtain real-time holographic image data.

[0017] Furthermore, the image preprocessing includes Gaussian denoising processing, image enhancement processing, viewing angle adjustment processing, image grayscale processing and size normalization processing performed in sequence.

[0018] Furthermore, collecting the real-time image edge information of each pre-processed real-time image data, and extracting a plurality of real-time feature points of each pre-processed real-time image data according to the real-time image edge information, comprises the following steps:

[0019] Use edge detection algorithm to perform edge detection on pre-processed real-time image data to obtain corresponding real-time image edge information;

[0020] According to the edge information of the real-time image, a feature point extraction algorithm is used to extract several real-time feature points of the pre-processed real-time image data;

[0021] All pre-processed real-time image data are traversed to obtain a number of real-time feature points of each pre-processed real-time image data.

[0022] Furthermore, the feature point matching model is constructed based on the RF-BiGCN algorithm, and the feature point matching model includes a key feature point screening module constructed based on the RF algorithm and a feature point matching module constructed based on the BiGCN algorithm.

[0023] Furthermore, using a pre-trained feature point matching model, feature point matching is performed on a plurality of real-time feature points of all pre-processed real-time image data to obtain a plurality of feature point matching results, including the following steps:

[0024] Converting a number of real-time feature points of the pre-processed real-time image data into real-time feature point vectors, and inputting the real-time feature point vectors into a pre-trained feature point matching model;

[0025] Use the key feature point screening module to extract several key real-time feature points in the real-time feature point vector;

[0026] According to several key real-time feature points, a feature point matching module is used to perform feature point matching on all pre-processed real-time image data to obtain several initial feature point matching results;

[0027] The matching feature point screening algorithm is used to screen the initial several feature point matching results to obtain the final several feature point matching results.

[0028] Further, according to the matching results of the plurality of feature points, image registration is performed on all pre-processed real-time image data to obtain a plurality of real-time image data after image registration, including the following steps:

[0029] According to the geometric relationship between the real-time image data after preprocessing, a corresponding real-time geometric transformation model is selected;

[0030] Generating real-time model parameters of the real-time geometric transformation model according to the matching results of the plurality of feature points, and updating the real-time geometric transformation model according to the real-time model parameters to obtain an updated real-time geometric transformation model;

[0031] According to the updated real-time geometric transformation model, image registration is performed on all pre-processed real-time image data to obtain a number of real-time image data after image registration.

[0032] Furthermore, an improved multi-band fusion algorithm is used to perform image fusion on the real-time overlapping areas of all the real-time image data after image registration to obtain a number of real-time image data after image fusion, including the following steps:

[0033] Extracting the real-time overlapping area of ​​each image registered real-time image data and other image registered real-time image data;

[0034] Perform frequency band division on the real-time overlapping area to obtain corresponding low-frequency bands and high-frequency bands, and generate corresponding real-time initial weights according to the position of each pixel point in the real-time overlapping area;

[0035] According to the distance from each pixel point in the real-time overlapping area to the central pixel point, the corresponding real-time adaptive weight is generated using the convergence factor;

[0036] According to the real-time initial weight and the real-time adaptive weight, the low-frequency band and the high-frequency band of each pixel in the real-time overlapping area are weighted average fused to obtain the real-time overlapping area after multi-band fusion;

[0037] The real-time overlapping areas of all the real-time image data after image registration are traversed to obtain a number of real-time image data after image fusion including the real-time overlapping areas after multi-band fusion.

[0038] Further, according to the real-time stitching parameters, the real-time image data after all images are fused is stitched to obtain real-time holographic image data, which includes the following steps:

[0039] Use a swarm intelligence optimization algorithm to generate real-time splicing parameters for each image fused with real-time image data after fusion with other images;

[0040] According to the real-time stitching parameters, the real-time image data after all images are fused is stitched to obtain the initial real-time holographic image data;

[0041] The initial real-time holographic image data is post-processed to obtain the final real-time holographic image data.

[0042] Furthermore, the swarm intelligence optimization algorithm is an IWOA optimization algorithm.

[0043] A multi-camera image stitching system is used to implement a multi-camera image stitching method. The system comprises an image preprocessing unit, a feature point extraction unit, a feature point matching unit, an image registration unit, an image fusion unit and an image stitching unit which are connected in sequence.

[0044] The beneficial effects of the present invention are:

[0045] The present invention provides a multi-camera image stitching method and system. Image preprocessing ensures that images captured by different cameras have consistent quality and format before stitching, thereby improving the efficiency and accuracy of subsequent processing. The feature point matching model constructed based on the deep learning algorithm can more accurately extract and match feature points in real-time image data, and can maintain high matching accuracy even in complex scenes. It also reduces the computational complexity and improves the processing speed, meeting the needs of real-time stitching, especially in applications in high-resolution images and multi-camera systems. By analyzing the feature point matching results, an effective image registration process is proposed, which can accurately align multiple images. It reduces errors and seam problems in the stitching process; it adopts an improved multi-band fusion algorithm, which can better process the pixel values ​​in the overlapping areas of images, so that the transition of the fused image is natural, avoids halo and blur, and improves the overall visual quality of the image; it introduces the IWOA optimization algorithm to generate image stitching parameters, which can quickly find the optimal stitching parameters and improve the speed and accuracy of image stitching; the post-processing steps of the image further optimize the image quality and ensure that the holographic image data achieves the best visual effect; it has a high degree of automation, reduces manual intervention, and is suitable for a variety of scenarios and different types of image stitching, and has wide versatility.

[0046] Other beneficial effects of the present invention will be further described in the specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a flowchart of the multi-camera image stitching method in the present invention.

[0048] Figure 2 It is a structural block diagram of the multi-camera image stitching system in the present invention. DETAILED DESCRIPTION

[0049] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments.

[0050] Embodiment 1:

[0051] like Figure 1 As shown, this embodiment provides a multi-camera image stitching method, comprising the following steps:

[0052] S1: collecting a plurality of real-time image data captured by a multi-camera device, and performing image preprocessing on the plurality of real-time image data to obtain a plurality of preprocessed real-time image data, including the following steps:

[0053] S1-1: collecting a number of real-time image data captured by multiple camera devices, and performing Gaussian denoising on the number of real-time image data to obtain a number of real-time image data after denoising;

[0054] Gaussian denoising processing eliminates noise interference in real-time image data, improves the image quality of real-time image data, and provides support for subsequent feature point extraction and matching;

[0055] S1-2: performing image enhancement processing on a number of real-time image data after denoising processing to obtain a number of real-time image data after enhancement processing;

[0056] Image enhancement processing enhances the real-time image data after denoising to show the shape, structure and texture of the object, providing support for the subsequent extraction of image edge information;

[0057] S1-3: performing perspective adjustment processing on a plurality of enhanced real-time image data to obtain a plurality of perspective adjusted real-time image data;

[0058] During the preprocessing of images captured by multiple cameras, it is usually necessary to adjust the image perspective according to the camera angle, especially when the perspective difference between the cameras is large. This adjustment helps to reduce the image distortion caused by the perspective difference, thereby improving the quality of subsequent image stitching or fusion;

[0059] S1-4: performing image grayscale processing on a plurality of real-time image data after viewing angle adjustment processing to obtain a plurality of real-time image data after grayscale processing;

[0060] Image grayscale processing reduces the computational complexity of image processing, eliminates color deviation, thereby increasing processing speed, enhancing image features, and providing support for subsequent image registration;

[0061] S1-5: performing size normalization processing on a number of real-time image data after grayscale processing to obtain a number of real-time image data after preprocessing;

[0062] Size normalization simplifies the stitching process, improves feature point matching accuracy, reduces calculations, avoids image deformation, and provides support for subsequent image fusion;

[0063] S2: collecting the real-time image edge information of each pre-processed real-time image data, and extracting a number of real-time feature points of each pre-processed real-time image data according to the real-time image edge information, including the following steps:

[0064] S2-1: Use an edge detection algorithm to perform edge detection on the preprocessed real-time image data to obtain corresponding real-time image edge information;

[0065] Edge detection algorithms include Sobel operator, Canny edge detector, Prewitt operator, Laplacian operator, etc., which detect edges in images. Image edge information reflects key features such as shape, structure and texture of objects in the image. In this embodiment, a Canny edge detector is used to perform edge detection on pre-processed real-time image data, calculate the gradient strength and direction of each pixel in the image to determine the edge, and refine the detected edge to obtain an edge with a single pixel width;

[0066] S2-2: extracting several real-time feature points of the pre-processed real-time image data using a feature point extraction algorithm according to the edge information of the real-time image;

[0067] Feature point extraction algorithms include Scale-Invariant Feature Transform (SIFT) algorithm, Speeded Up Robust Features (SURF) algorithm, Oriented FAST and Rotated BRIEF (ORB) algorithm, etc., which not only use edge information but also consider the texture and scale invariance of the image to extract feature points. In this embodiment, the SIFT algorithm is used to extract several real-time feature points of the pre-processed real-time image data;

[0068] S2-3: traverse all pre-processed real-time image data to obtain a number of real-time feature points of each pre-processed real-time image data;

[0069] S3: Using a pre-trained feature point matching model, performing feature point matching on a number of real-time feature points of all pre-processed real-time image data to obtain a number of feature point matching results, including the following steps:

[0070] S3-1: converting a number of real-time feature points of the pre-processed real-time image data into real-time feature point vectors, and inputting the real-time feature point vectors into a pre-trained feature point matching model;

[0071] The feature point matching model is constructed based on the Random Forest (RF)-Bidirectional Graph Convolutional Network (BiGCN) algorithm, and the feature point matching model includes a key feature point screening module constructed based on the RF algorithm and a feature point matching module constructed based on the BiGCN algorithm;

[0072] The key feature point screening module screens the input real-time feature point vector through the internal Classification And Regression Tree (CART), which can process a large number of features and select the most stable and distinguishing feature points. The feature point matching module considers the neighbor information of the node and the neighbor information of the neighbor when processing the graph structure data, so as to better capture the structure and relationship in the graph, and further processes the feature points screened by the random forest, using its time series processing capability to consider the spatial relationship of the feature points in the image, so as to more accurately match the feature points. Compared with the existing feature point matching algorithm, the feature point matching model based on the deep learning algorithm can more accurately identify and match feature points, and maintain high stitching quality even in complex environments, thereby significantly improving the robustness and accuracy of multi-camera image stitching, especially when dealing with stitching tasks in complex environments;

[0073] S3-2: Use the key feature point screening module to extract several key real-time feature points in the real-time feature point vector, including the following steps:

[0074] S3-2-1: Use the RF structure trained in the key feature point screening module to extract the feature contribution of M candidate real-time feature points in the real-time feature point vector, where M is the number of key real-time feature points;

[0075] The formula is:

[0076]

[0077] In the formula, is the feature contribution of the j'th candidate real-time feature point; is the feature contribution of the j'th candidate real-time feature point in the i'th tree of the random forest; i' is the CART tree indicator; j' is the candidate real-time feature point indicator; n' is the total number of CARTs;

[0078]

[0079] In the formula, GI m , GI l' , GI r" is the Gini index of the CART tree node m, node l' and node r" of the random forest; p mk" is the proportion of category k" in CART tree node m; K' is the total number of categories; m, l', r" are node indicators; k" is the category indicator;

[0080] S3-2-2: normalizing the feature contributions of the M candidate real-time feature points to obtain a number of corresponding normalized feature contributions;

[0081] The formula is:

[0082]

[0083] Where VIM j' is the contribution of the feature after normalization; J is the total number of candidate real-time feature points;

[0084] S3-2-3: Generate feature selection standard values ​​for several candidate real-time feature points based on the normalized feature contribution;

[0085] The formula is:

[0086]

[0087] In the formula, CFC j' VIM is the feature selection standard value of the j'th candidate real-time feature point; j* is the normalized feature contribution of the j*th candidate real-time feature point; j* is the indicator of the candidate real-time feature point;

[0088] S4-1-5: According to the feature selection standard value, the candidate real-time feature points are sorted in descending order, and the first M candidate real-time feature points are selected as key real-time feature points to obtain M key real-time feature points;

[0089] S3-3: According to a number of key real-time feature points, a feature point matching module is used to perform feature point matching on all pre-processed real-time image data to obtain a number of initial feature point matching results;

[0090] S3-4: Use the Random Sample Consensus (RANSAC) algorithm to screen the initial matching results of several feature points, remove the mismatched feature points, retain the matching feature points with higher quality, and obtain the final matching results of several feature points;

[0091] S4: performing image registration on all pre-processed real-time image data according to the matching results of the plurality of feature points to obtain a plurality of real-time image data after image registration, including the following steps:

[0092] S4-1: According to the geometric relationship between the real-time image data after preprocessing, select the corresponding real-time geometric transformation model, such as affine transformation model, projection transformation model, homography transformation model, etc.;

[0093] In this embodiment, the real-time geometric transformation model is an affine transformation model, which is used to describe the changing relationship of pixel positions when rotating, translating, scaling and tilting the multi-camera image data when stitching;

[0094] The formula is:

[0095]

[0096] Where (x, y) is the original coordinate of the pixel; (x', y') is the transformed coordinate of the pixel; θ is the rotation angle; (P x ,P y ) is the translation vector;

[0097] S4-2: generating real-time model parameters of the real-time geometric transformation model according to the matching results of the plurality of feature points, including the real-time transformed coordinates and the translation vector of each pixel point, and updating the real-time geometric transformation model according to the real-time model parameters to obtain an updated real-time geometric transformation model;

[0098] S4-3: performing image registration on all pre-processed real-time image data according to the updated real-time geometric transformation model to obtain a number of real-time image data after image registration;

[0099] S5: Using the improved multi-band fusion algorithm, image fusion is performed on the real-time overlapping areas of all the real-time image data after image registration to obtain real-time image data after a number of image fusions, including the following steps:

[0100] S5-1: extracting the real-time overlapping area of ​​each image registered real-time image data and other image registered real-time image data;

[0101] S5-2: Divide the real-time overlapping area into frequency bands to obtain corresponding low-frequency bands and high-frequency bands, and generate corresponding real-time initial weights according to the position of each pixel point in the real-time overlapping area;

[0102] The low frequency band contains basic structural information of the image in the real-time overlapping area, while the high frequency band contains details and edge information of the image in the real-time overlapping area;

[0103] The initial weight is generated by the feature similarity or contrast of the pixels, which is used to measure the degree of the content expressed by the pixels in the real-time overlapping area;

[0104] S5-3: Generate corresponding real-time adaptive weights based on the distance from each pixel in the real-time overlapping area to the central pixel using the convergence factor;

[0105] The formula is:

[0106]

[0107] Where a is the convergence factor; tanh(.) is the hyperbolic tangent function; d, d max are the current distance and maximum distance from the pixel to the center pixel respectively; a max 、a minare the maximum and minimum values ​​of the convergence factor respectively; λ is the decreasing rate parameter, k is the decreasing period parameter, λ=-2π, k=π; the farther the distance from the pixel to the central pixel, the closer the adaptive weight is to 0, and the features of the other image are considered more to reduce the splicing traces. The adaptive weight of the central pixel approaches 1, and the image features of the area are retained more. The initial adaptive weight is normalized to ensure that the sum of all weights is 1 to maintain the illumination and color balance of the image;

[0108] S5-4: performing weighted average fusion of the low-frequency band and the high-frequency band of each pixel in the real-time overlapping area according to the real-time initial weight and the real-time adaptive weight, and obtaining the real-time overlapping area after multi-frequency band fusion;

[0109] For the weighted average fusion of low-frequency bands, the low-frequency bands in the overlapping area are smoothly transitioned to maintain the consistency of the image structure. For the weighted average fusion of high-frequency bands, the edge and detail information are retained while avoiding the generation of stitching seams. The improved multi-band fusion algorithm can effectively process the overlapping area, eliminate the stitching seams, and improve the visual quality of the stitched image. It is particularly suitable for application scenarios that require high-precision and high-quality panoramic images.

[0110] S5-5: traversing the real-time overlapping areas of all the real-time image data after image registration, and obtaining a number of real-time image data after image fusion including the real-time overlapping areas after multi-band fusion;

[0111] S6: According to the real-time stitching parameters, the real-time image data after all images are fused is stitched to obtain real-time holographic image data, including the following steps:

[0112] S6-1: Using the Improved Whale Optimization Algorithm (IWOA) optimization algorithm, generating real-time splicing parameters of each image fused real-time image data and other image fused real-time image data, including the following steps:

[0113] S6-1-1: According to the continuity, color consistency, brightness matching and other factors of the holographic image data, the fitness function of the IWOA optimization algorithm is set to evaluate the stitching quality of the stitching parameters corresponding to the IWOA individuals;

[0114] S6-1-2: Encode the splicing parameters into the encoding vector of the IWOA individual in the IWOA optimization algorithm;

[0115] S6-1-3: Initialize the IWOA optimization algorithm parameters and use the Circle chaotic sequence for initialization to obtain an initial IWOA population containing several initial IWOA individuals;

[0116] The formula for initializing the Circle chaotic sequence is:

[0117]

[0118] In the formula, x i+1,j+1 The initial IWOA individual position generated by Circle chaos mapping; x i,j is the randomly generated initial IWOA individual position; mod(·) is the mod function; i is the IWOA individual indicator; j is the dimension indicator;

[0119] S6-1-4: Based on the guidance coefficient of the search space node cluster for the IWOA individual, the IWOA optimization algorithm is used to surround, search, and attack the IWOA population with a bubble net, and the initial IWOA population is updated to obtain an updated IWOA population;

[0120] Specifically, p is randomly generated. If p<0.5 and |A|<1, the prey encirclement behavior is executed to update the initial IWOA population position. If p<0.5 and |A|≥1, the prey search behavior is executed to update the initial IWOA population position. If p≥0.5, the bubble net attack behavior is executed to update the initial IWOA population position. Where p is the update parameter and A is the step size coefficient.

[0121] The formula for encircling prey behavior is:

[0122] X 1 (t+1)=ρ n [X * (t)-AD]

[0123] Where, X 1 (t+1) is the IWOA individual position updated by the encirclement behavior; X * (t) is the optimal IWOA individual position; D is the distance between the current IWOA individual and the optimal IWOA individual; ρ n is the guidance coefficient of the nth search space node for the IWOA individual;

[0124] The formula for searching for prey behavior is:

[0125] X 2 (t+1)=ρ n [X rand (t)-AD]

[0126] Where, X 2 (t+1) is the IWOA individual position updated by the prey search behavior; X rand (t) is the position of a random IWOA individual selected from the IWOA population; ρ nis the guidance coefficient of the nth search space node for the IWOA individual;

[0127] The formula for the attack behavior of Bubble Network is:

[0128] X 3 (t+1)=ρ n [D'c bl cos(2πl)+X * (t)]

[0129] Where, X 3 (t+1) is the position of the IWOA individual updated by the attack behavior of the bubble network; D' is the distance between the current IWOA individual and the prey; b is the constant defining the spiral equation, b=1; l is a random number between [-1,1]; ρ n is the guidance coefficient of the nth search space node for the IWOA individual;

[0130] S6-1-5: Perform dynamic reverse learning based on the updated IWOA population to obtain a reverse IWOA population, and calculate the fitness value of each IWOA individual in the updated IWOA population and the reverse IWOA population;

[0131] S6-1-6: Update the optimal IWOA individual according to the fitness values ​​of all IWOA individuals to obtain an updated optimal IWOA individual;

[0132] S6-1-7: If the fitness value of the updated optimal IWOA individual meets the requirement or the number of iterations reaches the threshold, the optimal solution corresponding to the updated optimal IWOA individual is output to obtain the optimal real-time splicing parameters of the real-time image data after image fusion and the real-time image data after fusion of other images;

[0133] Real-time stitching parameters include but are not limited to:

[0134] Real-time alignment parameters: adjust the positional relationship between images to ensure that the images are correctly aligned and reduce the appearance of stitching seams;

[0135] Real-time brightness / contrast adjustment parameters: adjust the brightness or contrast of the image under different lighting conditions to make the stitched image more visually consistent;

[0136] Real-time color correction parameters: Since photos taken with different cameras or under different lighting conditions may have color differences, color correction parameters are used to adjust the image color so that the color transition of the stitching area is natural;

[0137] S6-2: performing image stitching on the real-time image data after all images are fused according to the real-time stitching parameters to obtain initial real-time holographic image data;

[0138] S6-3: post-processing the initial real-time holographic image data to obtain final real-time holographic image data;

[0139] Post-processing includes eliminating prominent seams, restoring excessive brightness / contrast, adjusting abnormal color correction, etc.

[0140] Embodiment 2:

[0141] like Figure 2 As shown, this embodiment provides a multi-camera image stitching system for implementing a multi-camera image stitching method, the system includes an image preprocessing unit, a feature point extraction unit, a feature point matching unit, an image registration unit, an image fusion unit and an image stitching unit connected in sequence;

[0142] An image preprocessing unit, used to perform image preprocessing on a plurality of real-time image data captured by a plurality of camera devices to obtain a plurality of preprocessed real-time image data;

[0143] A feature point extraction unit, used to collect real-time image edge information of each pre-processed real-time image data, and extract a number of real-time feature points of each pre-processed real-time image data according to the real-time image edge information;

[0144] A feature point matching unit is used to use a pre-trained feature point matching model to perform feature point matching on a plurality of real-time feature points of all pre-processed real-time image data to obtain a plurality of feature point matching results;

[0145] An image registration unit, used to perform image registration on all pre-processed real-time image data according to a plurality of feature point matching results, to obtain a plurality of real-time image data after image registration;

[0146] An image fusion unit, used to use an improved multi-band fusion algorithm to perform image fusion on real-time overlapping areas of all real-time image data after image registration, to obtain real-time image data after a number of image fusions;

[0147] The image stitching unit is used to stitch the real-time image data after all images are fused according to the real-time stitching parameters to obtain real-time holographic image data.

[0148] The present invention provides a multi-camera image stitching method and system. Image preprocessing ensures that images captured by different cameras have consistent quality and format before stitching, thereby improving the efficiency and accuracy of subsequent processing. The feature point matching model constructed based on the deep learning algorithm can more accurately extract and match feature points in real-time image data, and can maintain high matching accuracy even in complex scenes. It also reduces the computational complexity and improves the processing speed, meeting the needs of real-time stitching, especially in applications in high-resolution images and multi-camera systems. By analyzing the feature point matching results, an effective image registration process is proposed, which can accurately align multiple images. It reduces errors and seam problems in the stitching process; it adopts an improved multi-band fusion algorithm, which can better process the pixel values ​​in the overlapping areas of images, so that the transition of the fused image is natural, avoids halo and blur, and improves the overall visual quality of the image; it introduces the IWOA optimization algorithm to generate image stitching parameters, which can quickly find the optimal stitching parameters and improve the speed and accuracy of image stitching; the post-processing steps of the image further optimize the image quality and ensure that the holographic image data achieves the best visual effect; it has a high degree of automation, reduces manual intervention, and is suitable for a variety of scenarios and different types of image stitching, and has wide versatility.

[0149] The present invention is not limited to the above optional implementations, and anyone can derive other various forms of products under the enlightenment of the present invention. The above specific implementations should not be understood as limiting the scope of protection of the present invention. The scope of protection of the present invention should be based on the definition in the claims, and the description can be used to interpret the claims.

Claims

1. A multi-camera image stitching method, characterized by: The steps include: Collecting a number of real-time image data captured by multiple camera devices, and performing image preprocessing on the number of real-time image data to obtain a number of preprocessed real-time image data; Collecting real-time image edge information of each pre-processed real-time image data, and extracting a number of real-time feature points of each pre-processed real-time image data according to the real-time image edge information; Using a pre-trained feature point matching model, feature point matching is performed on a number of real-time feature points of all pre-processed real-time image data to obtain a number of feature point matching results; According to the matching results of several feature points, image registration is performed on all pre-processed real-time image data to obtain several real-time image data after image registration; Using an improved multi-band fusion algorithm, image fusion is performed on the real-time overlapping areas of all real-time image data after image registration to obtain real-time image data after several images are fused; According to the real-time stitching parameters, all images are fused and the real-time image data is stitched to obtain real-time holographic image data.

2. A multi-camera image stitching method according to claim 1, characterized in that: The image preprocessing includes Gaussian denoising processing, image enhancement processing, viewing angle adjustment processing, image grayscale processing and size normalization processing performed in sequence.

3. The multi-camera image stitching method according to claim 1, characterized in that: Collecting the real-time image edge information of each pre-processed real-time image data, and extracting a number of real-time feature points of each pre-processed real-time image data according to the real-time image edge information, comprises the following steps: Use edge detection algorithm to perform edge detection on pre-processed real-time image data to obtain corresponding real-time image edge information; According to the edge information of the real-time image, a feature point extraction algorithm is used to extract several real-time feature points of the pre-processed real-time image data; All pre-processed real-time image data are traversed to obtain a number of real-time feature points of each pre-processed real-time image data.

4. The multi-camera image stitching method according to claim 1, characterized in that: The feature point matching model is constructed based on the RF-BiGCN algorithm, and the feature point matching model includes a key feature point screening module constructed based on the RF algorithm and a feature point matching module constructed based on the BiGCN algorithm.

5. The multi-camera image stitching method according to claim 4, characterized in that: Using the pre-trained feature point matching model, feature point matching is performed on a number of real-time feature points of all pre-processed real-time image data to obtain a number of feature point matching results, including the following steps: Converting a number of real-time feature points of the pre-processed real-time image data into real-time feature point vectors, and inputting the real-time feature point vectors into a pre-trained feature point matching model; Use the key feature point screening module to extract several key real-time feature points in the real-time feature point vector; According to several key real-time feature points, a feature point matching module is used to perform feature point matching on all pre-processed real-time image data to obtain several initial feature point matching results; The matching feature point screening algorithm is used to screen the initial several feature point matching results to obtain the final several feature point matching results.

6. The multi-camera image stitching method according to claim 1, characterized in that: According to the matching results of several feature points, image registration is performed on all pre-processed real-time image data to obtain several real-time image data after image registration, including the following steps: According to the geometric relationship between the real-time image data after preprocessing, a corresponding real-time geometric transformation model is selected; Generating real-time model parameters of the real-time geometric transformation model according to the matching results of the plurality of feature points, and updating the real-time geometric transformation model according to the real-time model parameters to obtain an updated real-time geometric transformation model; According to the updated real-time geometric transformation model, image registration is performed on all pre-processed real-time image data to obtain a number of real-time image data after image registration.

7. The multi-camera image stitching method according to claim 1, characterized in that: Using the improved multi-band fusion algorithm, image fusion is performed on the real-time overlapping areas of all real-time image data after image registration to obtain real-time image data after several image fusions, including the following steps: Extracting the real-time overlapping area of ​​each image registered real-time image data and other image registered real-time image data; Perform frequency band division on the real-time overlapping area to obtain corresponding low-frequency bands and high-frequency bands, and generate corresponding real-time initial weights according to the position of each pixel point in the real-time overlapping area; According to the distance from each pixel point in the real-time overlapping area to the central pixel point, the corresponding real-time adaptive weight is generated using the convergence factor; According to the real-time initial weight and the real-time adaptive weight, the low-frequency band and the high-frequency band of each pixel in the real-time overlapping area are weighted average fused to obtain the real-time overlapping area after multi-band fusion; The real-time overlapping areas of all the real-time image data after image registration are traversed to obtain a number of real-time image data after image fusion including the real-time overlapping areas after multi-band fusion.

8. The multi-camera image stitching method according to claim 1, characterized in that: According to the real-time stitching parameters, all images are fused and the real-time image data is stitched to obtain real-time holographic image data, including the following steps: Use a swarm intelligence optimization algorithm to generate real-time splicing parameters for each image fused with real-time image data after fusion with other images; According to the real-time stitching parameters, the real-time image data after all images are fused is stitched to obtain the initial real-time holographic image data; The initial real-time holographic image data is post-processed to obtain the final real-time holographic image data.

9. The multi-camera image stitching method according to claim 8, characterized in that: The swarm intelligence optimization algorithm is the IWOA optimization algorithm.

10. A multi-camera image stitching system, used to implement the multi-camera image stitching method according to any one of claims 1 to 9, characterized in that: The system comprises an image preprocessing unit, a feature point extraction unit, a feature point matching unit, an image registration unit, an image fusion unit and an image stitching unit which are connected in sequence.

Images