A multi-projection image splicing method and system based on geometric correction

Abstract

The present invention relates to the field of projection display technology, and discloses a multi-projection screen splicing method and system based on geometric correction. The method includes: obtaining the initial parameters and original screen data of the multi-projection equipment, the original data containing overlapping area images and collected in a preset direction; performing geometric correction on the original screen, extracting geometric distortion feature points and generating a mapping relationship matrix; dividing the projection boundary based on the matrix, adjusting the parameters to form a correction parameter set; spatially aligning the screen data according to the parameter set to generate multi-channel data; fusing the overlapping areas of the multi-channel data to eliminate light intensity differences and color deviations, and generate a continuous spliced ​​screen. The system includes data acquisition, geometric correction, parameter adjustment, spatial registration and fusion processing modules. The present invention improves the accuracy and visual effects of multi-projection screen splicing through high-precision geometric correction, adaptive fusion and dynamic adjustment, is suitable for large-scale display scenes, and has the advantages of strong environmental adaptability and high degree of automation.

Description

Technical Field

[0001] The present invention relates to the field of projection display technology, and in particular to a multi-projection image splicing method and system based on geometric correction. Background Art

[0002] In the field of projection display, a single projector is limited by factors such as physical size, brightness output, and resolution, making it difficult to meet the needs of large-scale, high-definition display scenarios such as large conference rooms, digital exhibition halls, and virtual simulation systems. Multi-projection image splicing technology, by integrating the display images of multiple projectors, can build large-scale, high-resolution display systems, becoming an important technical direction to break through the limitations of a single device. However, this technology faces many challenges in practical application, as follows:

[0003] Differences in installation position, angle, and lens parameters between multiple projection devices can lead to nonlinear geometric distortions in the projected images, such as keystone and pincushion distortion. This geometric distortion prevents precise alignment of image content in overlapping areas, resulting in significant geometric misalignment in the stitched images, severely impacting visual quality. Traditional geometric correction methods, which rely on simple linear transformations, struggle to effectively handle complex nonlinear distortions and suffer from insufficient correction accuracy.

[0004] Different projectors have varying degrees of light source aging and color calibration parameters. Even after initial calibration, overlapping areas can still experience inconsistent light intensity and color deviation. For example, the borders between adjacent images may show bright or dark streaks or color discontinuities, disrupting the visual coherence of the overall image. Traditional fusion algorithms often use a fixed-weighted overlay approach, which is unable to adapt to varying lighting conditions and device states, resulting in suboptimal fusion results.

[0005] In real-world applications, ambient lighting conditions (such as natural light intensity and artificial light distribution) can change dynamically, and the operating status of the projection equipment itself (such as lamp brightness decay and lens displacement) can also change over time. Traditional splicing systems lack real-time monitoring and dynamic adjustment mechanisms, making it difficult to maintain stable splicing quality despite environmental changes. Frequent manual recalibration is required, resulting in high costs.

[0006] Existing multi-projection stitching solutions typically require complex manual calibration steps, such as manually marking feature points and adjusting projection parameters. These operations are tedious and time-consuming, and require high operator expertise. This not only limits the widespread application of this technology in non-professional fields, but also makes it difficult to meet the needs of scenarios that require rapid deployment, such as temporary exhibitions and emergency command systems.

[0007] Overlapping areas are crucial for multi-projection stitching, and their processing directly impacts stitching quality. Traditional methods handle overlapping areas in a crude manner, failing to fully consider differences in pixel-level brightness, color, and geometric features. For example, when blurring edges, the lack of analysis of gradients between adjacent pixels can lead to over- or under-blurring in transition areas, compromising image clarity and naturalness. Summary of the Invention

[0008] The object of the present invention is to provide a multi-projection image stitching method and system based on geometric correction to solve the problems raised in the above background technology.

[0009] To achieve the above-mentioned object, the present invention provides the following technical solution: a multi-projection image stitching method based on geometric correction, the method comprising:

[0010] Acquiring initial projection parameters of multiple projection devices and original image data corresponding to the projection devices; the original image data includes image content of overlapping areas and is collected according to a preset projection direction;

[0011] Performing geometric correction processing on the original picture data, extracting geometric distortion feature points between each picture, and generating a mapping relationship matrix according to the spatial distribution of the geometric distortion feature points;

[0012] Dividing the screen area of ​​each projection device into projection boundaries based on the mapping relationship matrix, and adjusting the projection parameters of each projection device according to the division result to form a correction parameter set;

[0013] Performing spatial registration on the adjusted image data of each projection device according to the correction parameter set to generate registered multi-channel image data;

[0014] The multi-channel image data is subjected to overlapping area fusion processing to eliminate light intensity differences and color deviations at the splicing boundaries and generate a continuous splicing image.

[0015] Preferably, performing geometric correction processing on the original picture data includes:

[0016] Calculating initial distortion parameters of each image based on the installation position and lens parameters of the projection device;

[0017] According to the distribution density of the geometric distortion feature points, points with distortion rates exceeding a threshold on both sides are selected from the overlapping area as distortion demarcation points;

[0018] Calculating the spatial transformation matrix of the distortion demarcation point based on the initial distortion parameters, and taking the center point of the picture as a reference transformation point;

[0019] Distortion data interpolation is performed according to the spatial transformation matrix, the reference transformation point, and the position of the distortion demarcation point to generate geometrically corrected picture data.

[0020] Preferably, interpolating the distortion data according to the spatial transformation matrix, the reference transformation point, and the position of the distortion demarcation point includes:

[0021] using the area between the reference transformation point and the distortion demarcation point as a main correction area, and other areas as auxiliary correction areas;

[0022] Constructing a picture distortion compensation function based on the initial distortion parameters; the picture distortion compensation function is a correspondence between pixel positions in the projected picture and actual spatial positions;

[0023] interpolating the main correction area according to the image distortion compensation function to generate first correction coordinates, and interpolating the auxiliary correction area to generate second correction coordinates;

[0024] In combination with the distortion feature point distribution of the second correction coordinates, the second correction coordinates are corrected using a distortion gradient function and the image distortion compensation function to generate third correction coordinates;

[0025] The first correction coordinates and the third correction coordinates are combined to form geometrically corrected picture data.

[0026] Preferably, dividing the projection boundary of the screen area of ​​each projection device based on the mapping relationship matrix includes:

[0027] Calculating the projection coverage range corresponding to the reference transformation point in the spatial coordinate system according to the installation angle of the projection device as the reference range;

[0028] Determine the boundary coordinates of each projection device screen in the spatial coordinate system based on the reference range and the spatial transformation matrix;

[0029] The image resolution and projection angle of each projection device are adjusted according to the boundary coordinates to form a correction parameter set.

[0030] Preferably, performing overlapping area fusion processing on the multi-channel image data includes:

[0031] The multi-channel image data is input into an image fusion model, and an output result of the image fusion model is used as a spliced ​​image.

[0032] Preferably, the establishment of the image fusion model includes:

[0033] Collecting projection image samples under various lighting conditions, and dividing the samples into brightness fusion samples, color fusion samples, and boundary transition samples;

[0034] Establish fusion standards for brightness fusion, color fusion and boundary transition in HSL color space;

[0035] Constructing a fusion weight distribution model according to the fusion standard;

[0036] Training a convolutional neural network based on the classified projection image samples to form a brightness fusion sub-model, a color fusion sub-model, and a boundary transition sub-model;

[0037] The fusion weight distribution model, the brightness fusion sub-model, the color fusion sub-model and the boundary transition sub-model are combined into a picture fusion model.

[0038] Preferably, the multi-channel image data is input into an image fusion model, and the output result of the image fusion model is used as a spliced ​​image, including:

[0039] Assigning brightness weight, color weight and transition weight to each pixel in the overlapping area by using the fusion weight assignment model;

[0040] Performing light intensity smoothing processing on the pixel area assigned to the brightness weight through the brightness fusion sub-model;

[0041] Performing color difference compensation processing on the pixel area assigned to the color weight through the color fusion sub-model;

[0042] Performing edge blurring processing on the pixel area assigned to the transition weight through the boundary transition sub-model;

[0043] Combine the output results of each sub-model into a continuous mosaic image.

[0044] Preferably, the fusion standards for brightness fusion, color fusion and boundary transition are respectively established in the HSL color space, including:

[0045] Calculating an average value of brightness values ​​in the projected image samples as a reference brightness, an average value of saturation as a reference saturation, and an average value of hue as a reference hue;

[0046] The brightness fusion standard is that the fluctuation range of the brightness value in the overlapping area does not exceed a preset ratio of the reference brightness, and the saturation fluctuation range is less than a first threshold;

[0047] The color fusion standard is that the distribution variance of the hue value in the overlapping area is less than the second threshold, and the difference between the brightness value and the reference brightness is less than the third threshold;

[0048] The boundary transition standard is that the hue gradient change rate of adjacent pixels in the overlapping area is less than a fourth threshold, and the saturation gradient change rate is less than a fifth threshold.

[0049] Preferably, after generating the continuous spliced ​​images, the method further includes:

[0050] Real-time monitoring of the working status of each projection device and ambient lighting parameters;

[0051] The projection angle, resolution and fusion weight in the correction parameter set are dynamically adjusted according to the monitoring results to adapt to environmental changes.

[0052] Preferably, the present invention further includes a multi-projection image splicing system based on geometric correction, the system comprising:

[0053] A data acquisition module, configured to acquire initial projection parameters of a plurality of projection devices and raw image data corresponding to the projection devices, wherein the raw image data includes image content of overlapping areas and is collected according to a preset projection direction;

[0054] A geometric correction module, configured to perform geometric correction processing on the original picture data, extract geometric distortion feature points between each picture, and generate a mapping relationship matrix according to the spatial distribution of the geometric distortion feature points;

[0055] a parameter adjustment module, configured to divide the screen area of ​​each projection device into projection boundaries based on the mapping relationship matrix, and adjust the projection parameters of each projection device according to the division result to form a correction parameter set;

[0056] A spatial registration module, configured to perform spatial registration on the adjusted image data of each projection device according to the correction parameter set to generate registered multi-channel image data;

[0057] The fusion processing module is used to perform overlapping area fusion processing on the multi-channel image data, eliminate the light intensity difference and color deviation at the splicing boundary, and generate a continuous splicing image.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] In terms of geometric correction, by extracting geometric distortion feature points and generating a mapping relationship matrix, and combining the installation position of the projection equipment and lens parameters to calculate the initial distortion parameters, the nonlinear distortion characteristics of each screen can be accurately analyzed. By screening the distortion demarcation points, constructing the spatial transformation matrix and the image distortion compensation function, the main correction area and the auxiliary correction area are interpolated separately, and the distortion gradient function is introduced to correct the auxiliary area, achieving multi-level geometric correction from the pixel level to the regional level. This refined correction method can effectively eliminate complex distortions such as trapezoidal and pincushion shapes, accurately align the geometric positions of each projected image, and significantly reduce the geometric misalignment errors of the splicing boundaries, thereby constructing a large-scale image with extremely high geometric consistency.

[0060] In terms of overlapping area fusion processing, by establishing a fusion standard based on the HSL color space, quantitative indicators are formulated from the three dimensions of brightness, color, and boundary transition to ensure that the light intensity fluctuation, color deviation, and pixel gradient changes in the overlapping area are controlled within a reasonable range. Convolutional neural networks are used to construct brightness fusion, color fusion, and boundary transition sub-models, and pixel-level weight distribution is achieved through a fusion weight distribution model. This can adaptively perform light intensity smoothing, color difference compensation, and edge blurring based on the characteristics of different areas. This deep learning-based fusion model can effectively eliminate bright and dark stripes and color faults at the splicing boundaries, making the transition between overlapping areas natural and smooth, and greatly improving the visual coherence of the overall picture, approaching the effect of a single-screen display.

[0061] In terms of environmental adaptability and dynamic adjustment, by real-time monitoring of the projector's operating status (such as bulb brightness and lens parameters) and ambient lighting parameters (such as light intensity and color temperature), the system can dynamically adjust the projection angle, resolution, and fusion weight within the correction parameter set. For example, when increased ambient lighting causes a decrease in image brightness and contrast, the system automatically increases the brightness fusion weight in the overlapping area to improve light intensity consistency. When the aging of a device's bulb causes color shift, the color fusion sub-model can adjust the chromatic aberration compensation parameters in real time to ensure color accuracy. This dynamic adaptation mechanism enables the system to maintain stable splicing quality under varying environmental conditions and device states, reducing the frequency of manual calibration, lowering operating costs, and enhancing the system's practicality and reliability.

[0062] In terms of automation and processing efficiency, the entire stitching process, from raw image acquisition to final stitched image generation, is fully automated through the collaborative work of modules including data acquisition, geometric correction, parameter adjustment, spatial registration, and fusion processing. Removing the need for manual feature point marking or parameter adjustment significantly reduces operational complexity and operator expertise, shortening system deployment time and making it suitable for rapidly building large-scale display systems, such as temporary exhibitions and emergency command centers. Furthermore, the convolutional neural network-based fusion model, trained on a large number of samples, can rapidly process multi-channel image data, ensuring real-time performance and meeting the demands of dynamic image display.

[0063] In terms of refined processing of overlapping areas, by classifying the pixels in the overlapping areas by brightness, color, and transition weight, and processing them using corresponding sub-models, multi-dimensional, pixel-level refined control of the overlapping areas is achieved. For example, in edge blurring, the hue and saturation gradient change rates of adjacent pixels are analyzed in conjunction with the distortion gradient function to ensure that the degree of blur matches the image content, avoiding loss of detail due to excessive blurring or residual boundaries due to insufficient blurring. This refined processing ensures that the stitched image not only maintains overall continuity, but also makes the details clearer and more natural, improving the visual quality of the image.

[0064] Furthermore, by constructing a mapping matrix and correction parameter sets, the present invention defines precise projection boundaries for each projector and adjusts resolution and projection angle, enabling coordinated control and parameter optimization of multiple devices. This standardized approach facilitates system expansion and maintenance, flexibly adapting to varying numbers and models of projectors. It offers strong versatility and scalability, meeting diverse large-scale display needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 The multi-projection image splicing method based on geometric correction of the present invention;

[0066] Figure 2 Flowchart of geometric correction processing of original picture data;

[0067] Figure 3 Flowchart of the distorted data interpolation process;

[0068] Figure 4 Flowchart established for the image fusion model;

[0069] Figure 5 Flowchart of pixel weight assignment and sub-model processing in overlapping areas. DETAILED DESCRIPTION

[0070] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0071] See also Figure 1-Figure 5 The present invention relates to a multi-projection image stitching method based on geometric correction, and the specific implementation steps are as follows:

[0072] The data acquisition device acquires initial projection parameters for multiple projection devices, including at least the device's installation location, lens parameters, initial projection angle, and resolution. Simultaneously, raw image data corresponding to each projection device is collected according to a preset projection direction. This raw image data includes image content from the overlapping areas of adjacent projection images. The preset projection direction is determined based on the installation layout of the projection devices and the target splicing display area, ensuring spatial splicing of the raw image data.

[0073] Geometric correction is performed on the original image data. An image processing algorithm is used to extract geometric distortion feature points between each image. These feature points are points in the image where the actual pixel position deviates from the theoretical position by more than a preset threshold due to factors such as projection device lens distortion and installation angle deviation. Based on the spatial distribution of these geometric distortion feature points, a coordinate transformation algorithm is used to generate a mapping matrix. This matrix represents the correspondence between the actual position of each pixel in the image in the spatial coordinate system and the corrected target position.

[0074] Based on the mapping matrix, the projection boundary is divided for each projector's image area. Specifically, by analyzing the spatial coordinate distribution of each pixel in the mapping matrix, the coverage of each projected image within the target stitching area and the overlap boundary of adjacent images are determined. Based on the division results, the projection parameters of each projector, including projection angle, resolution, and image offset, are adjusted to form a correction parameter set.

[0075] The adjusted image data from each projection device is spatially registered according to the parameters in the calibration parameter set. Using a coordinate transformation algorithm, each image data is converted from the original coordinate system to a unified target coordinate system, ensuring that the overlapping areas of adjacent images are precisely aligned in space, generating registered multi-channel image data.

[0076] The overlapping areas of multi-channel image data are fused, and a fusion algorithm is used to eliminate light intensity differences and color deviations at the stitching boundaries. Specifically, by calculating the brightness and color weights of the pixels in the overlapping area, the overlapping parts of adjacent images are smoothed using a gradual transition method, ultimately generating a visually continuous stitching image.

[0077] Example 1:

[0078] The initial distortion parameters for each image are calculated based on the projector's installation position and lens parameters. The installation position is defined by a three-dimensional coordinate system (X, Y, and Z axes), which includes the spatial coordinates of the projector's optical center and the rotation angles around each axis (pitch, yaw, and roll). Lens parameters include inherent properties such as focal length, pixel size, and sensor size. Using the projector's pinhole imaging model and lens distortion models (such as radial and tangential distortion), the initial distortion parameters are calculated. These parameters reflect the geometric distortion trends of the image caused by the lens' optical characteristics and installation position.

[0079] According to the distribution density of geometric distortion feature points, points with distortion rates exceeding the threshold on both sides are selected from the overlapping area as distortion demarcation points. In the specific operation, feature point detection is first performed on the overlapping area of ​​adjacent projection images (such as using SIFT, ORB and other algorithms) to obtain the pixel coordinates of the feature points. For each feature point, the coordinate deviation value between the current image and the adjacent image is calculated, and the ratio of the deviation value to the theoretical coordinate value is defined as the distortion rate. The preset threshold is set according to the accuracy requirements of the projection system (such as 5%), and all feature points are traversed. Points with distortion rates exceeding the threshold are marked as distortion demarcation points. These points are usually located at the edge of the image or in areas where lens distortion is significantly affected, and serve as key control points for subsequent correction.

[0080] The spatial transformation matrix of the distortion boundary point is calculated based on the initial distortion parameters. The center point of the image is used as the reference transformation point. This point is usually located at the geometric center of the image, and its theoretical coordinates have a small deviation from the actual coordinates, serving as the reference for correction. The spatial transformation matrix is ​​composed of basic transformations such as translation, rotation, and scaling, and aims to map the actual coordinates of the distortion boundary point to the theoretical coordinates. For example, if the actual coordinates of a distortion boundary point are (u1, v1) and the theoretical coordinates are (u0, v0), the translation transformation component is (u0-u1, v0-v1). The rotation and scaling components are determined by fitting the transformation relationship between adjacent feature points using the least squares method, ultimately forming a spatial transformation matrix containing translation, rotation, and scaling parameters.

[0081] Distortion data is interpolated based on the spatial transformation matrix, the reference transformation point, and the position of the distortion demarcation point. The area between the reference transformation point and the distortion demarcation point is divided into the main correction area. Because this area contains the distortion demarcation point, it is the main concentration of geometric distortion and requires high-density interpolation correction. Other areas are auxiliary correction areas with relatively low distortion levels, and low-density interpolation is used. Within the main correction area, with the reference transformation point as the center, the spatial transformation matrix is ​​used to transform the coordinates of each pixel point. The corrected pixel value is calculated using the bicubic interpolation algorithm to ensure the accuracy of distortion compensation. In the auxiliary correction area, with the boundary of the main correction area as the reference, a linear interpolation algorithm is used to expand the correction result to evenly compensate for the distortion of the entire image. Through the above steps, the geometrically corrected image data is generated, providing a basis for coordinate alignment for subsequent image stitching.

[0082] Example 2:

[0083] Based on the spatial relationship between the reference transformation point and the distortion demarcation point, the area between them is designated the primary correction area. This area covers areas of the image with significant geometric distortion, such as areas with concentrated distortion at the edges of the lens. The remaining area is designated the auxiliary correction area, primarily covering areas near the center of the image with less distortion. This division into primary and auxiliary correction areas aims to adopt differentiated correction strategies for different degrees of distortion, balancing correction accuracy and computational efficiency.

[0084] A distortion compensation function is constructed based on the initial distortion parameters. This function describes the correspondence between pixel positions in the projected image and their real-world spatial locations through a nonlinear mapping relationship. This can be implemented through polynomial fitting or a parameterized model. For example, using the pixel's horizontal and vertical coordinates (u, v) as input variables, a polynomial containing quadratic and cubic terms is fitted to output the pixel's three-dimensional coordinates (X, Y, Z) in real space, thereby establishing a distortion compensation mapping from the pixel coordinate system to the spatial coordinate system.

[0085] The primary correction area is interpolated using the distortion compensation function to generate the first correction coordinates. For each pixel within the primary correction area, the distortion compensation function is used to calculate its corresponding actual spatial position. This is then converted to the target correction position (i.e., the ideal projection position) using the spatial transformation matrix, resulting in the first correction coordinates for that pixel. This process utilizes point-by-point interpolation. Taking advantage of the high distortion rate within the primary correction area, high-density sampling and refined calculations ensure that each pixel's positional deviation is accurately corrected.

[0086] For the auxiliary correction area, interpolation is also performed based on the image distortion compensation function to generate the second correction coordinates. However, considering the lower degree of distortion in this area, a relatively sparse sampling strategy is adopted during the interpolation calculation. For example, the boundary coordinates of the main correction area are used as a reference, and linear interpolation is used to expand to the auxiliary area to quickly generate preliminary correction coordinates. This processing method can reduce the amount of calculation and improve processing efficiency while ensuring the basic correction effect.

[0087] Based on the distribution of distortion feature points in the second correction coordinates, the distortion gradient function and the image distortion compensation function are combined to correct the second correction coordinates to generate the third correction coordinates. The distortion gradient function is used to quantify the distortion differences between adjacent pixels within the auxiliary correction area. By calculating the coordinate deviation rate between a pixel and its neighboring pixels, the distortion trend in that area is determined. For areas with large distortion gradients (i.e., areas with significant distortion changes), quadratic interpolation correction is performed using the image distortion compensation function to adjust the pixel correction coordinates to eliminate errors that may be introduced by linear interpolation. For areas with smaller distortion gradients, the original second correction coordinates are maintained to avoid overcalculation. This dynamic correction mechanism optimizes the correction accuracy of the auxiliary correction area, ensuring uniform distortion compensation across the entire image.

[0088] The first correction coordinates generated by the primary correction area are combined with the third correction coordinates corrected by the auxiliary correction area to form the complete geometrically corrected image data. During the merging process, the final correction coordinates of each pixel are recorded in a coordinate mapping table to ensure that all pixels in the image have undergone targeted distortion compensation. This achieves comprehensive correction of the geometric distortion of the projected image and provides a high-precision coordinate foundation for subsequent spatial registration and image stitching.

[0089] Example 3:

[0090] Based on the installation angle of the projection device, the projection coverage corresponding to the reference transformation point is calculated in the spatial coordinate system as the reference range. The installation angle is represented by the projection device's pitch angle (rotation angle around the X-axis), yaw angle (rotation angle around the Y-axis), and roll angle (rotation angle around the Z-axis). Combined with the spatial coordinates of the device's optical center (X0, Y0, Z0), the projection direction vector is determined using the principles of three-dimensional coordinate transformation. Using the lens's field of view (such as horizontal field of view angle α and vertical field of view angle β) as boundary constraints, the coverage of the reference transformation point on the target stitching plane is calculated through geometric projection, forming a rectangular reference range centered at the reference transformation point (for example, horizontally from X1 to X2, vertically from Y1 to Y2). This range reflects the projection device's ideal image coverage capability.

[0091] Based on the reference range and the spatial transformation matrix, the boundary coordinates of each projector's image in the spatial coordinate system are determined. The spatial transformation matrix, established through a geometric correction process, is used to map the pixel coordinates of the original image to the spatial coordinate system. All pixels (or feature points) in each projected image are traversed and converted into spatial coordinates (X, Y) using the spatial transformation matrix. These coordinates are then compared with the reference range. For pixels outside the reference range, the actual image boundary is determined by analyzing their distribution: horizontally, the minimum X_min and maximum X_max values ​​of all pixels are taken, and vertically, the minimum Y_min and maximum Y_max values ​​are taken. This forms the rectangular boundary (X_min, Y_min, X_max, Y_max) of the projected image in the spatial coordinate system. The boundaries of adjacent images must ensure that the overlap width meets system design requirements (e.g., 10%-20% of the width of a single image). If the overlap is insufficient or excessive, the boundary coordinates are recalculated by adjusting them.

[0092] Based on the boundary coordinates, the image resolution and projection angle of each projector are adjusted to form a correction parameter set. If the boundary coordinates indicate that the image horizontally exceeds the target stitching area (for example, X_max exceeds the preset stitching width), the image is adapted by reducing the horizontal resolution (reducing the number of pixel columns) or decreasing the horizontal projection angle (narrowing the field of view). If vertical coverage is insufficient (for example, Y_min is greater than the preset stitching height), the vertical projection angle is increased or the device installation height is adjusted to expand coverage. Simultaneously, the image offset parameters must be adjusted to ensure that the center of the overlapping area of ​​adjacent images aligns with the theoretical stitching position. The correction parameter set includes parameters such as resolution (horizontal and vertical pixels), projection angle (horizontal and vertical field of view), and offset (horizontal and vertical offset). Through this adjustment process, each projected image is precisely positioned in the spatial coordinate system, providing clear boundary constraints for subsequent spatial registration.

[0093] Example 4:

[0094] Building an image fusion model requires sample collection and classification. Projection image samples are collected under various lighting conditions (e.g., low brightness, high brightness, and mixed color temperature environments), covering different projector operating states (e.g., full power output, half-power energy-saving mode). After collection, the samples are classified into three categories based on the processing objectives: brightness fusion samples, which focus on uneven light intensity in overlapping areas; color fusion samples, which focus on hue and saturation deviations; and boundary transition samples, which include pixel gradient variations at the splicing boundaries. Classification is accomplished through manual annotation or automated algorithmic recognition to ensure representativeness of each sample type.

[0095] When establishing the fusion standard in the HSL color space, the statistics of all projected image samples are first calculated: the brightness values ​​of the overlapping areas of the samples are extracted and their average value is calculated as the benchmark brightness; the saturation values ​​are extracted and their average value is calculated as the benchmark saturation; and the hue values ​​are extracted and their average value is calculated as the benchmark hue.

[0096] Brightness fusion standard: The brightness value fluctuation range of each pixel in the overlapping area must not exceed a preset ratio of the baseline brightness (for example, ±15%), and the saturation fluctuation range must be less than a first threshold (for example, 10%) to avoid abnormal changes in color saturation caused by brightness adjustment.

[0097] Color fusion standard: This stipulates that the distribution variance of hue values ​​in the overlapping area must be less than a second threshold (e.g., 5°), and the difference between the brightness value of a single pixel and the reference brightness must be less than a third threshold (e.g., 15%) to ensure color consistency while maintaining brightness balance.

[0098] Boundary transition standard: The hue gradient change rate of adjacent pixels in the overlapping area is required to be less than the fourth threshold (such as 2° / pixel), and the saturation gradient change rate is required to be less than the fifth threshold (such as 3% / pixel). By controlling the spatial change rate of color parameters, a smooth transition of the stitching boundary can be achieved.

[0099] The fusion weight allocation model is constructed based on the aforementioned fusion criteria. For each pixel in the overlapping area, three weights are calculated based on the difference between its HSL value and the corresponding reference value: brightness weight, color weight, and transition weight. For example, if a pixel's brightness value deviates significantly from the reference brightness, it is assigned a higher brightness weight, and vice versa. Color weight is positively correlated with the degree of deviation from the hue and saturation. Transition weight is dynamically adjusted based on the gradient values ​​of adjacent pixels, with larger gradients resulting in higher transition weights. The sum of these three weights is 1, and this weight allocation guides different pixel regions into their corresponding sub-model processing.

[0100] In the sub-model training phase, convolutional neural network (CNN) is used to construct brightness fusion sub-model, color fusion sub-model and boundary transition sub-model respectively.

[0101] Brightness fusion sub-model: The input is the brightness value of the pixel and the corresponding brightness weight. The light intensity smoothing rule is learned through the fully connected layer or convolutional layer, and the corrected brightness value is output. The goal is to compress the brightness fluctuation into the standard range.

[0102] Color fusion sub-model: The input is the HSL value and color weight of the pixel, the convolution layer is used to extract the color features, and the hue and saturation are adjusted through matrix transformation or parameter mapping so that the color distribution meets the fusion standard.

[0103] Boundary transition sub-model: The input is the hue, saturation gradient values ​​and transition weights of adjacent pixels. It uses a recurrent neural network (RNN) or convolutional layer to capture spatial gradient features, and generates an edge blur effect through Gaussian blur or linear interpolation to reduce boundary visibility.

[0104] When combining models, the fusion weight distribution model is connected in series with the three sub-models: the pixels in the overlapping area first enter the weight distribution model to obtain three types of weights, and then enter the corresponding sub-models for processing according to the weight distribution. The specific fusion process is as follows:

[0105] Weight distribution: Each pixel is calculated based on the HSL value to obtain the brightness weight (ω_L), color weight (ω_S), and transition weight (ω_T).

[0106] Brightness processing: For pixel areas assigned to brightness weights, the brightness fusion sub-model is used to adjust the light intensity. For example, the brightness value of overly bright pixels is lowered, and the brightness value of overly dark pixels is increased, so that the overall brightness approaches the baseline value.

[0107] Color processing: Pixel areas assigned to color weights are adjusted for hue and saturation using the color fusion sub-model. For example, reddish hues are shifted toward the baseline hue, and excessive saturation is appropriately reduced.

[0108] Boundary processing: Pixel regions assigned to transition weights are used to perform gradient blending of adjacent pixels using the boundary transition sub-model, such as generating brightness and color gradients from one image to another at the stitching boundary.

[0109] Result merging: The output results of the three sub-models are superimposed according to pixel positions to obtain a fused overlapping area image, which is then spliced ​​with the non-overlapping area image to form a complete continuous image.

[0110] Through the above process, the image fusion model can achieve multi-dimensional automated processing for light intensity differences, color deviations, and abrupt boundaries in overlapping areas, ensuring the visual continuity of the stitched images.

[0111] Example 5:

[0112] After generating continuous mosaic images, the system monitors two key parameters in real time through the integrated sensor module:

[0113] Projection device operating status parameters: including device temperature (obtained through temperature sensors), projection power (calculated through current and voltage monitoring), current resolution parameters (configuration information output by the device), etc., used to determine whether the device is in a stable working state.

[0114] Ambient lighting parameters: The ambient brightness value (unit: lux) and color temperature value (unit: K) are collected by the light sensor to analyze the impact of ambient light on the visual effect of the projected image.

[0115] When the parameter changes are detected, the system triggers the dynamic adjustment mechanism. Taking the change of ambient light brightness as an example, assuming that the current ambient brightness is , the reference brightness is (determined by historical data or initial settings), the absolute value of the difference between the two As the basis for adjustment. The specific adjustment steps are as follows:

[0116] Calibration parameter set adjustment: If Exceeding the preset threshold (like ), indicating that changes in ambient light intensity significantly affect image brightness consistency, necessitating adjustments to the projection device's projection angle and resolution. For example, when ambient brightness increases, increase the projection angle to expand image coverage and reduce light intensity per unit area; simultaneously, reduce the resolution to reduce pixel density and avoid loss of detail due to overbrightness.

[0117] Fusion weight adjustment: According to and The proportional relationship between the brightness and the brightness weight coefficient in the fusion weight distribution model is dynamically adjusted. The calculation formula is: in, is the initial brightness weight coefficient (such as ), is the maximum brightness weight coefficient (such as ). This formula indicates that as the difference between the ambient brightness and the reference brightness increases, the weight of the brightness fusion sub-model increases linearly to enhance the compensation effect for the light intensity difference. At the same time, the color weight and transition weights according to Perform a reverse adjustment to ensure that the sum of the three category weights is always 1.

[0118] For changes in the working status of the projection device (such as the device temperature exceeds the threshold ), the system determines that the lens distortion parameter drift may be caused by overheating. At this time, the geometric correction module is called again, and the initial distortion parameters are recalculated based on the real-time collected device installation parameters (such as the lens thermal expansion coefficient at the current temperature). The mapping relationship matrix is ​​updated, and the projection angle and resolution in the correction parameter set are adjusted to compensate for the image geometric deviation caused by the change in device status.

[0119] Through this real-time monitoring and dynamic adjustment mechanism, the system automatically optimizes projection parameters and fusion strategies based on ambient lighting changes and device operating conditions, maintaining geometric alignment accuracy and color consistency across the spliced ​​image, and avoiding issues like blurred splicing boundaries and uneven light intensity caused by changing external conditions. This entire process requires no human intervention, enabling adaptive adjustments through closed-loop feedback, ensuring the continuous and stable operation of the multi-projection splicing system in diverse scenarios.

[0120] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0121] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A multi-projection image stitching method based on geometric correction, characterized in that: include: Acquiring initial projection parameters of multiple projection devices and original image data corresponding to the projection devices; The original image data includes image content of the overlapping area and is collected according to a preset projection direction; Performing geometric correction processing on the original picture data, extracting geometric distortion feature points between each picture, and generating a mapping relationship matrix according to the spatial distribution of the geometric distortion feature points; Dividing the screen area of ​​each projection device into projection boundaries based on the mapping relationship matrix, and adjusting the projection parameters of each projection device according to the division result to form a correction parameter set; Performing spatial registration on the adjusted image data of each projection device according to the correction parameter set to generate registered multi-channel image data; Performing overlapping area fusion processing on the multi-channel image data to eliminate light intensity differences and color deviations at the splicing boundaries and generate a continuous splicing image; The geometric correction processing of the original picture data includes: Calculating initial distortion parameters of each image based on the installation position and lens parameters of the projection device; According to the distribution density of the geometric distortion feature points, points with distortion rates exceeding a threshold on both sides are selected from the overlapping area as distortion demarcation points; Calculating the spatial transformation matrix of the distortion demarcation point based on the initial distortion parameters, and taking the center point of the picture as a reference transformation point; interpolating the distortion data according to the spatial transformation matrix, the reference transformation point, and the position of the distortion demarcation point to generate geometrically corrected picture data; Interpolating the distortion data according to the spatial transformation matrix, the reference transformation point, and the position of the distortion demarcation point includes: using the area between the reference transformation point and the distortion demarcation point as a main correction area, and other areas as auxiliary correction areas; Constructing a picture distortion compensation function based on the initial distortion parameters; the picture distortion compensation function is a correspondence between pixel positions in the projected picture and actual spatial positions; interpolating the main correction area according to the image distortion compensation function to generate first correction coordinates, and interpolating the auxiliary correction area to generate second correction coordinates; In combination with the distortion feature point distribution of the second correction coordinates, the second correction coordinates are corrected using a distortion gradient function and the image distortion compensation function to generate third correction coordinates; The first correction coordinates and the third correction coordinates are combined to form geometrically corrected picture data.

2. The multi-projection image stitching method based on geometric correction according to claim 1, characterized in that: Dividing the projection boundary of the screen area of ​​each projection device based on the mapping relationship matrix includes: Calculating the projection coverage range corresponding to the reference transformation point in the spatial coordinate system according to the installation angle of the projection device as the reference range; Determine the boundary coordinates of each projection device screen in the spatial coordinate system based on the reference range and the spatial transformation matrix; The image resolution and projection angle of each projection device are adjusted according to the boundary coordinates to form a correction parameter set.

3. The multi-projection image stitching method based on geometric correction according to claim 1, characterized in that: Performing overlapping area fusion processing on the multi-channel image data includes: The multi-channel image data is input into an image fusion model, and an output result of the image fusion model is used as a spliced ​​image.

4. The multi-projection image stitching method based on geometric correction according to claim 3, characterized in that: The establishment of the picture fusion model includes: Collecting projection image samples under various lighting conditions, and dividing the samples into brightness fusion samples, color fusion samples, and boundary transition samples; Establish fusion standards for brightness fusion, color fusion and boundary transition in HSL color space; Constructing a fusion weight distribution model according to the fusion standard; Training a convolutional neural network based on the classified projection image samples to form a brightness fusion sub-model, a color fusion sub-model, and a boundary transition sub-model; The fusion weight distribution model, the brightness fusion sub-model, the color fusion sub-model and the boundary transition sub-model are combined into a picture fusion model.

5. The multi-projection image stitching method based on geometric correction according to claim 4, characterized in that: Inputting the multi-channel image data into an image fusion model, and using the output result of the image fusion model as a spliced ​​image, includes: Allocate brightness weight, color weight and transition weight to each pixel in the overlapping area by using the fusion weight allocation model; Performing light intensity smoothing processing on the pixel area assigned to the brightness weight through the brightness fusion sub-model; Performing color difference compensation processing on the pixel area assigned to the color weight through the color fusion sub-model; Performing edge blurring processing on the pixel area assigned to the transition weight through the boundary transition sub-model; Combine the output results of each sub-model into a continuous mosaic image.

6. The multi-projection image stitching method based on geometric correction according to claim 4, characterized in that: The fusion standards for brightness fusion, color fusion, and boundary transition in the HSL color space are as follows: Calculating an average value of brightness values ​​in the projected image samples as a reference brightness, an average value of saturation as a reference saturation, and an average value of hue as a reference hue; The brightness fusion standard is that the fluctuation range of the brightness value in the overlapping area does not exceed a preset ratio of the reference brightness, and the saturation fluctuation range is less than a first threshold; The color fusion standard is that the distribution variance of the hue value in the overlapping area is less than the second threshold, and the difference between the brightness value and the reference brightness is less than the third threshold; The boundary transition standard is that the hue gradient change rate of adjacent pixels in the overlapping area is less than a fourth threshold, and the saturation gradient change rate is less than a fifth threshold.

7. The multi-projection image stitching method based on geometric correction according to claim 1, characterized in that: After generating continuous splicing images, it also includes: Real-time monitoring of the working status of each projection device and ambient lighting parameters; The projection angle, resolution and fusion weight in the correction parameter set are dynamically adjusted according to the monitoring results to adapt to environmental changes.

8. A multi-projection image splicing system based on geometric correction, characterized in that: include: A data acquisition module, configured to acquire initial projection parameters of a plurality of projection devices and raw image data corresponding to the projection devices, wherein the raw image data includes image content of overlapping areas and is collected according to a preset projection direction; A geometric correction module, configured to perform geometric correction processing on the original picture data, extract geometric distortion feature points between each picture, and generate a mapping relationship matrix according to the spatial distribution of the geometric distortion feature points; a parameter adjustment module, configured to divide the screen area of ​​each projection device into projection boundaries based on the mapping relationship matrix, and adjust the projection parameters of each projection device according to the division result to form a correction parameter set; A spatial registration module, configured to perform spatial registration on the adjusted image data of each projection device according to the correction parameter set to generate registered multi-channel image data; A fusion processing module, configured to perform overlapping region fusion processing on the multi-channel image data, eliminate light intensity differences and color deviations at the splicing boundaries, and generate a continuous spliced ​​image; The geometric correction processing of the original picture data includes: Calculating initial distortion parameters of each image based on the installation position and lens parameters of the projection device; According to the distribution density of the geometric distortion feature points, points with distortion rates exceeding a threshold on both sides are selected from the overlapping area as distortion demarcation points; Calculating the spatial transformation matrix of the distortion demarcation point based on the initial distortion parameters, and taking the center point of the picture as a reference transformation point; interpolating the distortion data according to the spatial transformation matrix, the reference transformation point, and the position of the distortion demarcation point to generate geometrically corrected picture data; Interpolating the distortion data according to the spatial transformation matrix, the reference transformation point, and the position of the distortion demarcation point includes: using the area between the reference transformation point and the distortion demarcation point as a main correction area, and other areas as auxiliary correction areas; Constructing a picture distortion compensation function based on the initial distortion parameters; the picture distortion compensation function is a correspondence between pixel positions in the projected picture and actual spatial positions; interpolating the main correction area according to the image distortion compensation function to generate first correction coordinates, and interpolating the auxiliary correction area to generate second correction coordinates; In combination with the distortion feature point distribution of the second correction coordinates, the second correction coordinates are corrected using a distortion gradient function and the image distortion compensation function to generate third correction coordinates; The first correction coordinates and the third correction coordinates are combined to form geometrically corrected picture data.

Images