Spherical screen projection three-dimensional image correction and optimization system

Abstract

The invention discloses a spherical screen projection three-dimensional image correction and optimization system, and belongs to the technical field of spherical screen projection image correction, in the spherical screen projection three-dimensional image correction and optimization system, a three-dimensional reconstruction module serves as a core component of the spherical screen projection three-dimensional image correction and optimization system, and a three-dimensional matching module in the three-dimensional reconstruction module is matched with the three-dimensional reconstruction module through an advanced SGBM algorithm. Depth information in a multi-angle image can be accurately extracted, and a detailed three-dimensional scene is constructed. The accurate depth information not only provides a reliable data basis for subsequent image correction, but also greatly improves the stereoscopic impression and reality sense of the three-dimensional image. And the curved surface reconstruction module adopts a Poisson curved surface reconstruction algorithm according to the special geometric shape of the spherical screen, so that the three-dimensional structure of the spherical screen can be accurately restored, and the geometric distortion of the image can be effectively eliminated. The accurate curved surface reconstruction not only ensures the correct projection of the image on the spherical screen, but also improves the visual effect of the image, so that audiences can obtain more immersive film watching experience.

Description

Technical Field

[0001] The invention belongs to the technical field of spherical screen projection image correction, and in particular relates to a spherical screen projection three-dimensional image correction and optimization system. Background Art

[0002] Spherical screen projection, also known as dome projection or dome projection, is a projection technology that projects images onto a spherical screen. It is widely used in planetariums, science and technology museums, theme parks, simulated flight training and other fields to provide audiences with an immersive visual experience. Spherical screen projection usually uses a combination of multiple high-resolution projectors to seamlessly splice multiple images together through edge fusion technology to form a continuous, huge image covering the entire sphere. With the rapid development of digital projection technology, spherical screen projection, as an emerging display method, has been widely used in science popularization education, film and television entertainment and other fields. With its unique immersive experience, spherical screen projection can bring strong visual impact and immersive experience to the audience. However, spherical screen projection faces many technical challenges in the implementation process, among which the correction and optimization of three-dimensional images is one of the key links. Existing spherical screen projection three-dimensional image correction and optimization systems usually use traditional image processing methods, such as global histogram equalization and simple edge detection.

[0003] However, the existing correction and optimization systems are difficult to accurately extract the depth information of the image, resulting in poor 3D reconstruction and insufficient stereoscopic and realistic images. Secondly, the special geometric shape of the dome makes it easy for the image to produce geometric distortion during the projection process, and the existing systems often lack effective surface reconstruction technology to eliminate these distortions. Summary of the invention

[0004] The purpose of the present invention is to provide a spherical screen projection three-dimensional image correction and optimization system in order to solve the above-mentioned problems.

[0005] The technical solution adopted by the present invention is as follows: a spherical screen projection three-dimensional image correction and optimization system, characterized in that: the system includes: an image acquisition module, a three-dimensional reconstruction module, an image correction module, a real-time optimization module, a personalized adjustment module, a system integration and control module, and an auxiliary function module;

[0006] The three-dimensional reconstruction module is internally provided with a stereo matching module and a curved surface reconstruction module;

[0007] The real-time optimization module is internally provided with a dynamic adjustment module and an edge enhancement module;

[0008] The image acquisition module, as the front end of the system, is responsible for synchronously capturing the dome screen image through a multi-angle camera array to provide raw data for subsequent processing. These image data are first transmitted to the 3D reconstruction module, in which the stereo matching algorithm submodule performs feature extraction, matching and parallax calculation on the image to extract depth information; at the same time, the surface reconstruction technology submodule performs parametric modeling, mesh generation and depth information fusion according to the special shape of the dome screen, and finally reconstructs the 3D scene of the dome screen.

[0009] The image correction module uses a geometric correction algorithm and a color correction algorithm to correct geometric distortion and unify the color of the image to ensure the accuracy and consistency of the image. The corrected image then enters the real-time optimization module.

[0010] The real-time optimization module automatically optimizes image parameters according to environmental changes through a real-time dynamic adjustment algorithm to maintain the best viewing effect.

[0011] The personalized adjustment module allows the user to further adjust the image according to personal preferences, and provides a variety of preset modes for the user to choose.

[0012] The system integration and control module is responsible for the scheduling and management of the overall system, ensuring the smooth flow of data and collaborative work between modules, and realizing automatic calibration and fault diagnosis of the system.

[0013] The auxiliary function module provides additional support functions, background user interface design, data storage and backup, and enhances the usability and reliability of the system.

[0014] In a preferred embodiment, the image acquisition module is the front end of the entire system, responsible for capturing the original image data of the ball screen projection with high precision and high resolution. The module uses a multi-angle camera array to ensure that the ball screen image is fully captured from different perspectives. All cameras are synchronized through a unified trigger signal to ensure the time consistency of the image. The camera parameters can be automatically adjusted according to the ambient light to ensure image quality. The collected image data is transmitted to the subsequent processing module in real time through a high-speed data transmission interface to provide basic data for three-dimensional reconstruction and image correction.

[0015] In a preferred embodiment, the stereo matching module uses the SGBM algorithm to approximate global energy minimization through one-dimensional path optimization in multiple directions, thereby obtaining a more accurate disparity map, which specifically includes the following steps:

[0016] S1: Cost calculation, for each pixel, calculate its grayscale difference or color difference in the left and right images as the initial cost.

[0017] The formula is: C(p,d)=|IL(p)-IR(pd)|;

[0018] Where C(p,d) is the cost of pixel p under disparity d.

[0019] I L (p): Gray value of pixel p in the left image.

[0020] I R (pd): The grayscale value of the pixel in the right image with a disparity of d corresponding to the pixel p in the left image.

[0021] S2: Cost aggregation, the SGBM algorithm accumulates costs by optimizing one-dimensional paths in 16 directions.

[0022] The calculation formula is:

[0023] S(p,d)=C(p,d)+min(S(p-Δp,d),S(p-Δp,d-1)+P1,S(p-Δp,d+1)+P1,max(S(p-Δp,d-1),S(p-Δp,d+1))+P2);

[0024] Where S(p,d) represents the cumulative cost of pixel p under disparity d.

[0025] Δp represents the adjacent pixel points on the path.

[0026] P1 and P2 represent penalty parameters, which are used to control parallax smoothness and parallax jump.

[0027] S3: Disparity calculation, select the minimum cost disparity: For each pixel, select the disparity with the minimum cumulative cost as the final disparity;

[0028] The calculation formula is: D(p) = arg min d S(p,d), where D(p) represents the final disparity of pixel p. d represents the possible disparity value;

[0029] S4: Post-processing: further optimize the disparity map through filtering and sub-pixel interpolation post-processing technology to improve the disparity accuracy and smoothness.

[0030] In a preferred embodiment, the surface reconstruction module uses a Poisson surface reconstruction algorithm to convert scattered point cloud data into a continuous surface model by solving the Poisson equation. This method can effectively process noise and holes and generate a smooth surface. The specific method includes the following steps:

[0031] S1: Point cloud preprocessing: Denoising: Remove noise points in the point cloud to improve data quality. Resampling: Resample the point cloud to make the point distribution more uniform.

[0032] S2: Normal vector estimation, local plane fitting: Fit a local plane near each point and calculate the normal vector of the point. The calculation formula is:

[0033] n i = arg min n ∑ j∈N(i) (p j -p i )·n,

[0034] where ni is the normal vector of point i.

[0035] N(i) is the neighborhood point set of point i.

[0036] pj, pi are the position vectors of point j and point i.

[0037] S3: Poisson equation is constructed to calculate the gradient field according to the normal vector of the point cloud. The formula is:

[0038] in Represents the gradient of the scalar field \phiφ. g is the gradient field obtained by transforming the normal vector;

[0039] S4: Poisson equation solution: discretize the Poisson equation and convert it into a system of linear equations.

[0040] formula: where Δφ represents the Laplace operator acting on the scalar field φ. Represents the divergence of the gradient field g.

[0041] Then solve the linear equations: Use the iterative method to solve the linear equations and get the scalar field φ

[0042] S5: Surface extraction, the final surface model is obtained by extracting the isosurface of the scalar field \phiφ. The formula is: S = {x|φ(x) = φ0}, where S represents the reconstructed surface. x represents a point in space. φ0 represents the value of the isosurface, which is taken as 0.

[0043] In a preferred embodiment, the image correction module uses a geometric correction method based on a polynomial distortion model. First, the distortion parameters of each camera are determined through a calibration process, and then the polynomial distortion model is constructed using these parameters. Each image is subjected to inverse distortion correction to restore its original geometric shape. At the same time, a color correction algorithm is used to unify the color differences between different cameras to ensure the color consistency of the image. The corrected image is more consistent with the actual scene, providing accurate basic data for subsequent three-dimensional reconstruction and optimization.

[0044] In a preferred embodiment, the real-time optimization module uses an adaptive histogram equalization algorithm to adjust the brightness distribution of the image so that the image obtains better contrast in different areas; the specific contents include: image segmentation: dividing the image into multiple sub-blocks, and each sub-block is independently subjected to histogram equalization, specifically including the following steps:

[0045] S1: Local histogram calculation: Calculate the local histogram of each sub-block to represent the brightness distribution of the area. The calculation formula is:

[0046] Where H(i,j) represents the histogram of the sub-block at position (i,j)(i,j); b(k) represents the brightness value of pixel k; δ represents the Dirac function, which is 1 when the expression is true, otherwise it is 0;

[0047] S2: Histogram equalization: Equalize the local histogram of each sub-block to enhance the local contrast. The calculation formula is:

[0048] Where T(rk) represents the equalized brightness value; rk represents the original brightness value; N represents the total number of pixels in the sub-block;

[0049] S3: Overlapping area processing: In order to reduce the boundary effect between blocks, the overlapping areas of adjacent sub-blocks are weighted averaged. The calculation formula is:

[0050] Where O(i,j) represents the output brightness value of the overlapping area; TA(i,j) and TB(i,j) represent the equalized brightness values ​​of adjacent sub-blocks A and B at positions (i,j) and (i,j).

[0051] In a preferred embodiment, the real-time optimization module uses the Canny edge detection algorithm to accurately identify the edges in the image through non-maximum suppression and double threshold detection, specifically including the following steps:

[0052] S1: Gaussian filtering, Gaussian filtering is performed on the image to reduce the impact of noise. The calculation formula is:

[0053] Where G(i,j) represents the pixel value of the filtered image; I(i,j) represents the pixel value of the original image; H(k,l) represents the Gaussian filter kernel;

[0054] S2: Gradient calculation, calculate the gradient magnitude and direction of the image for edge detection; the gradient magnitude calculation formula is: The gradient direction calculation formula is: where Gx and Gy are the gradients in the x and y directions respectively; M(i,j) represents the gradient magnitude; θ(i,j) represents the gradient direction;

[0055] S3: Non-maximum suppression. In the gradient direction, only the local maximum is retained, and non-edge points are suppressed; if M(i,j) is not the local maximum in the gradient direction, it is set to 0.

[0056] S4: Double-threshold detection. Set a high threshold and a low threshold, and determine the final edge through the hysteresis threshold method; if M(i,j) ≥ Th, it is marked as a strong edge; if Tl ≤ M(i,j) < Th, it is marked as a weak edge; otherwise, it is marked as a non-edge, where Th represents the high threshold; Tl represents the low threshold.

[0057] S5: Edge tracking. Through edge tracking, the disconnected edge points are connected to form a complete edge.

[0058] In a preferred embodiment, the personalized adjustment module provides adjustable parameters, including brightness, contrast, and color saturation. The user can adjust these parameters in real time through an intuitive interface and preview the adjustment effect. In addition, the module also supports the user to save the personalized settings for quick application next time. Through personalized adjustment, the user can obtain satisfactory image effects according to different scenarios and preferences, improving the comfort and satisfaction of viewing.

[0059] In a preferred embodiment, the system integration and control module is based on the task scheduling method of a real-time operating system, uses a real-time operating system for task management, divides the image acquisition, processing, and optimization tasks into different priorities, and schedules them according to the real-time requirements. At the same time, the module exchanges data and transmits instructions with each sub-module through a communication interface to achieve the collaborative work of the system. In addition, the system integration and control module is also responsible for the automatic calibration, fault diagnosis, and recovery of the system to ensure the stability and reliability of the system.

[0060] In a preferred embodiment, the auxiliary function module stores and backs up the key data during the processing. Redundant storage technology is adopted to ensure the security and recoverability of the data. At the same time, the module also provides a user interface design, and through an intuitive operation interface, it is convenient for the user to set parameters, preview effects, and control the system. In addition, the auxiliary function module also supports remote monitoring and maintenance, allowing technicians to remotely diagnose and upgrade the system through the network, further improving the maintainability and scalability of the system.

[0061] In summary, due to the adoption of the above technical solutions, the beneficial effects of the present invention are:

[0062] 1. In the present invention, the three-dimensional reconstruction module is the core component of the three-dimensional image correction and optimization system for ball screen projection, and its internal stereo matching module and surface reconstruction module have significant beneficial effects on the entire system. The stereo matching module can accurately extract the depth information in the multi-angle image through the advanced SGBM algorithm to construct a detailed three-dimensional scene. This precise depth information not only provides a reliable data basis for subsequent image correction, but also greatly improves the stereoscopic sense and realism of the three-dimensional image. The surface reconstruction module adopts the Poisson surface reconstruction algorithm for the special geometric shape of the ball screen, which can accurately restore the three-dimensional structure of the ball screen and effectively eliminate the geometric distortion of the image. This accurate surface reconstruction not only ensures the correct projection of the image on the ball screen, but also improves the visual effect of the image, so that the audience can obtain a more immersive viewing experience. Therefore, the effective work of the three-dimensional reconstruction module ensures the accuracy and efficiency of the entire system in three-dimensional image processing, and lays a solid foundation for the high-quality presentation of ball screen projection.

[0063] 2. In the present invention, the real-time optimization module plays a vital role in the spherical screen projection three-dimensional image correction and optimization system, and its internal dynamic adjustment module and edge enhancement module bring significant beneficial effects to the system. The dynamic adjustment module can adjust the image's brightness, contrast and other parameters in real time through algorithms such as adaptive histogram equalization to adapt to different ambient light and projection content. This dynamic adjustment not only improves the clarity and layering of the image, but also ensures that the audience can get the best viewing effect under any conditions. The edge enhancement module uses algorithms such as Canny edge detection to accurately detect and enhance the edges of the image. This edge enhancement processing not only makes the details of the image more prominent, but also greatly improves the sharpness and clarity of the image, allowing the audience to better perceive the subtle changes in the image. These functions of the real-time optimization module work together to ensure the real-time optimization and high-quality presentation of the spherical screen projection image, greatly improving the audience's viewing experience and satisfaction. Therefore, the real-time optimization module is the key link in the entire system to achieve efficient and accurate image processing, and provides a strong guarantee for the excellent performance of the spherical screen projection. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 is the overall system block diagram of the present invention;

[0065] Figure 2 This is a system block diagram of a three-dimensional reconstruction module in the present invention;

[0066] Figure 3 This is a system block diagram of the real-time optimization module in the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0068] Reference Figure 1-3 ,

[0069] Example:

[0070] A spherical screen projection three-dimensional image correction and optimization system, the system includes: an image acquisition module, a three-dimensional reconstruction module, an image correction module, a real-time optimization module, a personalized adjustment module, a system integration and control module, and an auxiliary function module;

[0071] The 3D reconstruction module is internally provided with a stereo matching module and a surface reconstruction module;

[0072] The real-time optimization module is internally provided with a dynamic adjustment module and an edge enhancement module;

[0073] As the front end of the system, the image acquisition module is responsible for synchronously capturing the dome screen images through a multi-angle camera array to provide raw data for subsequent processing. These image data are first transmitted to the 3D reconstruction module, where the stereo matching algorithm submodule performs feature extraction, matching and parallax calculation on the image to extract depth information; at the same time, the surface reconstruction technology submodule performs parametric modeling, mesh generation and depth information fusion according to the special shape of the dome screen, and finally reconstructs the 3D scene of the dome screen.

[0074] The image correction module uses geometric correction algorithms and color correction algorithms to correct geometric distortion and unify colors of images to ensure the accuracy and consistency of images. The corrected images then enter the real-time optimization module.

[0075] The real-time optimization module automatically optimizes image parameters according to environmental changes through real-time dynamic adjustment algorithms to maintain the best viewing effect.

[0076] The personalized adjustment module allows users to further adjust the image according to personal preferences, and provides a variety of preset modes for users to choose from.

[0077] The system integration and control module is responsible for the scheduling and management of the overall system, ensuring data flow and collaborative work between modules, and realizing automatic calibration and fault diagnosis of the system.

[0078] The auxiliary function module provides additional support functions, background user interface design, data storage and backup, etc., to enhance the usability and reliability of the system.

[0079] The image acquisition module is the front end of the entire system, responsible for capturing the original image data of the ball screen projection with high precision and high resolution. The module uses a multi-angle camera array to ensure that the ball screen image is fully captured from different perspectives. All cameras shoot synchronously through a unified trigger signal to ensure the temporal consistency of the image. The camera parameters (such as exposure time, gain, etc.) can be automatically adjusted according to the ambient light to ensure image quality. The collected image data is transmitted to the subsequent processing module in real time through a high-speed data transmission interface to provide basic data for three-dimensional reconstruction, image correction, etc.

[0080] The stereo matching module uses the SGBM algorithm to approximate global energy minimization through one-dimensional path optimization in multiple directions to obtain a more accurate disparity map. Specifically, it includes the following steps:

[0081] S1: Cost calculation, for each pixel, calculate its grayscale difference or color difference in the left and right images as the initial cost.

[0082] The formula is: C(p,d)=|IL(p)-IR(pd)|;

[0083] Where C(p,d) is the cost of pixel p under disparity d.

[0084] I L (p): Gray value of pixel p in the left image.

[0085] I R (pd): The grayscale value of the pixel in the right image with a disparity of d corresponding to the pixel p in the left image.

[0086] S2: Cost aggregation, the SGBM algorithm accumulates costs by optimizing one-dimensional paths in 16 directions.

[0087] The calculation formula is:

[0088] S(p,d)=C(p,d)+min(S(p-Δp,d),S(p-Δp,d-1)+P1,S(p-Δp,d+1)+P1,max(S(p-Δp,d-1),S(p-Δp,d+1))+P2);

[0089] Where S(p,d) represents the cumulative cost of pixel p under disparity d.

[0090] Δp represents the adjacent pixel points on the path.

[0091] P1 and P2 represent penalty parameters, which are used to control parallax smoothness and parallax jump.

[0092] S3: Disparity calculation, select the minimum cost disparity: For each pixel, select the disparity with the minimum cumulative cost as the final disparity;

[0093] The calculation formula is: D(p) = arg min d S(p,d), where D(p) represents the final disparity of pixel p. d represents the possible disparity value;

[0094] S4: Post-processing: further optimize the disparity map through post-processing techniques such as filtering and sub-pixel interpolation to improve disparity accuracy and smoothness.

[0095] The surface reconstruction module uses the Poisson surface reconstruction algorithm to convert scattered point cloud data into a continuous surface model by solving the Poisson equation. This method can effectively handle noise and holes and generate a smooth surface. The specific method includes the following steps:

[0096] S1: Point cloud preprocessing: Denoising: Remove noise points in the point cloud to improve data quality. Resampling: Resample the point cloud to make the point distribution more uniform.

[0097] S2: Normal vector estimation, local plane fitting: Fit a local plane near each point and calculate the normal vector of the point. The calculation formula is:

[0098] n i = arg min n ∑ j∈N(i) (p j -p i )·n,

[0099] where ni is the normal vector of point i.

[0100] N(i) is the neighborhood point set of point i.

[0101] pj, pi are the position vectors of point j and point i.

[0102] S3: Poisson equation is constructed to calculate the gradient field according to the normal vector of the point cloud. The formula is:

[0103] in Represents the gradient of the scalar field \phiφ. g is the gradient field obtained by transforming the normal vector;

[0104] S4: Poisson equation solution: discretize the Poisson equation and convert it into a system of linear equations.

[0105] formula: where Δφ represents the Laplace operator acting on the scalar field φ. Represents the divergence of the gradient field g.

[0106] Then solve the linear equations: Use an iterative method (such as the conjugate gradient method) to solve the linear equations and obtain the scalar field φ

[0107] S5: Surface extraction, the final surface model is obtained by extracting the isosurface of the scalar field \phiφ. The formula is: S = {x|φ(x) = φ0}, where S represents the reconstructed surface. x represents a point in space. φ0 represents the value of the isosurface, which is taken as 0.

[0108] The image correction module uses a geometric correction method based on a polynomial distortion model. First, the distortion parameters of each camera are determined through a calibration process, and then a polynomial distortion model is constructed using these parameters. Each image is subjected to inverse distortion correction to restore its original geometry. At the same time, a color correction algorithm (such as white balance adjustment) is used to unify the color differences between different cameras to ensure the color consistency of the image. The corrected image is more in line with the actual scene, providing accurate basic data for subsequent 3D reconstruction and optimization.

[0109] The real-time optimization module uses an adaptive histogram equalization algorithm to adjust the brightness distribution of the image so that the image can obtain better contrast in different areas; the specific contents include: image segmentation: dividing the image into multiple sub-blocks, and each sub-block is independently subjected to histogram equalization, specifically including the following steps:

[0110] S1: Local histogram calculation: Calculate the local histogram of each sub-block to represent the brightness distribution of the area. The calculation formula is:

[0111] Where H(i,j) represents the histogram of the sub-block at position (i,j)(i,j); b(k) represents the brightness value of pixel k; δ represents the Dirac function, which is 1 when the expression is true, otherwise it is 0;

[0112] S2: Histogram equalization: Equalize the local histogram of each sub-block to enhance the local contrast. The calculation formula is:

[0113] Where T(rk) represents the equalized brightness value; rk represents the original brightness value; N represents the total number of pixels in the sub-block;

[0114] S3: Overlapping area processing: In order to reduce the boundary effect between blocks, the overlapping areas of adjacent sub-blocks are weighted averaged. The calculation formula is:

[0115] Where O(i,j) represents the output brightness value of the overlapping area; TA(i,j) and TB(i,j) represent the equalized brightness values ​​of adjacent sub-blocks A and B at positions (i,j) and (i,j).

[0116] The real-time optimization module uses the Canny edge detection algorithm to accurately identify edges in the image through non-maximum suppression and double-threshold detection, specifically including the following steps:

[0117] S1: Gaussian filtering, performing Gaussian filtering on the image to reduce the influence of noise, and the calculation formula is:

[0118] where G(i,j) represents the pixel value of the filtered image; I(i,j) represents the pixel value of the original image; H(k,l) represents the Gaussian filter kernel;

[0119] S2: Gradient calculation, calculating the gradient magnitude and direction of the image for edge detection; the calculation formula for the gradient magnitude is: The calculation formula for the gradient direction is: where Gx and Gy are the gradients in the x and y directions respectively; M(i,j) represents the gradient magnitude; θ(i,j) represents the gradient direction;

[0120] S3: Non-maximum suppression, only retaining the local maximum in the gradient direction and suppressing non-edge points; if M(i,j) is not the local maximum in the gradient direction, it is set to 0;

[0121] S4: Double-threshold detection, setting a high threshold and a low threshold, and determining the final edge through the hysteresis threshold method; if M(i,j)≥Th, it is marked as a strong edge; if Tl≤M(i,j)<Th, it is marked as a weak edge; otherwise, it is marked as a non-edge, where Th represents the high threshold; Tl represents the low threshold;

[0122] S5: Edge tracking, connecting the disconnected edge points through edge tracking to form a complete edge.

[0123] The personalized adjustment module provides a series of adjustable parameters, such as brightness, contrast, color saturation, etc. Users can adjust these parameters in real time through an intuitive interface and preview the adjustment effects. In addition, the module supports users to save personalized settings for quick application next time. Through personalized adjustment, users can obtain satisfactory image effects according to different scenarios and preferences, improving the comfort and satisfaction of viewing.

[0124] The system integration and control module is based on the task scheduling method of a real-time operating system, uses a real-time operating system (RTOS) for task management, divides tasks such as image acquisition, processing, and optimization into different priorities, and schedules them according to real-time requirements. At the same time, the module exchanges data and transfers instructions with each sub-module through a communication interface to achieve the collaborative work of the system. In addition, the system integration and control module is also responsible for the automatic calibration, fault diagnosis, and recovery of the system to ensure the stability and reliability of the system.

[0125] The auxiliary function module stores and backs up key data (such as calibration parameters, user settings, etc.) during the processing process. Redundant storage technology is used to ensure data security and recoverability. At the same time, the module also provides user interface design, which facilitates users to set parameters, preview effects and control the system through an intuitive operation interface. In addition, the auxiliary function module also supports remote monitoring and maintenance, allowing technicians to remotely diagnose and upgrade the system through the network, further improving the maintainability and scalability of the system.

[0126] In the present invention, the three-dimensional reconstruction module is the core component of the three-dimensional image correction and optimization system for ball screen projection, and its internal stereo matching module and surface reconstruction module have significant beneficial effects on the entire system. The stereo matching module can accurately extract the depth information in the multi-angle image through the advanced SGBM algorithm to construct a detailed three-dimensional scene. This precise depth information not only provides a reliable data basis for subsequent image correction, but also greatly improves the stereoscopic sense and realism of the three-dimensional image. The surface reconstruction module adopts the Poisson surface reconstruction algorithm for the special geometric shape of the ball screen, which can accurately restore the three-dimensional structure of the ball screen and effectively eliminate the geometric distortion of the image. This accurate surface reconstruction not only ensures the correct projection of the image on the ball screen, but also improves the visual effect of the image, so that the audience can obtain a more immersive viewing experience. Therefore, the effective work of the three-dimensional reconstruction module ensures the accuracy and efficiency of the entire system in three-dimensional image processing, and lays a solid foundation for the high-quality presentation of ball screen projection.

[0127] In the present invention, the real-time optimization module plays a vital role in the ball screen projection three-dimensional image correction and optimization system, and its internal dynamic adjustment module and edge enhancement module bring significant beneficial effects to the system. The dynamic adjustment module can adjust the image parameters such as brightness and contrast in real time through algorithms such as adaptive histogram equalization to adapt to different ambient light and projection content. This dynamic adjustment not only improves the clarity and layering of the image, but also ensures that the audience can obtain the best viewing effect under any conditions. The edge enhancement module uses algorithms such as Canny edge detection to accurately detect and enhance the edges of the image. This edge enhancement processing not only makes the details of the image more prominent, but also greatly improves the sharpness and clarity of the image, so that the audience can better perceive the subtle changes in the image. These functions of the real-time optimization module work together to ensure the real-time optimization and high-quality presentation of the ball screen projection image, greatly improving the audience's viewing experience and satisfaction. Therefore, the real-time optimization module is the key link for the entire system to achieve efficient and accurate image processing, and provides a strong guarantee for the excellent performance of the ball screen projection.

[0128] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0129] The above description enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A spherical screen projection three-dimensional image correction and optimization system, characterized in that: The system includes: an image acquisition module, a three-dimensional reconstruction module, an image correction module, a real-time optimization module, a personalized adjustment module, a system integration and control module, and an auxiliary function module; The three-dimensional reconstruction module is internally provided with a stereo matching module and a curved surface reconstruction module; The real-time optimization module is internally provided with a dynamic adjustment module and an edge enhancement module; The image acquisition module, as the front end of the system, is responsible for synchronously capturing the spherical screen images through a multi-angle camera array to provide raw data for subsequent processing; these image data are first transmitted to the 3D reconstruction module, in which the stereo matching algorithm submodule performs feature extraction, matching and parallax calculation on the image to extract depth information; at the same time, the surface reconstruction technology submodule performs parameterized modeling, mesh generation and depth information fusion according to the special shape of the spherical screen, and finally reconstructs the 3D scene of the spherical screen; The image correction module uses a geometric correction algorithm and a color correction algorithm to correct geometric distortion and unify the color of the image to ensure the accuracy and consistency of the image; the corrected image then enters the real-time optimization module; The real-time optimization module automatically optimizes image parameters according to environmental changes through a real-time dynamic adjustment algorithm to maintain the best viewing effect; The personalized adjustment module allows the user to further adjust the image according to personal preferences, and provides a variety of preset modes for the user to choose; The system integration and control module is responsible for the scheduling and management of the overall system, ensuring data flow and collaborative work between modules, and realizing automatic calibration and fault diagnosis of the system; The auxiliary function module provides additional support functions, including background user interface design, data storage and backup.

2. A spherical screen projection three-dimensional image correction and optimization system as claimed in claim 1, characterized in that: The image acquisition module is the front end of the entire system, responsible for capturing the original image data of the ball screen projection with high precision and high resolution; the module adopts a multi-angle camera array to ensure that the ball screen image is fully captured from different viewing angles; All cameras shoot synchronously through a unified trigger signal to ensure the time consistency of the image; the camera parameters can be automatically adjusted according to the ambient light to ensure image quality; the collected image data is transmitted to the subsequent processing module in real time through a high-speed data transmission interface, providing basic data for 3D reconstruction and image correction.

3. The spherical screen projection three-dimensional image correction and optimization system according to claim 1, characterized in that: The stereo matching module uses the SGBM algorithm to approximate global energy minimization through one-dimensional path optimization in multiple directions, thereby obtaining a more accurate disparity map, which specifically includes the following steps: S1: Cost calculation: for each pixel, the grayscale difference or color difference between the left and right images is calculated as the initial cost; The formula is: C(p,d)=|IL(p)-IR(pd)|; Where C(p,d): the cost of pixel p under disparity d; I L (p): gray value of pixel p in the left image; I R (pd): the gray value of the pixel in the right image with a disparity of d corresponding to the pixel p in the left image; S2: Cost aggregation, the SGBM algorithm accumulates costs by optimizing one-dimensional paths in 16 directions. The calculation formula is: S(p,d)=C(p,d)+min(S(p-Δp,d),S(p-Δp,d-1)+P1,S(p-Δp,d+1)+P1,max(S(p-Δp,d-1),S(p-Δp,d+1))+P2); Where S(p,d) represents the cumulative cost of pixel p under disparity d; Δp represents the adjacent pixel points on the path; P1 and P2 represent penalty parameters, which are used to control parallax smoothness and parallax jump; S3: Disparity calculation, select the minimum cost disparity: For each pixel, select the disparity with the minimum cumulative cost as the final disparity; The calculation formula is: D(p) = arg min d S(p,d), where D(p) represents the final disparity of pixel p; d represents the possible disparity value; S4: Post-processing: further optimize the disparity map through filtering and sub-pixel interpolation post-processing technology to improve the disparity accuracy and smoothness.

4. The spherical screen projection three-dimensional image correction and optimization system according to claim 1, characterized in that: The surface reconstruction module uses the Poisson surface reconstruction algorithm to convert scattered point cloud data into a continuous surface model by solving the Poisson equation; this method can effectively process noise and holes and generate a smooth surface; the specific method includes the following steps: S1: Point cloud preprocessing: Denoising: remove noise points in the point cloud to improve data quality; Resampling: resample the point cloud to make the point distribution more uniform; S2: Normal vector estimation, local plane fitting: Fit a local plane near each point and calculate the normal vector of the point. The calculation formula is: n i =my anger n ∑ j∈N(i) (p j -p i )·n, Where ni is the normal vector of point i; N(i) is the neighborhood point set of point i; pj, pi are the position vectors of point j and point i; S3: Poisson equation is constructed to calculate the gradient field according to the normal vector of the point cloud. The formula is: in represents the gradient of the scalar field \phiφ; g is the gradient field obtained by transforming the normal vector; S4: Poisson equation solution: discretize the Poisson equation and convert it into a linear equation system; formula: where Δφ represents the Laplace operator acting on the scalar field φ; represents the divergence of the gradient field g; Then solve the linear equations: Use the iterative method to solve the linear equations and get the scalar field φ S5: Surface extraction, the final surface model is obtained by extracting the isosurface of the scalar field \phiφ; the formula is: S = {x|φ(x) = φ0}, where S represents the reconstructed surface; x represents a point in space; φ0 represents the value of the isosurface, which is taken as 0.

5. The spherical screen projection three-dimensional image correction and optimization system according to claim 1, characterized in that: The image correction module uses a geometric correction method based on a polynomial distortion model. First, the distortion parameters of each camera are determined through a calibration process, and then a polynomial distortion model is constructed using these parameters. Inverse distortion correction is performed on each image to restore its original geometric shape. At the same time, a color correction algorithm is used to unify the color differences between different cameras to ensure the color consistency of the image. The corrected image is more in line with the actual scene and provides accurate basic data for subsequent three-dimensional reconstruction and optimization.

6. A spherical screen projection three-dimensional image correction and optimization system as claimed in claim 1, characterized in that: The real-time optimization module uses an adaptive histogram equalization algorithm to adjust the brightness distribution of the image so that the image can obtain better contrast in different areas; the specific contents include: image segmentation: dividing the image into multiple sub-blocks, and each sub-block is independently subjected to histogram equalization, specifically including the following steps: S1: Local histogram calculation: Calculate the local histogram of each sub-block to represent the brightness distribution of the area. The calculation formula is: Where H(i,j) represents the histogram of the sub-block at position (i,j)(i,j); b(k) represents the brightness value of pixel k; δ represents the Dirac function, which is 1 when the expression is true, otherwise it is 0; S2: Histogram equalization: Equalize the local histogram of each sub-block to enhance the local contrast. The calculation formula is: Where T(rk) represents the equalized brightness value; rk represents the original brightness value; N represents the total number of pixels in the sub-block; S3: Overlapping area processing: In order to reduce the boundary effect between blocks, the overlapping areas of adjacent sub-blocks are weighted averaged. The calculation formula is: Where O(i,j) represents the output brightness value of the overlapping area; TA(i,j) and TB(i,j) represent the equalized brightness values ​​of adjacent sub-blocks A and B at positions (i,j) and (i,j).

7. A spherical screen projection three-dimensional image correction and optimization system as claimed in claim 1, characterized in that: The real-time optimization module uses the Canny edge detection algorithm to accurately identify edges in the image through non-maximum suppression and double-threshold detection, which specifically includes the following steps: S1: Gaussian filtering, Gaussian filtering is performed on the image to reduce the impact of noise. The calculation formula is: Among them, G(i,j) represents the pixel value of the filtered image; I(i,j) represents the pixel value of the original image; H(k,l) represents the Gaussian filter kernel; S2: Gradient calculation, calculate the gradient magnitude and direction of the image for edge detection; the gradient magnitude calculation formula is: The gradient direction calculation formula is: Where Gx and Gy are the gradients in the x and y directions respectively; M(i,j) represents the gradient amplitude; θ(i,j) represents the gradient direction; S3: Non-maximum suppression. In the gradient direction, only the local maximum value is retained, and non-edge points are suppressed; if M(i,j) is not the local maximum value in the gradient direction, it is set to 0; S4: Double-threshold detection. Set a high threshold and a low threshold, and determine the final edge through the hysteresis threshold method; if M(i,j)≥Th, it is marked as a strong edge; if Tl≤M(i,j)<Th, it is marked as a weak edge; otherwise, it is marked as a non-edge, where Th represents the high threshold; Tl represents the low threshold; S5: Edge tracking. Through edge tracking, the disconnected edge points are connected to form a complete edge.

8. The spherical screen projection three-dimensional image correction and optimization system according to claim 1, characterized in that: The personalized adjustment module provides adjustable parameters, including brightness, contrast, and color saturation; users can adjust these parameters in real time through an intuitive interface and preview the adjustment effects.

9. The spherical screen projection three-dimensional image correction and optimization system according to claim 1, characterized in that: The system integration and control module is based on the task scheduling method of a real-time operating system, uses a real-time operating system for task management, divides the image acquisition, processing, and optimization tasks into different priorities, and schedules them according to real-time requirements; at the same time, the module exchanges data and transmits instructions with each sub-module through a communication interface to achieve the collaborative work of the system.

10. The spherical screen projection three-dimensional image correction and optimization system according to claim 1, characterized in that: The auxiliary function module stores and backs up the key data during the processing process; adopts redundant storage technology to ensure the security and recoverability of the data; at the same time, the module also provides a user interface design, and through an intuitive operation interface, it is convenient for users to set parameters, preview effects, and control the system.

Images