A method and system for distortion correction of image projection

Abstract

The present application provides a method and system for distortion correction of image projection, which relates to the technical field of image processing. By constructing a bidirectional distortion detection model, a geometric deformation parameter matrix and a light field attenuation vector of the projection plane are generated, and the geometric deformation parameter matrix is analyzed in terms of regions to obtain curvature features. The priority correction levels are divided based on the curvature features, and then the non-linear compensation factors are determined from the priority correction levels and the light field attenuation vector; an adaptive projection network is constructed based on the distortion-sensitive features and the non-linear compensation factors of the projection image, and the projection image is subjected to inverse coordinate mapping through the adaptive projection network to obtain an initial corrected image; an artifact suppression weight map is determined by the high-frequency detail threshold and the low-frequency smoothing threshold of the initial corrected image; the pixel intensity distribution of the initial corrected image is adjusted according to the artifact suppression weight map. The present application can realize the optimization of the geometric deformation parameters and the compensation of the light field attenuation of image projection, so as to improve the accuracy of distortion correction and the anti-artifact ability.

Description

Technical Field

[0001] The present application relates to the field of image processing technology. More specifically, the present application relates to a method and system for distortion correction of image projection. Background Art

[0002] Image processing technology is one of the core technologies in the field of digital information and is widely used in scenarios such as computer vision, intelligent display, and augmented reality. Among them, projection display technology relies on high-precision image correction algorithms to eliminate geometric distortion and brightness distortion caused by deformation of the projection surface or environmental light interference.

[0003] In the field of distortion correction of image projection, the existing technologies mainly include parametric correction based on geometric calibration, brightness equalization method based on light field attenuation compensation, and coordinate transformation algorithm based on inverse mapping; however, the existing methods are still difficult to adapt to the real-time deformation of dynamic surfaces (such as flexible screens, mobile projection surfaces), resulting in lag in updating correction parameters, and also ignoring the non-linear influence of the projection surface material and curvature on light reflection, which is likely to cause brightness distortion and artifacts in the edge region, resulting in insufficient response of image projection to high-curvature regions and a decrease in local correction accuracy. Therefore, how to optimize the geometric deformation parameters of image projection and compensate for light field attenuation to improve the accuracy of distortion correction and anti-artifact ability is a difficult problem faced by the current industry. Summary of the Invention

[0004] The present application provides a method and system for distortion correction of image projection, which can optimize the geometric deformation parameters of image projection and compensate for light field attenuation to improve the accuracy of distortion correction and anti-artifact ability.

[0005] In a first aspect, the present application provides a method and system for distortion correction of image projection. The distortion correction method includes the following steps:

[0006] Construct a bidirectional distortion detection model based on the spatio-temporal domain, and generate a geometric deformation parameter matrix and a light field attenuation vector of the projection surface based on the bidirectional distortion detection model;

[0007] Perform regional analysis on the geometric deformation parameter matrix to obtain the curvature characteristics of the projection surface, divide the priority correction levels through the curvature characteristics, and then determine the non-linear compensation factor from the priority correction levels and the light field attenuation vector;

[0008] Obtain the distortion-sensitive characteristics of the projection image, construct an adaptive projection network based on the distortion-sensitive characteristics and the non-linear compensation factor, and perform inverse coordinate mapping on the projection image through the adaptive projection network to obtain an initial corrected image;

[0009] Determine an artifact suppression weight map through the high-frequency detail threshold and the low-frequency smoothing threshold of the initial corrected image;

[0010] Adjust the pixel intensity distribution of the initial corrected image according to the artifact suppression weight map, so as to obtain a projection image after distortion correction.

[0011] In this embodiment, a spatio-temporal sequence of a projection plane is obtained by an image acquisition device, and the spatio-temporal sequence is input into a bidirectional long short-term memory network including a spatio-temporal convolutional layer, so as to obtain a bidirectional distortion detection model.

[0012] In this embodiment, generating a geometric deformation parameter matrix and a light field attenuation vector of a projection plane based on the bidirectional distortion detection model specifically includes:

[0013] Perform spatial geometric modeling on the projection plane through the bidirectional distortion detection model to obtain a feature reprojection error;

[0014] Determine the geometric deformation parameter matrix of the projection plane according to the feature reprojection error;

[0015] The bidirectional distortion detection model determines the light field attenuation vector by evaluating the light information amount of the projection plane.

[0016] In this embodiment, performing region analysis on the geometric deformation parameter matrix to obtain the curvature feature of the projection plane specifically includes:

[0017] Divide the geometric deformation parameter matrix into multiple dynamic regions;

[0018] Perform curvature mapping on the geometric deformation parameters in each dynamic region respectively to obtain the curvature parameters of each dynamic region;

[0019] Determine the curvature feature of the projection plane based on all the curvature parameters.

[0020] In this embodiment, dividing the priority correction level through the curvature feature, and then determining the non-linear compensation factor from the priority correction level and the light field attenuation vector specifically includes:

[0021] Perform dynamic priority division on the projection plane based on the gradient distribution characteristics of the curvature feature to obtain the priority correction level;

[0022] Determine the low-curvature compensation weight according to the priority correction level based on the linear coefficient in the light field attenuation vector, so as to obtain a compensation weight matrix;

[0023] Perform non-linear optimization on the compensation weight matrix to obtain the non-linear compensation factor.

[0024] In this embodiment, obtain the distortion-sensitive features of the projection image through a multi-scale convolutional network.

[0025] In this embodiment, constructing an adaptive projection network based on the distortion-sensitive feature and the non-linear compensation factor specifically includes:

[0026] Construct a projection network architecture;

[0027] Convert the non-linear compensation factor into a trainable scaling factor, and then dynamically adjust the weights of each channel in the constructed projection network architecture through the scaling factor to obtain a non-linear activation function;

[0028] Construct an adaptive projection network through the high-frequency components of the distortion-sensitive feature and the non-linear activation function.

[0029] In this embodiment, performing inverse coordinate mapping on the projection image through the adaptive projection network to obtain an initial corrected image specifically includes:

[0030] Input the pixel coordinates of the projection image into the adaptive projection network, and extract the local gradient features of the projection image through the adaptive projection network;

[0031] The adaptive projection network performs inverse coordinate transformation on the projection image through the local gradient features, and then performs bilinear interpolation sampling on the projection image after the inverse coordinate transformation to obtain an initial corrected image.

[0032] In this embodiment, the initial corrected image is a luminance channel image.

[0033] In a second aspect, the present application provides an image projection distortion correction system for performing an image projection distortion correction method. The distortion correction system includes:

[0034] A distortion detection module for constructing a bidirectional distortion detection model based on the spatio-temporal domain, and generating a geometric deformation parameter matrix and a light field attenuation vector of the projection plane based on the bidirectional distortion detection model;

[0035] A compensation factor generation module for performing regional analysis on the geometric deformation parameter matrix to obtain the curvature characteristics of the projection plane, dividing the priority correction levels through the curvature characteristics, and then determining the non-linear compensation factor from the priority correction levels and the light field attenuation vector;

[0036] An inverse mapping module for obtaining the distortion-sensitive features of the projection image, constructing an adaptive projection network based on the distortion-sensitive features and the non-linear compensation factor, and performing inverse coordinate mapping on the projection image through the adaptive projection network to obtain an initial corrected image;

[0037] A threshold calculation module for determining an artifact suppression weight map through the high-frequency detail threshold and the low-frequency smoothing threshold of the initial corrected image;

[0038] An image optimization module, configured to adjust the pixel intensity distribution of the initial corrected image according to the artifact suppression weight map, so as to obtain a projection image after distortion correction.

[0039] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects:

[0040] By constructing a bidirectional distortion detection model based on the spatio-temporal domain, generating a geometric deformation parameter matrix and a light field attenuation vector of the projection plane based on the bidirectional distortion detection model; then performing regional analysis on the geometric deformation parameter matrix to obtain the curvature characteristics of the projection plane, dividing the priority correction levels through the curvature characteristics, and further determining a non-linear compensation factor from the priority correction levels and the light field attenuation vector; by obtaining the distortion-sensitive characteristics of the projection image, constructing an adaptive projection network based on the distortion-sensitive characteristics and the non-linear compensation factor, performing inverse coordinate mapping on the projection image through the adaptive projection network to obtain an initial corrected image; further determining an artifact suppression weight map through the high-frequency detail threshold and the low-frequency smoothing threshold of the initial corrected image; adjusting the pixel intensity distribution of the initial corrected image according to the artifact suppression weight map, so as to obtain a projection image after distortion correction.

[0041] It can be seen that in this application, the optimization of geometric deformation parameters and light field attenuation compensation for image projection can be realized. First, the spatio-temporal convolutional layer and the bidirectional long short-term memory network are used to synchronously capture the time-varying characteristics of the dynamic deformation of the projection plane and the light field attenuation, generating a high-precision geometric deformation parameter matrix and a light field attenuation vector, providing a dynamic parameter basis for geometric correction and light compensation, solving the lag problem that the traditional static model cannot update parameters in real time, and improving the correction accuracy. Among them, the light field attenuation vector quantifies the influence of the projection plane material and curvature on the light intensity through a non-linear light reflection model, avoiding the edge brightness distortion caused by linear compensation, and providing data support for subsequent anti-artifact processing; Secondly, the geometric deformation parameters are optimized specifically to improve the local correction accuracy, and combined with the non-linear coefficient in the light field attenuation vector, a dynamic non-linear compensation factor is generated to differentially compensate for light attenuation according to the curvature and material characteristics of different regions, avoiding overexposure or underexposure problems caused by traditional uniform compensation, and suppressing artifacts from the source; Then, an adaptive projection network is constructed based on the distortion-sensitive features and the non-linear compensation factor, and the compensation factor is converted into a trainable scaling factor to dynamically adjust the network weights, so that the inverse coordinate mapping process can simultaneously adapt to the non-linear relationship between geometric deformation and light field attenuation to improve the mapping accuracy. By guiding bilinear interpolation sampling through local gradient features, the detail blurring caused by traditional global interpolation can be reduced, and high-frequency artifacts can be further suppressed; Finally, an artifact suppression weight map is generated based on the high-frequency detail threshold and the low-frequency smoothing threshold, and the pixel intensity adjustment amplitude is controlled layer by layer, eliminating low-frequency artifacts while retaining details, realizing the closed-loop optimization of the anti-artifact ability, and then dynamically adjusting the pixel distribution through the weight map to balance the brightness consistency after geometric correction, avoiding local overcorrection problems caused by fixed thresholds in traditional methods, and improving the visual quality.

[0042] In summary, the technical solution adopted in this application can realize the optimization of geometric deformation parameters and light field attenuation compensation for image projection to improve the accuracy of distortion correction and the anti-artifact ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0044] Figure 1 is a flowchart of a distortion correction method for image projection provided by the present application;

[0045] Figure 2 is an exemplary flowchart for determining the curvature characteristics of the projection plane provided by the present application;

[0046] Figure 3It is an exemplary flowchart for determining an initial corrected image provided by the present application;

[0047] Figure 4 It is a module structure diagram of a distortion correction system provided by the present application. Detailed implementation manners

[0048] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0049] The embodiments of the present application provide an image projection distortion correction method and system. The core is to construct a bidirectional distortion detection model based on the spatio-temporal domain, and generate a geometric deformation parameter matrix and a light field attenuation vector of the projection plane based on the bidirectional distortion detection model; then perform regional analysis on the geometric deformation parameter matrix to obtain the curvature characteristics of the projection plane, divide the priority correction levels through the curvature characteristics, and further determine the non-linear compensation factor from the priority correction levels and the light field attenuation vector; obtain the distortion-sensitive characteristics of the projection image, construct an adaptive projection network based on the distortion-sensitive characteristics and the non-linear compensation factor, perform inverse coordinate mapping on the projection image through the adaptive projection network to obtain an initial corrected image; and then determine an artifact suppression weight map through the high-frequency detail threshold and the low-frequency smoothing threshold of the initial corrected image; adjust the pixel intensity distribution of the initial corrected image according to the artifact suppression weight map, and further obtain the projection image after distortion correction.

[0050] Embodiment 1. To better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners. Refer to Figure 1 As shown, this figure is an exemplary flowchart of an image projection distortion correction method shown in this embodiment of the present application. The distortion correction method includes the following steps:

[0051] In step S1, construct a bidirectional distortion detection model based on the spatio-temporal domain, and generate a geometric deformation parameter matrix and a light field attenuation vector of the projection plane based on the bidirectional distortion detection model.

[0052] It should be noted that the spatio-temporal sequence of the projection plane is obtained by an image acquisition device, and then the spatio-temporal sequence is input into a bidirectional long short-term memory network containing a spatio-temporal convolutional layer, and then the bidirectional distortion detection model can be specifically implemented in the following manner, that is: First, continuous sequence data of the projection plane in the time and space dimensions are obtained by a multi-frame image acquisition device to form a spatio-temporal sequence input; Second, the spatio-temporal sequence is input into the spatio-temporal convolutional layer, which extracts the geometric features of each region of the projection plane in the spatial domain through a convolutional kernel, and analyzes the dynamic distortion characteristics between adjacent frames in the time domain; Then, the output features of the spatio-temporal convolutional layer are input into the bidirectional long short-term memory network, which captures the bidirectional propagation characteristics of the projection plane distortion in the time dimension through forward and backward time propagation, and fuses the local features of the spatio-temporal convolutional layer with the temporal features of the bidirectional long short-term memory network through spatio-temporal domain residual connection to generate a comprehensive representation of the dynamic distortion features; Finally, in combination with the light field attenuation model, the fused dynamic distortion features are mapped into a geometric deformation parameter matrix of the projection plane, and a light field attenuation vector is output synchronously.

[0053] In this embodiment, the geometric deformation parameter matrix and the light field attenuation vector of the projection plane are generated based on the bidirectional distortion detection model, which can be specifically implemented in the following manner, that is:

[0054] Spatial geometric modeling is performed on the projection plane through the bidirectional distortion detection model to obtain a feature reprojection error;

[0055] The geometric deformation parameter matrix of the projection plane is determined according to the feature reprojection error;

[0056] The bidirectional distortion detection model determines the light field attenuation vector by evaluating the light information amount of the projection plane.

[0057] When specifically implemented, first, the three-dimensional point cloud data of the projection plane is obtained through stereo vision calibration technology, and the iterative closest point algorithm is used to align the ideal projection plane coordinate system with the actual distorted surface coordinate system. The Euclidean distance between the projection point and the reconstructed point is calculated through the OpenCV library, and the average value of the calculated Euclidean distance is used as the feature reprojection error, which can quantify the geometric deformation degree of the projection plane; Then, based on the feature reprojection error, a non-linear least squares optimization algorithm is used to fit the polynomial distortion model, and the parameter weights are iteratively optimized through the least squares optimizer in the SciPy library to generate a geometric deformation parameter matrix (two-dimensional floating-point array) describing the dynamic deformation of the projection plane, which can map the feature reprojection error into quantifiable compensation parameters; Finally, the reflectivity and material properties of the projection plane are obtained through multi-spectral imaging technology. The bidirectional distortion detection model combines the Lambertian illumination model to calculate the light attenuation coefficient of each region, and uses MATLAB to generate a normalized light field attenuation vector, which can quantify the non-linear characteristics of light intensity attenuation and provide data input for light compensation.

[0058] It should be noted that the geometric deformation parameter matrix in this application refers to a two-dimensional compensation weight matrix generated by analyzing the dynamic deformation characteristics of the projection plane. The dimension of this two-dimensional compensation weight matrix corresponds to the grid area divided on the projection plane, and each element represents the deformation compensation intensity of the corresponding area (such as stretching and compression weights). It can be used to quantify the degree of geometric distortion of the projection plane caused by material bending, environmental vibration, etc., provide a dynamic correction basis for subsequent inverse coordinate mapping, thereby optimizing the local deformation error in the high-curvature area and improving the overall geometric correction accuracy. Additionally, the light field attenuation vector refers to a set of regional light attenuation coefficients (one-dimensional array) generated based on the reflectivity of the projection plane material, curvature distribution, and light source distance. Each element in the light field attenuation vector represents the non-linear light intensity attenuation ratio of the corresponding area, which can provide data support for the generation of the light compensation factor, thereby solving the problems of brightness distortion and artifact residue generated by the traditional linear light compensation model in the high-curvature area.

[0059] In step S2, perform regional analysis on the geometric deformation parameter matrix to obtain the curvature characteristics of the projection plane, divide the priority correction levels through the curvature characteristics, and then determine the non-linear compensation factor based on the priority correction levels and the light field attenuation vector.

[0060] Preferably, in this embodiment, refer to Figure 2 As shown, this figure is an exemplary flowchart for determining the curvature characteristics of the projection plane in the embodiment of this application. In this embodiment, performing regional analysis on the geometric deformation parameter matrix to obtain the curvature characteristics of the projection plane can be specifically implemented by the following steps:

[0061] First, in step S21, divide the geometric deformation parameter matrix into multiple dynamic regions;

[0062] Then, in step S22, perform curvature mapping on the geometric deformation parameters in each dynamic region to obtain the curvature parameters of each dynamic region;

[0063] Finally, in step S23, determine the curvature characteristics of the projection plane based on all the curvature parameters.

[0064] In specific implementation, first, calculate the deformation gradient through the Sobel operator, and adaptively divide the matrix into multiple dynamic grid regions according to the deformation gradient. For example, in high-gradient regions (such as high-deformation regions at the edges), dense grid division (such as a 10×10 grid) is adopted, and in low-gradient regions, sparse grids (such as a 50×50 grid) are used. Among them, the grid division is implemented through the cv2.divideGrid function of OpenCV, and the grid density is dynamically adjusted in combination with the deformation parameter gradient threshold. Then, for the geometric deformation parameters in each dynamic grid region, local surface modeling is performed using quadratic polynomial surface fitting. Among them, the modeling process can be completed by SciPy in the prior art, and the curvature parameters of each region are calculated through the second-order derivative of the fitted surface. The curvature parameters can be calculated through the following formula, that is:

[0065]

[0066] Among them, is the curvature parameter, 、 and are all the second-order partial derivatives of the surface. Finally, normalize the curvature parameters of all regions, and generate a global curvature feature map through weighted average fusion (the weights are determined by the region area and the deformation gradient). Then, use Gaussian filtering to smooth the feature map to eliminate the boundary noise of the global curvature feature map and obtain the curvature feature. Among them, the smoothing process and the curvature calculation process (the second-order partial derivative of the surface) can both be completed by OpenCV, which is beneficial to providing data support for the subsequent priority correction level division.

[0067] In this embodiment, the priority correction level is divided through the curvature feature, and then the nonlinear compensation factor is determined by the priority correction level and the light field attenuation vector. Specifically, the following method can be adopted, that is:

[0068] Perform dynamic priority division on the projection surface based on the gradient distribution characteristics of the curvature feature to obtain the priority correction level;

[0069] Determine the low-curvature compensation weight according to the priority correction level based on the linear coefficient in the light field attenuation vector, and then obtain the compensation weight matrix;

[0070] Perform nonlinear optimization on the compensation weight matrix to obtain the nonlinear compensation factor.

[0071] In specific implementation, first, the curvature gradient is calculated through the Sobel operator to dynamically divide the priority levels of the projection plane. Specifically, the projection plane can be divided into three priority regions, namely high, medium, and low, according to the curvature characteristics. For example, the region with a curvature gradient ≥ 0.8 is the priority correction region, the region with a curvature gradient between 0.3 and 0.8 is the medium correction region, and the region with a curvature gradient less than 0.3 is the low-priority correction region. Among them, the division process can be implemented through the cv2.threshold function in OpenCV. Then, by combining the light field attenuation vector with the priority correction level, a compensation weight matrix is generated. Through dynamic priority division, according to the linear attenuation coefficient in the light field attenuation vector and the priority correction level, and through the matrix operation of NumPy, a compensation weight matrix is generated, which can quantify the light attenuation compensation requirements of different regions. The weight of the high-priority region is enhanced to suppress edge artifacts, and the weight of the low-priority region is reduced to reduce computational redundancy. Finally, the compensation weight matrix is nonlinearly optimized to generate the final nonlinear compensation factor, and the minimization of the global brightness difference is used as the objective function for optimization. Furthermore, the optimized weight matrix is normalized into a nonlinear compensation factor. Among them, the overcompensation or undercompensation phenomenon is suppressed through nonlinear optimization, which can ensure the uniformity of the brightness distribution of the projection plane and the retention of details.

[0072] It should be noted that the priority correction level in this application refers to dynamically dividing the region priority based on the curvature gradient, which is used to solve the problem of resource waste caused by traditional uniform correction and improve the correction accuracy of high-curvature regions. The compensation weight matrix is a fusion of the light field attenuation coefficient and the priority weight, which can be used to achieve the collaborative compensation of geometric deformation and light attenuation and avoid brightness distortion caused by high reflectivity in the edge region. The nonlinear compensation factor refers to balancing the local compensation intensity through global optimization, reducing interpolation artifacts (such as jaggedness), and finally outputting a corrected image with high visual quality. In this embodiment, through the multi-level linkage of priority division, weight matrix generation, and nonlinear optimization, the dynamic coupling of geometric correction and light compensation is realized, which significantly improves the robustness of distortion correction in complex projection scenarios.

[0073] In step S3, the distortion-sensitive features of the projection image are obtained, an adaptive projection network is constructed based on the distortion-sensitive features and the nonlinear compensation factor, and the projection image is subjected to inverse coordinate mapping through the adaptive projection network to obtain an initial corrected image.

[0074] In specific implementation, the distortion-sensitive features of the projected image can be obtained through a multi-scale convolutional network. In actual implementation, by designing convolutional kernels of multiple different sizes, local details and overall structures of the input projected image can be extracted respectively to obtain the distortion-sensitive features of the projected image, that is: small-sized convolutional kernels are mainly used to capture local high-frequency information such as fine edges and textures in the image, reflecting local geometric deformations, while large-sized convolutional kernels focus on extracting global low-frequency features of the image, reflecting the overall contour and the attenuation trend of the light field; it should be noted that through the fusion of multi-level and multi-scale features, a feature map highly sensitive to image distortion can be constructed, which is beneficial to providing an accurate input basis for the subsequent adaptive projection network, thereby realizing more accurate inverse coordinate mapping and correction. In addition, the advantage of using a multi-scale convolutional network is that it improves the robustness of feature extraction, can effectively reduce errors caused by the loss of local details or scale mismatches, and ensures a high calibration accuracy even in a complex projection environment.

[0075] In this embodiment, constructing an adaptive projection network based on the distortion-sensitive features and the non-linear compensation factor can be specifically implemented in the following manner, that is:

[0076] Construct a projection network architecture;

[0077] Convert the non-linear compensation factor into a trainable scaling factor, and then dynamically adjust the weights of each channel in the constructed projection network architecture through the scaling factor to obtain a non-linear activation function;

[0078] Construct an adaptive projection network through the high-frequency components of the distortion-sensitive features and the non-linear activation function.

[0079] In specific implementation, the projection network architecture consists of multiple convolutional modules. Through convolutional layers, normalization layers, and non-linear activation layers, an end-to-end projection network architecture is formed. Residual connections are used to strengthen feature transfer and gradient flow, ensuring that the model can obtain stable training effects when capturing both global information and local details of the projected image. The construction process can be implemented using existing deep learning frameworks (such as TensorFlow), which is convenient for modular design and subsequent optimization. Then, using the non-linear compensation factor calculated from the existing geometric deformation parameters and light field attenuation vector, the non-linear compensation factor is converted into a trainable scaling factor through a mapping layer (such as a convolutional mapping layer). This scaling factor is then used as a network parameter to participate in training, and the weight distribution in each convolutional channel is dynamically adjusted. The activation function of the mapping layer after adjusting the weights (such as a parameterized ReLU or sigmoid function) is used as a non-linear function to directly embed the compensation factor into the activation process of the network. Finally, the high-frequency components represent the details and edge information in the image, reflecting the sensitive areas of local geometric deformation. They are extracted through an edge detection algorithm. The non-linear activation function is regulated by the trainable scaling factor to perform non-linear mapping on the feature responses of each channel. Among them, by inputting the high-frequency features into the network adjusted non-linearly, the inverse coordinate mapping can be achieved through operations such as convolution and pooling in the adjusted adaptive projection network, and the initial corrected image can be obtained.

[0080] It should be noted that in this embodiment, by constructing an adaptive projection network, the technical solution directly embeds the non-linear compensation factor into the network weight adjustment to form a data-driven dynamic activation function, thereby greatly improving the adaptability of the model to different projection regions. The deep convolutional network architecture can capture both global structure and local details simultaneously, ensuring that important edge information is not lost during the correction process. Converting the non-linear compensation factor into a trainable scaling factor mainly has the advantage that the network can automatically adjust the weights of each channel during training to achieve adaptive adjustment of the distortion strength, avoiding the limitations of traditional fixed activation functions. Using the high-frequency components of the distortion-sensitive features can more precisely reflect local geometric changes. By fusing with the non-linear activation function, artifacts and over-correction phenomena can be effectively suppressed.

[0081] Preferably, in this embodiment, referring to Figure 3 As shown, this figure is an exemplary flowchart for determining the initial corrected image in the embodiment of the present application. In this embodiment, the inverse coordinate mapping of the projected image is performed through the adaptive projection network, and the initial corrected image can be specifically obtained by the following steps:

[0082] First, in step S31, the pixel coordinates of the projected image are input into the adaptive projection network, and the local gradient features of the projected image are extracted through the adaptive projection network.

[0083] Then, in step S32, the adaptive projection network performs inverse coordinate transformation on the projected image through the local gradient features, and then performs bilinear interpolation sampling on the projected image after the inverse coordinate transformation to obtain an initial corrected image.

[0084] Specifically, first, the pixel coordinates of the projected image are input into the adaptive projection network. The convolutional layer at the front end of the adaptive projection network processes the input image, and the local gradient features around each pixel point in the image are extracted through a convolutional kernel improved based on the Sobel operator. These local gradient features reflect the edge, texture, and local structure information of the image and can accurately describe the subtle geometric distortion caused by the deformation of the projection plane. Then, the adaptive projection network passes through an embedded spatial transformation module. This module estimates the coordinate mapping function of the original distorted image based on the local gradient features, that is, through the learned inverse mapping relationship, maps the actual distorted pixel coordinates to the corrected state, and uses bilinear interpolation sampling technology for pixel reconstruction, and outputs an initial corrected image, which can achieve a high reduction accuracy in terms of geometric deformation and local details, laying a foundation for further optimization later.

[0085] It should be noted that, in this embodiment, the initial corrected image is a luminance channel image. The key to realizing inverse coordinate mapping through the adaptive projection network lies in making full use of the fact that the local gradient features can reflect the tiny geometric distortion information in the projected image. Using bilinear interpolation sampling can not only smooth the discrete problems brought by non-integer coordinates, but also maintain the continuity of image details and edge information while ensuring computational efficiency, significantly improving the geometric accuracy and visual quality of the corrected image, and providing an efficient technical solution for real-time image correction in complex projection scenarios.

[0086] In step S4, an artifact suppression weight map is determined through the high-frequency detail threshold and the low-frequency smoothing threshold of the initial corrected image.

[0087] In this embodiment, determining the artifact suppression weight map through the high-frequency detail threshold and the low-frequency smoothing threshold of the initial corrected image can be specifically implemented in the following way, that is:

[0088] A high-pass filter is used to extract the high-frequency region of the initial corrected image, and binary segmentation is performed on the high-frequency region according to a preset high-frequency detail threshold. Furthermore, the region where the detail intensity exceeds this threshold is used as a high-frequency detail segmentation map.

[0089] The initial corrected image is smoothed using a low-pass filter to extract the overall low-frequency information of the image, and then, according to a preset low-frequency smoothing threshold, the smoothed region is distinguished to obtain a low-frequency smoothing segmentation map.

[0090] Perform weighted fusion on the high-frequency detail segmentation map and the low-frequency smooth segmentation map to obtain an artifact suppression weight map.

[0091] Specifically, when implementing, the OpenCV image processing tool library can be used to complete the above steps. First, perform high-pass filtering on the initial corrected image through the cv2.Sobel function to extract the fine edges and texture information in the image. According to the preset high-frequency detail threshold, use the cv2.threshold function to perform threshold segmentation on the filtering result, and use the generated binary map of the high-frequency sensitive area as the high-frequency detail segmentation map. The area where the detail intensity exceeds this threshold is used as the high-frequency detail segmentation map. Among them, the high-frequency detail threshold can be determined through statistical analysis of the high-frequency response values of the image (for example, after calculating the high-frequency response mean and standard deviation, select the mean plus 1.5 times the standard deviation as the threshold); then, perform smoothing processing on the initial corrected image using cv2.GaussianBlur to extract the low-frequency information of the image, and then segment the result according to the low-frequency smooth threshold to obtain the low-frequency smooth segmentation map. Among them, the low-frequency smooth threshold is usually determined according to the overall brightness distribution of the image (for example, by calculating the image brightness histogram and selecting a certain percentage of the brightness mean as the boundary) to distinguish the overall smooth area and form the low-frequency smooth segmentation map; finally, use NumPy to perform image matrix weighted operations on the high-frequency detail segmentation map and the low-frequency smooth segmentation map to generate an artifact suppression weight map, which is used to reflect the local details and global smooth features of the image; in addition, in this embodiment, the threshold parameters (that is, the high-frequency detail threshold and the low-frequency smooth threshold) can be dynamically adjusted according to different scenarios to meet different correction requirements, which will not be limited here.

[0092] It should be noted that the artifact suppression weight map in this application is a pixel matrix with the same size as the image. Each pixel value represents the suppression strength required for this area in the subsequent pixel intensity adjustment. The artifact suppression weight map can provide an accurate regional control basis for the subsequent adjustment of the pixel intensity distribution. For example: during the image correction process, the high-frequency areas prone to artifacts will be given a higher suppression weight, thereby reducing their pixel gain and preventing overcorrection; while in the low-frequency smooth areas, the suppression weight is appropriately reduced to retain the overall smoothness and brightness consistency of the image; in this embodiment, the combination of the high-frequency detail threshold and the low-frequency smooth threshold can balance the contradiction between image detail retention and overall smoothness control. Among them, through high-frequency detail extraction, the local edge areas caused by geometric deformation can be located, while the low-frequency smooth areas can reflect the overall brightness distribution and texture continuity of the image. The artifact suppression weight map formed by fusing these two parts of information can, during the subsequent pixel intensity adjustment process, give a higher suppression weight to the areas prone to artifacts, while appropriately reducing the regulation intensity for the smooth areas.

[0093] In step S5, the pixel intensity distribution of the initial corrected image is adjusted according to the artifact suppression weight map, so as to obtain the projection image after distortion correction.

[0094] In this embodiment, adjusting the pixel intensity distribution of the initial corrected image according to the artifact suppression weight map to obtain the projection image after distortion correction can be specifically implemented in the following manner, that is:

[0095] Lower the pixel intensity of the high-weight region of the initial corrected image according to the artifact suppression weight map, and increase the pixel intensity of the low-weight region of the initial corrected image to obtain the initial corrected image after weight adjustment;

[0096] Perform local contrast adjustment and smoothing filtering on the local region of the initial corrected image after weight adjustment to obtain the projection image after distortion correction.

[0097] Specifically, the OpenCV library can be used to implement the above steps, that is: First, use cv2.multiply to perform element-wise multiplication on the initial corrected image and the artifact suppression weight map to achieve pixel-level weight weighting; then, use cv2.createCLAHE to perform adaptive histogram equalization on the sub-region image to adjust the local contrast; finally, call cv2.medianBlur to perform smoothing filtering on the processed image to remove the noise and discontinuity introduced by local adjustment.

[0098] It should be noted that by reducing the region with higher weights, artifacts caused by overcorrection can be effectively prevented, and increasing the pixels in the smoother region with lower weights is beneficial to enhancing the local details of the projection. Performing local adaptive contrast adjustment and smoothing filtering on the image after weight adjustment can further eliminate local noise and discontinuity, ensuring that the projected output image is natural, smooth, and has a balanced visual effect.

[0099] It can be seen that in this application, the geometric deformation parameters of image projection can be optimized and the light field attenuation can be compensated. First, the spatio-temporal convolutional layer and the bidirectional long short-term memory network are used to synchronously capture the time-varying characteristics of the dynamic deformation of the projection plane and the light field attenuation, generating a high-precision geometric deformation parameter matrix and a light field attenuation vector, providing a dynamic parameter basis for geometric correction and light compensation, solving the lag problem that the traditional static model cannot update parameters in real time, and improving the correction accuracy. Among them, the light field attenuation vector quantifies the influence of the projection plane material and curvature on the light intensity through a non-linear light reflection model, avoiding the edge brightness distortion caused by linear compensation, and providing data support for subsequent anti-artifact processing; Second, the geometric deformation parameters are optimized specifically to improve the local correction accuracy, and a dynamic non-linear compensation factor is generated through the non-linear coefficient in the light field attenuation vector, compensating for light attenuation differently according to the curvature and material characteristics of different regions, avoiding overexposure or underexposure problems caused by traditional uniform compensation, and suppressing artifacts from the source; Then, an adaptive projection network is constructed based on the distortion-sensitive features and the non-linear compensation factor, and the compensation factor is converted into a trainable scaling factor to dynamically adjust the network weights, so that the inverse coordinate mapping process can adapt to the non-linear relationship between geometric deformation and light field attenuation at the same time, improving the mapping accuracy. By guiding bilinear interpolation sampling through local gradient features, the detail blur caused by traditional global interpolation can be reduced, further suppressing high-frequency artifacts; Finally, an artifact suppression weight map is generated based on the high-frequency detail threshold and the low-frequency smoothing threshold, hierarchically controlling the pixel intensity adjustment amplitude, eliminating low-frequency artifacts while retaining details, realizing the closed-loop optimization of the anti-artifact ability, and then dynamically adjusting the pixel distribution through the weight map to balance the brightness consistency after geometric correction, avoiding local overcorrection problems caused by fixed thresholds in traditional methods, and improving the visual quality.

[0100] In summary, the technical solution adopted in this application can optimize the geometric deformation parameters of image projection and compensate for light field attenuation to improve the accuracy of distortion correction and the anti-artifact ability.

[0101] Embodiment 2. This application provides a distortion correction system for image projection, referring to Figure 4 As shown, this figure is a module structure diagram of the distortion correction system according to this embodiment of this application. The distortion correction system includes:

[0102] A distortion detection module 100, configured to construct a bidirectional distortion detection model based on the spatio-temporal domain, and generate a geometric deformation parameter matrix and a light field attenuation vector of the projection plane based on the bidirectional distortion detection model;

[0103] A compensation factor generation module 200, configured to perform regional analysis on the geometric deformation parameter matrix to obtain the curvature characteristics of the projection plane, divide the priority correction levels through the curvature characteristics, and further determine the non-linear compensation factor from the priority correction levels and the light field attenuation vector;

[0104] The inverse mapping module 300 is configured to obtain the distortion-sensitive features of the projected image, construct an adaptive projection network based on the distortion-sensitive features and the non-linear compensation factor, and perform inverse coordinate mapping on the projected image through the adaptive projection network to obtain an initial corrected image;

[0105] The threshold calculation module 400 is configured to determine an artifact suppression weight map based on the high-frequency detail threshold and the low-frequency smoothing threshold of the initial corrected image;

[0106] The image optimization module 500 is configured to adjust the pixel intensity distribution of the initial corrected image according to the artifact suppression weight map, so as to obtain a distortion-corrected projected image.

[0107] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It should be understood that each process and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions, and the combination of processes and / or blocks in the flowcharts and / or block diagrams can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0108] Those of ordinary skill in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disc memories, tape memories, or any other medium that can be used to carry or store data and is computer-readable.

[0109] It should also be noted that the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent in such a process, method, commodity or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, commodity or device including the element.

Claims

1. A distortion correction method for image projection, characterized in that, The distortion correction method includes the following steps: Construct a bidirectional distortion detection model based on the spatio-temporal domain, and generate a geometric deformation parameter matrix of the projection plane and a light field attenuation vector based on the bidirectional distortion detection model; Perform regional analysis on the geometric deformation parameter matrix to obtain the curvature characteristics of the projection plane, divide the priority correction levels through the curvature characteristics, and then determine the non-linear compensation factor from the priority correction levels and the light field attenuation vector; Obtain the distortion-sensitive features of the projection image, construct an adaptive projection network based on the distortion-sensitive features and the non-linear compensation factor, and perform inverse coordinate mapping on the projection image through the adaptive projection network to obtain an initial corrected image; Determine the artifact suppression weight map through the high-frequency detail threshold and the low-frequency smoothing threshold of the initial corrected image; Adjust the pixel intensity distribution of the initial corrected image according to the artifact suppression weight map, and then obtain the projection image after distortion correction; Among them, a spatio-temporal sequence of the projection plane is obtained through an image acquisition device, and then the spatio-temporal sequence is input into a bidirectional long short-term memory network including spatio-temporal convolutional layers, and then a bidirectional distortion detection model is obtained; Among them, constructing an adaptive projection network based on the distortion-sensitive features and the non-linear compensation factor specifically includes: Construct a projection network architecture; Convert the non-linear compensation factor into a trainable scaling factor, and then dynamically adjust the weights of each channel in the constructed projection network architecture through the scaling factor to obtain a non-linear activation function; Construct an adaptive projection network through the high-frequency components of the distortion-sensitive features and the non-linear activation function.

2. The distortion correction method for image projection according to claim 1, wherein, Generating a geometric deformation parameter matrix of the projection plane and a light field attenuation vector based on the bidirectional distortion detection model specifically includes: Perform spatial geometric modeling on the projection plane respectively through the bidirectional distortion detection model to obtain the feature reprojection error; Determine the geometric deformation parameter matrix of the projection plane according to the feature reprojection error; The bidirectional distortion detection model determines the light field attenuation vector by evaluating the light information amount of the projection plane.

3. The distortion correction method for image projection according to claim 1, characterized in that, Performing regional analysis on the geometric deformation parameter matrix to obtain the curvature characteristics of the projection plane specifically includes: Divide the geometric deformation parameter matrix into multiple dynamic regions; Perform curvature mapping on the geometric deformation parameters in each dynamic region respectively to obtain the curvature parameters of each dynamic region; Determine the curvature characteristics of the projection plane based on all the curvature parameters.

4. The distortion correction method for image projection according to claim 1, wherein Dividing the priority correction levels through the curvature characteristics, and then determining the non-linear compensation factor from the priority correction levels and the light field attenuation vector specifically includes: Perform dynamic priority division on the projection plane based on the gradient distribution characteristics of the curvature characteristics to obtain the priority correction levels; Determine the low-curvature compensation weight according to the priority correction levels through the linear coefficient in the light field attenuation vector, and then obtain the compensation weight matrix; Perform non-linear optimization on the compensation weight matrix to obtain the non-linear compensation factor.

5. The distortion correction method for image projection according to claim 1, wherein, Obtain the distortion-sensitive features of the projection image through a multi-scale convolutional network.

6. The distortion correction method for image projection according to claim 1, characterized in that, Performing inverse coordinate mapping on the projection image through the adaptive projection network to obtain an initial corrected image specifically includes: Input the pixel coordinates of the projected image into the adaptive projection network, and extract the local gradient features of the projected image through the adaptive projection network; The adaptive projection network performs inverse coordinate transformation on the projected image through the local gradient features, and then performs bilinear interpolation sampling on the projected image after the inverse coordinate transformation to obtain an initial corrected image.

7. A distortion correction method for image projection according to claim 1, characterized in that, The initial corrected image is a luminance channel image.

8. An image projection distortion correction system for performing an image projection distortion correction method according to any one of claims 1 to 7, characterized in that, The distortion correction system includes: A distortion detection module, configured to construct a bidirectional distortion detection model based on the spatio-temporal domain, and generate a geometric deformation parameter matrix and a light field attenuation vector of the projection plane based on the bidirectional distortion detection model; A compensation factor generation module, configured to perform region analysis on the geometric deformation parameter matrix to obtain the curvature feature of the projection plane, divide the priority correction level through the curvature feature, and then determine the nonlinear compensation factor from the priority correction level and the light field attenuation vector; An inverse mapping module, configured to obtain the distortion-sensitive features of the projected image, construct an adaptive projection network based on the distortion-sensitive features and the nonlinear compensation factor, and perform inverse coordinate mapping on the projected image through the adaptive projection network to obtain an initial corrected image; A threshold calculation module, configured to determine an artifact suppression weight map through the high-frequency detail threshold and the low-frequency smoothing threshold of the initial corrected image; An image optimization module, configured to adjust the pixel intensity distribution of the initial corrected image according to the artifact suppression weight map, and then obtain the projected image after distortion correction.

Images