Distortion correction method and system for image projection

Abstract

The invention provides a distortion correction method and system for image projection, and relates to the technical field of image processing, and the method comprises the steps: generating a geometric deformation parameter matrix and a light field attenuation vector of a projection surface through a constructed bidirectional distortion detection model, carrying out the regional analysis of the geometric deformation parameter matrix, and obtaining a curvature feature, dividing a priority correction level through curvature features, and determining a nonlinear compensation factor according to the priority correction level and a light field attenuation vector; constructing a self-adaptive projection network based on the distortion sensitive features of the projection image and the nonlinear compensation factor, and performing reverse coordinate mapping on the projection image through the self-adaptive projection network to obtain an initial correction image; determining an artifact suppression weight map through the high-frequency detail threshold value and the low-frequency smooth threshold value of the initial correction image; according to the invention, geometric deformation parameter optimization and light field attenuation compensation of image projection can be realized, so that the distortion correction precision and the anti-artifact capability are improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to a method and system for image projection distortion correction. Background Art

[0002] Image processing technology is one of the core technologies in the field of digital information and is widely used in computer vision, intelligent display, augmented reality and other scenarios. Among them, projection display technology relies on high-precision image correction algorithms to eliminate geometric distortion and brightness distortion caused by projection surface deformation or ambient light interference.

[0003] In the field of distortion correction of image projection, existing technologies mainly include parametric correction based on geometric calibration, brightness equalization method based on light field attenuation compensation, and coordinate transformation algorithm based on inverse mapping; however, existing methods are still difficult to adapt to the real-time deformation of dynamic surfaces (such as flexible screens and mobile projection surfaces), resulting in delayed update of correction parameters. They also ignore the nonlinear effect of the projection surface material and curvature on light reflection, which is prone to brightness distortion and artifacts in the edge area, resulting in insufficient response of image projection in high curvature areas, resulting in reduced local correction accuracy. Therefore, how to achieve geometric deformation parameter optimization and light field attenuation compensation for image projection to improve the accuracy of distortion correction and anti-artifact capability is a difficult problem currently faced by the industry. Summary of the invention

[0004] The present application provides a method and system for image projection distortion correction, which can achieve geometric deformation parameter optimization and light field attenuation compensation for image projection, so as to improve the accuracy of distortion correction and anti-artifact capability.

[0005] In a first aspect, the present application provides a method and system for image projection distortion correction, the distortion correction method comprising the following steps: Constructing a bidirectional distortion detection model based on the spatiotemporal domain, and generating a geometric deformation parameter matrix and a light field attenuation vector of the projection surface based on the bidirectional distortion detection model; Performing regional analysis on the geometric deformation parameter matrix to obtain curvature characteristics of the projection surface, dividing priority correction levels according to the curvature characteristics, and then determining a nonlinear compensation factor according to the priority correction levels and the light field attenuation vector; Acquire distortion-sensitive features of the projection image, construct an adaptive projection network based on the distortion-sensitive features and the nonlinear compensation factor, and perform inverse coordinate mapping on the projection image through the adaptive projection network to obtain an initial corrected image; Determine an artifact suppression weight map by using a high-frequency detail threshold and a low-frequency smoothing threshold of the initial corrected image; The pixel intensity distribution of the initial corrected image is adjusted according to the artifact suppression weight map, thereby obtaining a distortion-corrected projection image.

[0006] In this embodiment, a spatiotemporal sequence of the projection surface is acquired by an image acquisition device, and the spatiotemporal sequence is input into a bidirectional long short-term memory network including a spatiotemporal convolutional layer, thereby obtaining a bidirectional distortion detection model.

[0007] In this embodiment, generating a geometric deformation parameter matrix and a light field attenuation vector of a projection surface based on the bidirectional distortion detection model specifically includes: Using the bidirectional distortion detection model, spatial geometric modeling is performed on the projection surface to obtain a feature reprojection error; Determine a geometric deformation parameter matrix of the projection surface according to the characteristic reprojection error; The bidirectional distortion detection model determines the light field attenuation vector by evaluating the amount of light information on the projection surface.

[0008] In this embodiment, the region analysis is performed on the geometric deformation parameter matrix to obtain the curvature characteristics of the projection surface, which specifically include: Dividing the geometric deformation parameter matrix into a plurality of dynamic regions; Performing curvature mapping on the geometric deformation parameters in each dynamic region respectively to obtain the curvature parameters of each dynamic region; The curvature characteristics of the projection surface are determined based on all curvature parameters.

[0009] In this embodiment, dividing the priority correction levels according to the curvature characteristics, and then determining the nonlinear compensation factor according to the priority correction levels and the light field attenuation vector specifically includes: Based on the gradient distribution characteristics of the curvature features, the projection surface is dynamically prioritized to obtain the priority correction level; Determining the low curvature compensation weight according to the priority correction level through the linear coefficient in the light field attenuation vector, and then obtaining a compensation weight matrix; The compensation weight matrix is ​​nonlinearly optimized to obtain a nonlinear compensation factor.

[0010] In this embodiment, the distortion-sensitive features of the projection image are acquired through a multi-scale convolutional network.

[0011] In this embodiment, constructing an adaptive projection network based on the distortion sensitive feature and the nonlinear compensation factor specifically includes: Build the projection network architecture; The nonlinear compensation factor is converted into a trainable scaling factor, and then the weights of each channel in the projection network architecture are dynamically adjusted by the scaling factor to obtain a nonlinear activation function; An adaptive projection network is constructed by using the high-frequency components of the distortion-sensitive features and the nonlinear activation function.

[0012] In this embodiment, performing inverse coordinate mapping on the projection image through the adaptive projection network to obtain the initial corrected image specifically includes: Inputting pixel coordinates of the projection image into an adaptive projection network, and extracting local gradient features of the projection image through the adaptive projection network; The adaptive projection network performs inverse coordinate transformation on the projection image through the local gradient feature, and then performs bilinear interpolation sampling on the projection image after the inverse coordinate transformation to obtain an initial corrected image.

[0013] In this embodiment, the initial corrected image is a brightness channel image.

[0014] In a second aspect, the present application provides a distortion correction system for image projection, which is used to perform a distortion correction method for image projection, and the distortion correction system includes: A distortion detection module, used to construct a bidirectional distortion detection model based on the spatiotemporal domain, and to generate a geometric deformation parameter matrix and a light field attenuation vector of the projection surface based on the bidirectional distortion detection model; A compensation factor generation module, used for performing regional analysis on the geometric deformation parameter matrix to obtain curvature characteristics of the projection surface, dividing priority correction levels according to the curvature characteristics, and then determining a nonlinear compensation factor according to the priority correction levels and the light field attenuation vector; A reverse mapping module, used to obtain distortion-sensitive features of the projection image, construct an adaptive projection network based on the distortion-sensitive features and the nonlinear compensation factor, and perform reverse coordinate mapping on the projection image through the adaptive projection network to obtain an initial corrected image; A threshold calculation module, used to determine an artifact suppression weight map by using a high-frequency detail threshold and a low-frequency smoothing threshold of the initial corrected image; The image optimization module is used to adjust the pixel intensity distribution of the initial corrected image according to the artifact suppression weight map, so as to obtain a distortion-corrected projection image.

[0015] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects: A bidirectional distortion detection model based on the spatiotemporal domain is constructed, and a geometric deformation parameter matrix and a light field attenuation vector of the projection surface are generated based on the bidirectional distortion detection model; then, a regional analysis is performed on the geometric deformation parameter matrix to obtain the curvature characteristics of the projection surface, and priority correction levels are divided according to the curvature characteristics, and then a nonlinear compensation factor is determined according to the priority correction level and the light field attenuation vector; by acquiring the distortion-sensitive characteristics of the projection image, an adaptive projection network is constructed based on the distortion-sensitive characteristics and the nonlinear compensation factor, and the projection image is reversely coordinate mapped through the adaptive projection network to obtain an initial corrected image; then, an artifact suppression weight map is determined through the high-frequency detail threshold and the low-frequency smoothing threshold of the initial corrected image; the pixel intensity distribution of the initial corrected image is adjusted according to the artifact suppression weight map, and then a distortion-corrected projection image is obtained.

[0016] It can be seen that in the present application, the geometric deformation parameter optimization and light field attenuation compensation of image projection can be realized. First, the dynamic deformation of the projection surface and the time-varying characteristics of the light field attenuation are synchronously captured through the spatiotemporal convolution layer and the bidirectional long short-term memory network, and a high-precision geometric deformation parameter matrix and a light field attenuation vector are generated to provide a dynamic parameter basis for geometric correction and light compensation, solve the lag problem that the traditional static model cannot update the parameters in real time, and improve the correction accuracy. Among them, the light field attenuation vector quantifies the influence of the material and curvature of the projection surface on the light intensity through a nonlinear light reflection model, avoids the edge brightness distortion caused by linear compensation, and provides data support for subsequent anti-artifacts; secondly, the geometric deformation parameters are optimized in a targeted manner to improve the local correction accuracy, and the nonlinear coefficient in the light field attenuation vector is combined to generate a dynamic nonlinear compensation factor to differentially compensate for the light attenuation according to the curvature and material characteristics of different regions, so as to avoid the transmission. The overexposure or underexposure problem caused by uniform compensation can be solved to suppress artifacts from the source; then, an adaptive projection network is constructed based on distortion-sensitive features and nonlinear compensation factors, and the compensation factors are converted into trainable scaling factors. The network weights are dynamically adjusted so that the inverse coordinate mapping process can adapt to the nonlinear relationship between geometric deformation and light field attenuation at the same time to improve the mapping accuracy. The bilinear interpolation sampling is guided by local gradient features to reduce the detail blurring caused by traditional global interpolation and further suppress high-frequency artifacts; finally, an artifact suppression weight map is generated based on the high-frequency detail threshold and the low-frequency smoothing threshold, and the pixel intensity adjustment amplitude is controlled in layers to eliminate low-frequency artifacts while retaining details, thereby achieving closed-loop optimization of anti-artifact capabilities. The pixel distribution is then dynamically adjusted through the weight map to balance the brightness consistency after geometric correction, avoid the local over-correction problem caused by fixed thresholds in traditional methods, and improve visual quality.

[0017] In summary, the technical solution adopted in the present application can realize the optimization of geometric deformation parameters and light field attenuation compensation of image projection, so as to improve the accuracy of distortion correction and anti-artifact capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0019] Figure 1 is a flow chart of a method for image projection distortion correction provided by the present application; Figure 2 is an exemplary flow chart for determining the curvature characteristics of a projection surface provided by the present application; Figure 3 is an exemplary flow chart for determining an initial correction image according to the present application; Figure 4 It is a module structure diagram of the distortion correction system provided by this application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] The embodiment of the present application provides a method and system for image projection distortion correction, the core of which is to construct a bidirectional distortion detection model based on the spatiotemporal domain, and generate a geometric deformation parameter matrix and a light field attenuation vector of the projection surface based on the bidirectional distortion detection model; then perform regional analysis on the geometric deformation parameter matrix to obtain the curvature characteristics of the projection surface, divide the priority correction level according to the curvature characteristics, and then determine the nonlinear compensation factor according to the priority correction level and the light field attenuation vector; by acquiring the distortion sensitive characteristics of the projection image, an adaptive projection network is constructed based on the distortion sensitive characteristics and the nonlinear compensation factor, and the projection image is reversely mapped by the adaptive projection network to obtain an initial corrected image; then determine an artifact suppression weight map according to the high-frequency detail threshold and the low-frequency smoothing threshold of the initial corrected image; adjust the pixel intensity distribution of the initial corrected image according to the artifact suppression weight map, and then obtain the distortion-corrected projection image.

[0022] Embodiment 1: In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1As shown in FIG. 1 , this figure is an exemplary flow chart of a method for image projection distortion correction according to this embodiment of the present application, and the method for image projection distortion correction includes the following steps: In step S1, a bidirectional distortion detection model based on the spatiotemporal domain is constructed, and a geometric deformation parameter matrix and a light field attenuation vector of a projection surface are generated based on the bidirectional distortion detection model.

[0023] It should be noted that the spatiotemporal sequence of the projection surface is obtained by an image acquisition device, and then the spatiotemporal sequence is input into a bidirectional long short-term memory network including a spatiotemporal convolution layer, so as to obtain a bidirectional distortion detection model. Specifically, the following method can be used: first, continuous sequence data of the projection surface in the time and space dimensions is obtained by a multi-frame image acquisition device to form a spatiotemporal sequence input; second, the spatiotemporal sequence is input into the spatiotemporal convolution layer, which extracts the geometric features of each area of ​​the projection surface in the spatial domain through the convolution kernel, and analyzes the dynamic distortion characteristics between adjacent frames in the time domain; then, the output features of the spatiotemporal convolution layer are input into the bidirectional long short-term memory network, which captures the bidirectional propagation characteristics of the projection surface distortion in the time dimension through forward and backward time propagation, and fuses the local features of the spatiotemporal convolution layer with the temporal features of the bidirectional long short-term memory network through the spatiotemporal residual connection to generate a comprehensive representation of the dynamic distortion features; finally, combined with the light field attenuation model, the fused dynamic distortion features are mapped into the projection surface geometric deformation parameter matrix, and the light field attenuation vector is output synchronously.

[0024] In this embodiment, the geometric deformation parameter matrix and the light field attenuation vector of the projection surface are generated based on the bidirectional distortion detection model in the following manner, namely: Using the bidirectional distortion detection model, spatial geometric modeling is performed on the projection surface to obtain a feature reprojection error; Determine a geometric deformation parameter matrix of the projection surface according to the characteristic reprojection error; The bidirectional distortion detection model determines the light field attenuation vector by evaluating the amount of light information on the projection surface.

[0025] In the specific implementation, first, the three-dimensional point cloud data of the projection surface is obtained through the stereo vision calibration technology, and the iterative nearest point algorithm is used to align the ideal projection surface coordinate system with the actual distorted surface coordinate system. The Euclidean distance between the projection point and the reconstruction point is calculated through the OpenCV library, and the calculated Euclidean distance mean is used as the characteristic reprojection error, which can quantify the degree of geometric deformation of the projection surface; then, based on the characteristic reprojection error, a nonlinear least squares optimization algorithm is used to fit the polynomial distortion model, and the parameter weights are iteratively optimized through the least squares optimizer in the SciPy library to generate a geometric deformation parameter matrix (two-dimensional floating-point array) describing the dynamic deformation of the projection surface, which can map the characteristic reprojection error into a quantifiable compensation parameter; finally, the reflectivity and material properties of the projection surface are obtained through multispectral imaging technology, and the bidirectional distortion detection model is combined with the Lambertian body illumination model to calculate the light attenuation coefficient of each area, and the normalized light field attenuation vector is generated using MATLAB, which can quantify the nonlinear characteristics of light intensity attenuation and provide data input for light compensation.

[0026] It should be noted that the geometric deformation parameter matrix in the present application refers to a two-dimensional compensation weight matrix generated by analyzing the dynamic deformation characteristics of the projection surface. The dimension of the two-dimensional compensation weight matrix corresponds to the grid area divided by the projection surface, and each element represents the deformation compensation intensity of the corresponding area (such as stretching and compression weights), which can be used to quantify the degree of geometric distortion of the projection surface caused by material bending, environmental vibration, etc., and provide a dynamic correction basis for subsequent reverse coordinate mapping, thereby optimizing the local deformation error in the high curvature area and improving the overall geometric correction accuracy; in addition, the light field attenuation vector refers to a set of regional light attenuation coefficients (one-dimensional array) generated based on the reflectivity of the projection surface material, the curvature distribution and the light source distance. Each element in the light field attenuation vector represents the nonlinear light intensity attenuation ratio of the corresponding area, which can provide data support for the generation of light compensation factors, thereby solving the brightness distortion and residual artifacts problems caused by the traditional linear light compensation model in high curvature areas.

[0027] In step S2, a regional analysis is performed on the geometric deformation parameter matrix to obtain the curvature characteristics of the projection surface, and priority correction levels are divided according to the curvature characteristics. Furthermore, a nonlinear compensation factor is determined according to the priority correction levels and the light field attenuation vector.

[0028] Preferably, in this embodiment, reference Figure 2 As shown in FIG. 1 , this figure is an exemplary flow chart of determining the curvature characteristics of the projection surface in an embodiment of the present application. In this embodiment, the geometric deformation parameter matrix is ​​subjected to regional analysis to obtain the curvature characteristics of the projection surface, which can be specifically implemented by the following steps: First, in step S21, the geometric deformation parameter matrix is ​​divided into a plurality of dynamic regions; Then, in step S22, curvature mapping is performed on the geometric deformation parameters in each dynamic area to obtain the curvature parameters of each dynamic area; Finally, in step S23, the curvature characteristics of the projection surface are determined based on all the curvature parameters.

[0029] In the specific implementation, first, the deformation gradient is calculated by the Sobel operator, and the matrix is ​​adaptively divided into multiple dynamic grid areas by the deformation gradient. For example, dense grid division (for example, 10×10 grid) is used in high gradient areas (such as edge high deformation areas), and sparse grid (for example, 50×50 grid) is used in low gradient areas. The grid division is implemented by the cv2.divideGrid function of OpenCV, and the grid density is dynamically adjusted in combination with the deformation parameter gradient threshold. Then, for the geometric deformation parameters in each dynamic grid area, quadratic polynomial surface fitting is used to perform local surface modeling. The modeling process can be completed by SciPy in the prior art, and the curvature parameters of each area are calculated by the second-order derivative of the fitted surface. The curvature parameters can be calculated by the following formula, namely: in, is the curvature parameter, , and are all second-order partial derivatives of the surface; finally, the curvature parameters of all regions are normalized, and a global curvature feature map is generated by weighted average fusion (the weight is determined by the regional area and deformation gradient), and then the feature map is smoothed using Gaussian filtering to eliminate the boundary noise of the global curvature feature map and obtain the curvature feature. The smoothing process and the curvature calculation process (the second-order partial derivative of the surface) can be completed by OpenCV, which is conducive to providing data support for the subsequent priority correction level division.

[0030] In this embodiment, the priority correction levels are divided according to the curvature characteristics, and the nonlinear compensation factor is determined according to the priority correction levels and the light field attenuation vector in the following manner, namely: Based on the gradient distribution characteristics of the curvature features, the projection surface is dynamically prioritized to obtain the priority correction level; Determining the low curvature compensation weight according to the priority correction level through the linear coefficient in the light field attenuation vector, and then obtaining a compensation weight matrix; The compensation weight matrix is ​​nonlinearly optimized to obtain a nonlinear compensation factor.

[0031] In the specific implementation, first, the curvature gradient is calculated by the Sobel operator to dynamically prioritize the projection surface. Specifically, the projection surface can be divided into three priority areas: high, medium, and low according to the curvature characteristics. For example, the area with a curvature gradient ≥ 0.8 is the priority correction area, the area with a curvature gradient between 0.3 and 0.8 is the medium correction area, and the area with a curvature gradient less than 0.3 is the low priority correction area. The division process can be implemented by the cv2.threshold function in OpenCV; then, the compensation weight matrix is ​​generated by combining the light field attenuation vector with the priority correction level through dynamic priority division. The linear attenuation coefficient in the light field attenuation vector is combined with the priority correction level, and the compensation weight matrix is ​​generated through NumPy's matrix operation, which can quantify the light attenuation compensation requirements of different areas. The weights of high-priority areas are enhanced to suppress edge artifacts, and the weights of low-priority areas are reduced to reduce computational redundancy. Finally, the compensation weight matrix is ​​nonlinearly optimized to generate the final nonlinear compensation factor and minimize the global brightness difference as the objective function for optimization, and then the optimized weight matrix is ​​normalized into a nonlinear compensation factor. The nonlinear optimization can suppress over-compensation or under-compensation to ensure the uniformity of the brightness distribution of the projection surface and the retention of details.

[0032] It should be noted that the priority correction level in the present application refers to the dynamic division of regional priorities based on the curvature gradient, which is used to solve the problem of resource waste caused by traditional uniform correction and improve the correction accuracy of high curvature areas; the compensation weight matrix is ​​a fusion of the light field attenuation coefficient and the priority weight, which can be used to achieve coordinated compensation of geometric deformation and light attenuation, and avoid brightness distortion caused by high reflectivity in the edge area; the nonlinear compensation factor refers to balancing the local compensation intensity through global optimization, reducing interpolation artifacts (such as aliasing), and finally outputting a corrected image with high visual quality. In this embodiment, through multi-level linkage of priority division, weight matrix generation and nonlinear optimization, dynamic coupling of geometric correction and light compensation is achieved, which significantly improves the robustness of distortion correction in complex projection scenes.

[0033] In step S3, the distortion sensitive features of the projection image are obtained, an adaptive projection network is constructed based on the distortion sensitive features and the nonlinear compensation factor, and the projection image is reversely coordinate mapped through the adaptive projection network to obtain an initial corrected image.

[0034] In specific implementation, the distortion-sensitive features of the projection image can be obtained through a multi-scale convolutional network. In actual implementation, multiple convolution kernels of different sizes can be designed to extract local details and overall structures of the input projection image respectively, so as to obtain the distortion-sensitive features of the projection image, namely: small-size convolution kernels are mainly used to capture local high-frequency information such as subtle edges and textures in the image, reflecting local geometric deformation, while large-size convolution kernels focus on extracting global low-frequency features of the image, reflecting the overall contour and light field attenuation trend. It should be noted that through the fusion of multi-level and multi-scale features, a feature map that is highly sensitive to image distortion can be constructed, which is conducive to providing accurate input basis for the subsequent adaptive projection network, thereby realizing more accurate inverse coordinate mapping and correction. In addition, the advantage of using a multi-scale convolutional network is that it improves the robustness of feature extraction, can effectively reduce the errors caused by local detail loss or scale mismatch, and ensures that a high correction accuracy can be maintained in a complex projection environment.

[0035] In this embodiment, the adaptive projection network is constructed based on the distortion sensitive feature and the nonlinear compensation factor in the following manner, namely: Build the projection network architecture; The nonlinear compensation factor is converted into a trainable scaling factor, and then the weights of each channel in the projection network architecture are dynamically adjusted by the scaling factor to obtain a nonlinear activation function; An adaptive projection network is constructed by using the high-frequency components of the distortion-sensitive features and the nonlinear activation function.

[0036] In specific implementation, the projection network architecture is composed of multiple convolution modules, and an end-to-end projection network architecture is formed through convolution layers, normalization layers and nonlinear activation layers. The residual connection is used to strengthen feature transfer and gradient flow, ensuring that the model can obtain stable training effects when capturing global information and local details of the projected image. The construction process can be implemented using an existing deep learning framework (such as TensorFlow), which is convenient for modular design and subsequent tuning. Then, the nonlinear compensation factor calculated using the existing geometric deformation parameters and the light field attenuation vector is converted into a trainable scaling factor through a mapping layer (such as a convolution mapping layer), and the scaling factor is used as a network parameter to participate in the training. The method comprises the following steps: training, dynamically adjusting the weight distribution in each convolution channel, and using the activation function of the weight-adjusted mapping layer (such as a ReLU or sigmoid function with parameters) as a nonlinear function so as to embed the compensation factor directly into the activation process of the network; finally, the high-frequency component represents the details and edge information in the image, reflects the sensitive area of ​​local geometric deformation, and is extracted by an edge detection algorithm. The nonlinear activation function is regulated by a trainable scaling factor, and nonlinear mapping is performed on the feature responses of each channel, wherein the initial corrected image can be obtained by inputting the high-frequency features into the nonlinearly adjusted network, and realizing the reverse coordinate mapping through the convolution, pooling and other operations of the adjusted adaptive projection network.

[0037] It should be noted that in this embodiment, by constructing an adaptive projection network, the technical solution directly embeds the nonlinear compensation factor into the network weight adjustment to form a data-driven dynamic activation function, thereby greatly improving the adaptability of the model to different projection areas; the deep convolutional network architecture can capture global structure and local details at the same time, ensuring that important edge information is not lost during the correction process. The main advantage of converting the nonlinear compensation factor into a trainable scaling factor is that the network can automatically adjust the weights of each channel during training to achieve adaptive adjustment of the distortion strength, avoiding the limitations of traditional fixed activation functions; by using the high-frequency components of distortion-sensitive features, local geometric changes can be reflected more finely, and by fusing with the nonlinear activation function, artifacts and over-correction phenomena can be effectively suppressed.

[0038] Preferably, in this embodiment, reference Figure 3 As shown in FIG. 1 , this figure is an exemplary flow chart of determining an initial correction image in an embodiment of the present application. In this embodiment, the projection image is reversely mapped by the adaptive projection network to obtain the initial correction image, which can be specifically implemented by the following steps: First, in step S31, the pixel coordinates of the projection image are input into an adaptive projection network, and the local gradient features of the projection image are extracted through the adaptive projection network; Then, in step S32, the adaptive projection network performs inverse coordinate transformation on the projection image through the local gradient feature, and then performs bilinear interpolation sampling on the projection image after the inverse coordinate transformation to obtain an initial corrected image.

[0039] In the specific implementation, first, the pixel coordinates of the projected image are input into the adaptive projection network, the convolution layer at the front end of the adaptive projection network is used to process the input image, and the local gradient features around each pixel in the image are extracted through the convolution kernel improved based on the Sobel operator. These local gradient features reflect the edge, texture and local structure information of the image, and can accurately describe the subtle geometric distortion caused by the deformation of the projection surface; then, the adaptive projection network uses an embedded spatial transformation module, which estimates the coordinate mapping function of the original distorted image based on the local gradient features, that is, the inverse mapping relationship learned through training, maps the actual distorted pixel coordinates to the corrected state, and uses bilinear interpolation sampling technology to reconstruct pixels, and outputs the initial corrected image, which can achieve high restoration accuracy in both geometric deformation and local details, laying a foundation for subsequent further optimization.

[0040] It should be noted that, in this embodiment, the initial corrected image is a brightness channel image; the key to realizing inverse coordinate mapping through an adaptive projection network is to make full use of local gradient features to reflect the tiny geometric distortion information in the projected image, and the use of bilinear interpolation sampling can not only smoothly process the discrete problems caused by non-integer coordinates, but also maintain the continuity of image details and edge information while ensuring computational efficiency, significantly improving the geometric accuracy and visual quality of the corrected image, and providing an efficient technical solution for real-time image correction in complex projection scenes.

[0041] In step S4, an artifact suppression weight map is determined by using a high-frequency detail threshold and a low-frequency smoothing threshold of the initial corrected image.

[0042] In this embodiment, the artifact suppression weight map is determined by using the high-frequency detail threshold and the low-frequency smoothing threshold of the initial corrected image in the following manner, namely: Extracting the high-frequency region of the initial corrected image by high-pass filtering, performing binary segmentation on the high-frequency region according to a preset high-frequency detail threshold, and then taking the region where the detail intensity exceeds the threshold as a high-frequency detail segmentation map; The initial corrected image is smoothed by low-pass filtering to extract the overall low-frequency information of the image, and then the smoothed area is distinguished according to the preset low-frequency smoothing threshold to obtain a low-frequency smoothing segmentation map; The high-frequency detail segmentation map and the low-frequency smooth segmentation map are weightedly fused to obtain an artifact suppression weight map.

[0043] In the specific implementation, the OpenCV image processing tool library can be used to complete the above steps. First, the initial corrected image is high-pass filtered by the cv2.Sobel function to extract the subtle edge and texture information in the image. According to the preset high-frequency detail threshold, the filtering result is threshold segmented by the cv2.threshold function, and the binary image of the high-frequency sensitive area is generated as the high-frequency detail segmentation map, and the area where the detail intensity exceeds the threshold is used as the high-frequency detail segmentation map. The high-frequency detail threshold can be determined by statistical analysis of the high-frequency response value of the image (for example, after calculating the mean and standard deviation of the high-frequency response, the mean plus 1.5 times the standard deviation is selected as the threshold); then, the initial corrected image is subjected to the cv2.GaussianB lur is smoothed to extract the low-frequency information of the image, and then the result is segmented according to the low-frequency smoothing threshold to obtain a low-frequency smoothing segmentation map, wherein the low-frequency smoothing threshold is usually determined according to the overall brightness distribution of the image (for example, by calculating the image brightness histogram and selecting a certain percentage of the brightness mean as the boundary) to distinguish the overall smooth area and form a low-frequency smoothing segmentation map; finally, NumPy is used to perform image matrix weighting operation on the high-frequency detail segmentation map and the low-frequency smoothing segmentation map to generate an artifact suppression weight map for reflecting the local details and global smoothing features of the image; in addition, in this embodiment, the threshold parameters (i.e., the high-frequency detail threshold and the low-frequency smoothing threshold) can be dynamically adjusted according to different scenes to meet different correction requirements, which are not limited here.

[0044] It should be noted that the artifact suppression weight map in the present application is a pixel matrix of the same size as the image, in which each pixel value represents the suppression strength required for the area in the subsequent pixel intensity adjustment. The artifact suppression weight map can provide an accurate regional control basis for the subsequent pixel intensity distribution adjustment. For example, in the image correction process, high-frequency areas that are prone to artifacts will be given higher suppression weights, thereby reducing their pixel gain and preventing over-correction; and in low-frequency smooth areas, the suppression weights are appropriately reduced to retain the overall smoothness and brightness consistency of the image; in this embodiment, the combination of high-frequency detail thresholds and low-frequency smooth thresholds can balance the contradiction between image detail retention and overall smoothness control, wherein, through high-frequency detail extraction, the local edge areas caused by geometric deformation can be located, while the low-frequency smooth areas can reflect the overall brightness distribution and texture continuity of the image. The artifact suppression weight map formed by fusing these two parts of information can be used in the subsequent pixel intensity adjustment process. In the process, a higher suppression weight is given to areas that are prone to artifacts, while the regulation intensity is appropriately reduced for smooth areas.

[0045] In step S5, the pixel intensity distribution of the initial corrected image is adjusted according to the artifact suppression weight map, thereby obtaining a distortion-corrected projection image.

[0046] In this embodiment, the pixel intensity distribution of the initial corrected image is adjusted according to the artifact suppression weight map, so as to obtain the distortion-corrected projection image in the following manner, namely: According to the artifact suppression weight map, the pixel intensity of the high-weight region of the initial corrected image is reduced, and the pixel intensity of the low-weight region of the initial corrected image is increased to obtain the initial corrected image after weight adjustment; Local contrast adjustment and smoothing filtering are performed on the local area of ​​the initial corrected image after weight adjustment to obtain a distortion-corrected projection image.

[0047] In the specific implementation, the OpenCV library can be used to implement the above steps, namely: first, use cv2.multiply to perform element-by-element multiplication of the initial corrected image and the artifact suppression weight map to achieve pixel-level weighting; then, use cv2.createCLAHE to perform adaptive histogram equalization on the regional image to adjust the local contrast; finally, call cv2.medianBlur to smooth the processed image to remove noise and discontinuity introduced by local adjustment.

[0048] It should be noted that by reducing the areas with higher weights, artifacts caused by over-correction can be effectively prevented, and increasing the pixels in the smooth areas with lower weights is beneficial to enhancing the local details of the projection. Local adaptive contrast adjustment and smoothing filtering are performed on the weight-adjusted image to further eliminate local noise and discontinuities, ensuring that the projected output image is naturally smooth and the visual effect is balanced.

[0049] It can be seen that in the present application, the geometric deformation parameter optimization and light field attenuation compensation of image projection can be realized. First, the dynamic deformation of the projection surface and the time-varying characteristics of the light field attenuation are synchronously captured through the spatiotemporal convolution layer and the bidirectional long short-term memory network, and a high-precision geometric deformation parameter matrix and a light field attenuation vector are generated to provide a dynamic parameter basis for geometric correction and light compensation, solve the lag problem that the traditional static model cannot update the parameters in real time, and improve the correction accuracy. Among them, the light field attenuation vector quantifies the influence of the material and curvature of the projection surface on the light intensity through a nonlinear light reflection model, avoids the edge brightness distortion caused by linear compensation, and provides data support for subsequent anti-artifacts; secondly, the geometric deformation parameters are optimized in a targeted manner to improve the local correction accuracy, and the dynamic nonlinear compensation factor is generated through the nonlinear coefficient in the light field attenuation vector to differentially compensate for the light attenuation according to the curvature and material characteristics of different regions, so as to avoid the transmission. The overexposure or underexposure problem caused by uniform compensation can be solved to suppress artifacts from the source; then, an adaptive projection network is constructed based on distortion-sensitive features and nonlinear compensation factors, and the compensation factors are converted into trainable scaling factors. The network weights are dynamically adjusted so that the inverse coordinate mapping process can adapt to the nonlinear relationship between geometric deformation and light field attenuation at the same time to improve the mapping accuracy. The bilinear interpolation sampling is guided by local gradient features to reduce the detail blurring caused by traditional global interpolation and further suppress high-frequency artifacts; finally, an artifact suppression weight map is generated based on the high-frequency detail threshold and the low-frequency smoothing threshold, and the pixel intensity adjustment amplitude is controlled in layers to eliminate low-frequency artifacts while retaining details, thereby achieving closed-loop optimization of anti-artifact capabilities. The pixel distribution is then dynamically adjusted through the weight map to balance the brightness consistency after geometric correction, avoid the local over-correction problem caused by fixed thresholds in traditional methods, and improve visual quality.

[0050] In summary, the technical solution adopted in the present application can realize the optimization of geometric deformation parameters and light field attenuation compensation of image projection, so as to improve the accuracy of distortion correction and anti-artifact capability.

[0051] Embodiment 2: This application provides a distortion correction system for image projection, referring to Figure 4 As shown, this figure is a module structure diagram of the distortion correction system shown in this embodiment of the present application, and the distortion correction system includes: The distortion detection module 100 is used to construct a bidirectional distortion detection model based on the spatiotemporal domain, and to generate a geometric deformation parameter matrix and a light field attenuation vector of the projection surface based on the bidirectional distortion detection model; A compensation factor generation module 200 is used to perform regional analysis on the geometric deformation parameter matrix to obtain curvature characteristics of the projection surface, divide priority correction levels according to the curvature characteristics, and then determine a nonlinear compensation factor according to the priority correction level and the light field attenuation vector; The inverse mapping module 300 is used to obtain the distortion sensitive features of the projection image, construct an adaptive projection network based on the distortion sensitive features and the nonlinear compensation factor, and perform inverse coordinate mapping on the projection image through the adaptive projection network to obtain an initial corrected image; A threshold calculation module 400, configured to determine an artifact suppression weight map by using a high-frequency detail threshold and a low-frequency smoothing threshold of the initial corrected image; The image optimization module 500 is used to adjust the pixel intensity distribution of the initial corrected image according to the artifact suppression weight map, so as to obtain a distortion-corrected projection image.

[0052] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0053] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, the storage medium including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically-erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0054] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

Claims

1. A method for image projection distortion correction, characterized in that: The distortion correction method comprises the following steps: Constructing a bidirectional distortion detection model based on the spatiotemporal domain, and generating a geometric deformation parameter matrix and a light field attenuation vector of the projection surface based on the bidirectional distortion detection model; Performing regional analysis on the geometric deformation parameter matrix to obtain curvature characteristics of the projection surface, dividing priority correction levels according to the curvature characteristics, and then determining a nonlinear compensation factor according to the priority correction levels and the light field attenuation vector; Acquire distortion-sensitive features of the projection image, construct an adaptive projection network based on the distortion-sensitive features and the nonlinear compensation factor, and perform inverse coordinate mapping on the projection image through the adaptive projection network to obtain an initial corrected image; Determine an artifact suppression weight map by using a high-frequency detail threshold and a low-frequency smoothing threshold of the initial corrected image; The pixel intensity distribution of the initial corrected image is adjusted according to the artifact suppression weight map, thereby obtaining a distortion-corrected projection image.

2. The method for image projection distortion correction according to claim 1, characterized in that: The spatiotemporal sequence of the projection surface is acquired through an image acquisition device, and the spatiotemporal sequence is then input into a bidirectional long short-term memory network including a spatiotemporal convolutional layer, thereby obtaining a bidirectional distortion detection model.

3. The method for image projection distortion correction according to claim 1, characterized in that: Generating the geometric deformation parameter matrix and light field attenuation vector of the projection surface based on the bidirectional distortion detection model specifically includes: Using the bidirectional distortion detection model, spatial geometric modeling is performed on the projection surface to obtain a feature reprojection error; Determine a geometric deformation parameter matrix of the projection surface according to the characteristic reprojection error; The bidirectional distortion detection model determines the light field attenuation vector by evaluating the amount of light information on the projection surface.

4. The method for image projection distortion correction according to claim 1, characterized in that: The curvature characteristics of the projection surface obtained by performing regional analysis on the geometric deformation parameter matrix specifically include: Dividing the geometric deformation parameter matrix into a plurality of dynamic regions; Performing curvature mapping on the geometric deformation parameters in each dynamic region respectively to obtain the curvature parameters of each dynamic region; The curvature characteristics of the projection surface are determined based on all curvature parameters.

5. The method for image projection distortion correction according to claim 1, characterized in that: Dividing the priority correction levels by the curvature characteristics, and then determining the nonlinear compensation factor by the priority correction levels and the light field attenuation vector specifically includes: Based on the gradient distribution characteristics of the curvature features, the projection surface is dynamically prioritized to obtain the priority correction level; Determining the low curvature compensation weight according to the priority correction level through the linear coefficient in the light field attenuation vector, and then obtaining a compensation weight matrix; The compensation weight matrix is ​​nonlinearly optimized to obtain a nonlinear compensation factor.

6. The method for image projection distortion correction according to claim 1, characterized in that: The distortion-sensitive features of the projected image are obtained through a multi-scale convolutional network.

7. The method for image projection distortion correction according to claim 1, characterized in that: Constructing an adaptive projection network based on the distortion sensitive feature and the nonlinear compensation factor specifically includes: Build the projection network architecture; The nonlinear compensation factor is converted into a trainable scaling factor, and then the weights of each channel in the projection network architecture are dynamically adjusted by the scaling factor to obtain a nonlinear activation function; An adaptive projection network is constructed by using the high-frequency components of the distortion-sensitive features and the nonlinear activation function.

8. The method for image projection distortion correction according to claim 1, characterized in that: Performing inverse coordinate mapping on the projection image through the adaptive projection network to obtain an initial corrected image specifically includes: Inputting pixel coordinates of the projection image into an adaptive projection network, and extracting local gradient features of the projection image through the adaptive projection network; The adaptive projection network performs inverse coordinate transformation on the projection image through the local gradient feature, and then performs bilinear interpolation sampling on the projection image after the inverse coordinate transformation to obtain an initial corrected image.

9. The method for image projection distortion correction according to claim 1, characterized in that: The initial corrected image is a brightness channel image.

10. A distortion correction system for image projection, used to execute the distortion correction method for image projection according to any one of claims 1 to 9, characterized in that: The distortion correction system comprises: A distortion detection module, used to construct a bidirectional distortion detection model based on the spatiotemporal domain, and to generate a geometric deformation parameter matrix and a light field attenuation vector of the projection surface based on the bidirectional distortion detection model; A compensation factor generation module, used for performing regional analysis on the geometric deformation parameter matrix to obtain curvature characteristics of the projection surface, dividing priority correction levels according to the curvature characteristics, and then determining a nonlinear compensation factor according to the priority correction levels and the light field attenuation vector; A reverse mapping module, used to obtain distortion-sensitive features of the projection image, construct an adaptive projection network based on the distortion-sensitive features and the nonlinear compensation factor, and perform reverse coordinate mapping on the projection image through the adaptive projection network to obtain an initial corrected image; A threshold calculation module, used to determine an artifact suppression weight map by using a high-frequency detail threshold and a low-frequency smoothing threshold of the initial corrected image; The image optimization module is used to adjust the pixel intensity distribution of the initial corrected image according to the artifact suppression weight map, so as to obtain a distortion-corrected projection image.

Images