Projection image optimization method based on image processing

Abstract

The present application relates to the technical field of image processing, and more particularly to a projection image optimization method based on image processing, which first extracts image block samples from a target scene, uses an industrial camera to collect data and combines a texture separation model to accurately extract features to determine key texture distortion anchor points. Secondly, according to these anchor points, the non-flat wall surface is divided into multiple distortion transmission areas to realize local fine modeling. Then, the preset imaging parameters are independently optimized in multiple channels for different areas to generate projection correction parameters adapted to each area. Finally, the optimized parameters are used to control the imaging of the projection equipment. This method effectively decouples the inherent texture of the wall surface and the interference of the projection content, solves the problems of non-linear distortion, uneven brightness and blurring caused by complex curved surfaces through adaptive correction by area, significantly improves the clarity, contrast and visual consistency of the projection image, and does not require expensive three-dimensional scanning hardware, with high robustness and low cost advantage.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for optimizing projected images based on image processing. Background Technology

[0002] With the widespread application of projection display technology, uneven walls are commonly used as projection surfaces in exhibitions, immersive cinemas, and architectural projection shows. However, due to the unevenness, material uniformity, and complex textures of wall surfaces, traditional projection methods are prone to image distortion, local blurring, color distortion, and uneven brightness when projecting onto uneven walls. This severely affects the clarity and visual consistency of the image, leading to a significant deterioration in the quality of the projected image. Existing projection correction technologies mostly rely on geometric calibration and edge blending algorithms, typically assuming that the projection surface is an approximate plane or a known curved surface. This makes it difficult to effectively cope with the nonlinear distortions caused by the complex and varied wall textures and microstructures in real-world environments.

[0003] In recent years, projection enhancement methods based on image processing have gradually emerged. Some studies have attempted to acquire 3D information of the wall surface using depth cameras or structured light for pre-compensation, but these methods are costly in hardware, complex to deploy, and poorly adaptable to dynamic scenes. Other methods use image denoising or sharpening techniques to improve visual effects, but they do not fundamentally separate the interference between the wall's inherent texture and the projected content, resulting in limited enhancement effects and even introducing artifacts. Furthermore, existing technologies generally lack modeling of the propagation mechanism of local texture distortion on the wall surface, making it impossible to achieve differentiated parameter optimization for different distortion areas, thus limiting the accuracy and robustness of image enhancement.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a projection image optimization method based on image processing, which aims to solve the technical problem that existing projection technologies for non-flat walls are unable to effectively separate the interference between the inherent texture of the wall and the projection content, and lack the ability to model the propagation mechanism of local texture distortion and optimize differentiated parameters, resulting in the inability to accurately compensate for nonlinear image distortion caused by complex surface morphology, thus causing a decline in projection image quality.

[0006] To achieve the above objectives, the present invention provides a projection image optimization method based on image processing, the method comprising: Extract a predetermined number of image blocks from the non-flat wall projection image of the target projection scene to obtain a sample projection image block set; The original projection images of the sample projection image block set are obtained according to the preset projection imaging parameters, and the projection imaging data to be processed is obtained by calling an industrial camera. The texture features are extracted by combining the texture separation model to determine the positions of multiple key texture distortion anchor points. Based on the locations of the multiple key texture distortion anchor points, the distortion transmission area of ​​the uneven wall surface is divided to determine multiple texture distortion transmission areas; Based on the multiple texture distortion transmission regions, the preset projection imaging parameters are optimized in multiple regions to determine multiple region projection correction parameters. Based on the multiple area projection correction parameters, the projection device is controlled to project an image onto the uneven wall surface and output an enhanced projection image.

[0007] Optionally, the original projected images of the sample projected image patch set are obtained according to preset projection imaging parameters, and the projected imaging data to be processed is acquired by calling an industrial camera. Texture features are extracted by combining a texture separation model to determine the positions of multiple key texture distortion anchor points, including: The sample projection content is projected onto the target uneven wall surface according to the preset projection imaging parameters, and the industrial camera is called to complete the acquisition of the original projection image to obtain the original projection image to be processed. The texture separation model is called to perform texture decomposition on the original projection image to be processed. The texture weight is corrected by combining the original standard projection texture reference and the inherent texture reference of the wall surface, and the corrected wall surface unevenness distortion texture component, projection content texture component and ambient light interference texture component are obtained. The key texture distortion anchor point positions are identified by traversing the concave-convex distortion texture components, the projected content texture components, and the ambient lighting interference texture components, thereby obtaining the key texture distortion anchor point position sets for the convex region, the concave region, and the lighting offset region. By integrating the sets of key texture distortion anchor points in the raised region, the set of key texture distortion anchor points in the recessed region, and the set of key texture distortion anchor points in the illumination offset region, multiple key texture distortion anchor point positions are obtained.

[0008] Optionally, obtaining the set of key texture distortion anchor point locations in the raised region includes: Based on the three-dimensional point cloud structure information of the target non-flat wall surface, obtain the set of geometric prior anchor point positions; According to the preset distortion evaluation index, the set of geometric prior anchor points is scored by combining the concave and convex distortion texture components to obtain a set of candidate scores for geometric prior anchor points. The set of candidate scores for geometric prior anchor point positions is filtered based on a preset pixel spacing to determine the set of key texture distortion anchor point positions in the raised area.

[0009] Optionally, the preset distortion evaluation index includes at least texture offset, gradient distortion peak, and spatial consistency.

[0010] Optionally, the step of segmenting the non-flat wall surface into distortion transmission regions based on the positions of the multiple key texture distortion anchor points, and determining multiple texture distortion transmission regions, includes: A preset standard projection is performed on the target non-flat wall surface, and texture acquisition is performed on the multiple key texture distortion anchor point positions to obtain the texture sequence of multiple key texture distortion anchor point positions. Distortion feature analysis is performed on the texture sequences of the multiple key texture distortion anchor points to obtain distortion trend features of the multiple key texture distortion anchor points. Distortion coupling analysis is performed based on the distortion trend characteristics of multiple key texture distortion anchor points to determine the distortion coupling trend characteristics of multiple key texture distortion anchor points. Based on the distortion coupling trend characteristics of the multiple key texture distortion anchor points, the texture distortion propagation region is split to determine the multiple texture distortion propagation regions, wherein each texture distortion propagation region includes distortion integrated texture features.

[0011] Optionally, the distortion coupling analysis based on the distortion trend features of multiple key texture distortion anchor point positions, to determine the distortion coupling trend features of multiple key texture distortion anchor point positions, includes: Based on the pixel spacing distance between the multiple key texture distortion anchor points, the key texture distortion anchor points whose distance to the multiple key texture distortion anchor points is within a preset distortion distance threshold are added to their neighborhood, thereby obtaining the distortion trend feature neighborhood of the multiple key texture distortion anchor points. Based on the distortion trend feature neighborhood of the multiple key texture distortion anchor points, distortion coupling analysis is performed on the distortion trend features of the multiple key texture distortion anchor points to obtain the distortion coupling trend features of the multiple key texture distortion anchor points.

[0012] Optionally, the distortion trend feature neighborhood based on the positions of the multiple key texture distortion anchor points is used to perform distortion coupling analysis on the distortion trend features of the positions of the multiple key texture distortion anchor points to obtain the distortion coupling trend features of the positions of the multiple key texture distortion anchor points, including: Calculate the feature similarity set between the neighborhood of the distortion trend feature of multiple key texture distortion anchor points and the corresponding distortion trend feature of multiple key texture distortion anchor points, and calculate the mean to obtain multiple feature similarity mean sets. The multiple feature similarity mean sets are normalized respectively to construct multiple coupling analysis vectors. Then, a convolutional network is called to analyze the distortion trend features of the multiple coupling analysis vectors and multiple key texture distortion anchor point positions to obtain the distortion coupling trend features of multiple key texture distortion anchor point positions.

[0013] Optionally, the step of performing multi-region optimization on the preset projection imaging parameters based on the multiple texture distortion transmission regions to determine multiple region projection correction parameters includes: Based on the distortion-integrated texture features corresponding to the multiple texture distortion transmission regions, the region correction optimizer is called to perform analysis and determine multiple initial region projection correction parameters. Neighborhood interference analysis is performed on the multiple initial region projection correction parameters, and the parameters are optimized based on the analysis results to obtain the multiple region projection correction parameters.

[0014] Furthermore, to achieve the above objectives, the present invention also provides a projection image optimization device based on image processing, the device comprising: a memory, a processor, and a projection image optimization program based on image processing stored in the memory and executable on the processor, the projection image optimization program based on image processing being configured to implement the steps of the projection image optimization method based on image processing as described above.

[0015] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing an image processing-based projection image optimization program, which, when executed by a processor, implements the steps of the image processing-based projection image optimization method as described above.

[0016] This invention provides a projection image optimization method based on image processing. By introducing a texture separation model, this method can accurately decompose the texture of the projected content from the inherent unevenness of the wall surface, material noise, and ambient lighting interference. This decoupling process fundamentally eliminates the visual interference of the wall background on the projected image, avoiding the miscorrection or artifact problems caused by the inability to distinguish between wall imperfections and image details in traditional methods, significantly improving the clarity and color reproduction of the output image. Unlike traditional globally uniform correction strategies, this method identifies key texture distortion anchor points, enabling it to keenly capture the local nonlinear distortion characteristics caused by the wall's micro-morphology. Using this as a benchmark for region segmentation, the system can accurately perceive the degree and type of distortion at different locations, providing reliable data support for subsequent high-precision compensation and effectively solving the problems of image stretching, compression, and blurring under complex curved surfaces. By dividing the wall surface into multiple texture distortion transmission regions and independently optimizing the projection imaging parameters of each region, this method achieves site-specific differentiated correction. For areas with raised areas, depressions, or drastically varying textures, the system can automatically adjust corresponding geometric deformation, brightness gain, or focus parameters. This not only overcomes the problems of uneven brightness and focus drift caused by uneven walls but also significantly improves the algorithm's adaptability and robustness to different types of walls. This method relies on industrial cameras to collect data and processes it using an algorithm model, eliminating the need for expensive and complex depth cameras, structured light scanners, or high-precision LiDAR to acquire full 3D point clouds. This significantly reduces the system's hardware cost and calibration complexity while ensuring high-quality enhancement, making it easier to deploy and apply quickly in large-scale or dynamic scenarios such as exhibitions and architectural projections. From sample extraction and feature analysis to parameter optimization and final imaging control, the entire process forms a complete closed-loop optimization system. The system can dynamically adjust correction parameters based on real-time acquired imaging data, effectively addressing the effects of changes in ambient light or minor wall displacements, ensuring that the projected image maintains high consistency and stability throughout long-term operation. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an embodiment of the projection image optimization method based on image processing according to the present invention; Figure 2 This is a flowchart illustrating another embodiment of the projection image optimization method based on image processing of the present invention.

[0018] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0020] Reference Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the projection image optimization method based on image processing according to the present invention.

[0021] In one embodiment, the image processing-based projection image optimization method includes: Step S100: Extract a preset number of image blocks from the non-flat wall projection image of the target projection scene to obtain a sample projection image block set.

[0022] The non-flat wall projection image of the target projection scene can be an initial image projected onto a wall with unevenness, material inhomogeneity, or complex texture. It serves as the input source for the entire optimization process, containing mixed information of the projected content and wall interference. The non-flat wall can be a physical projection carrier with uneven surfaces, material inhomogeneity, or complex textures; its geometric and physical characteristics are the root cause of image distortion. A preset number of image blocks can be local image regions cropped from the non-flat wall projection image according to rules or randomly, used to construct a sample set to support subsequent texture feature extraction and anchor point localization. The sample projection image block set can be a collection of multiple preset number of image blocks used for batch processing and analysis, serving as input to the texture separation model and supporting the identification of key texture distortion anchor points. Extracting a preset number of image blocks from the non-flat wall projection image of the target projection scene can be done by cropping local areas from the entire projection image using sliding windows, grid division, or interest point sampling methods. Furthermore, this operation can be achieved by uniform sampling using a sliding window of a fixed size or by adaptive region selection based on image gradients, thereby constructing a representative local sample set that covers regions with different distortion types on the wall.

[0023] Step S200: Obtain the original projection images of the sample projection image block set according to the preset projection imaging parameters, and call the industrial camera to obtain the projection imaging data to be processed. Combine the texture separation model to extract texture features and determine the positions of multiple key texture distortion anchor points.

[0024] The preset projection imaging parameters can be the initial geometry, brightness, and focus settings of the projection device in its uncorrected state, serving as the basis for generating the original projected image and the starting point for subsequent multi-region optimization. The original projected image can be an unoptimized image projected onto a non-flat wall surface according to the preset projection imaging parameters and captured by an industrial camera. It reflects the true imaging state under wall interference and is compared and analyzed with sample image blocks. The industrial camera can be a highly stable imaging device used to acquire two-dimensional image data of the projection scene. By being fixedly installed in the projection environment, it captures high-resolution, low-noise images of the projected image on the wall, providing the projection imaging data to be processed. This replaces 3D sensing devices such as depth cameras, reducing system cost and deployment complexity. The projection imaging data to be processed can be two-dimensional image data containing the projected content and wall interference acquired by the industrial camera. It serves as input to the texture separation model for extracting texture features. Texture features can be visual attributes extracted from the projection imaging data to be processed that characterize the differences between the inherent texture of the wall and the projected content, supporting the determination of key texture distortion anchor point positions.

[0025] Texture separation models are image processing models used to decouple background textures caused by the inherent characteristics of the wall surface from the actual projected content in a projected image. This allows for precise separation of the projected content texture from the wall's uneven texture, material clutter, and ambient lighting interference, avoiding miscorrection and artifacts, and improving clarity and color reproduction. In one exemplary embodiment, the texture separation model can be based on a supervised or self-supervised learning framework, trained using paired or unpaired projection-wall image data, and decomposed through an encoder-decoder structure or a generative adversarial network. For example, texture separation models may include U-Net-type texture separation networks, CycleGAN-type domain transfer models, and physically guided decomposition models. Key texture distortion anchor points can be spatial feature points identified on uneven wall surfaces that characterize nonlinear distortions caused by local micro-morphology. These provide a high-precision spatial benchmark for wall region segmentation, supporting subsequent differential correction. In a specific embodiment, key texture distortion anchor point locations can be determined by performing gradient analysis, curvature estimation, or saliency detection on texture features to locate the center or extreme points of distortion-sensitive regions in sample projected image blocks. Furthermore, the location of key texture distortion anchor points can include stretch distortion anchor points, compression distortion anchor points, blur distortion anchor points, etc.

[0026] The original projected images are obtained for each sample projection image block set according to preset projection imaging parameters. This can be achieved by projecting a standard test pattern onto the wall using the current projection equipment configuration, and simultaneously acquiring the images with an industrial camera. Further, this operation can be implemented by projecting a checkerboard pattern and then acquiring the image, or by projecting grayscale gradient stripes and then acquiring the image, thereby establishing a mapping relationship between wall interference and projection parameters, providing a benchmark for subsequent analysis. The industrial camera is then invoked to acquire the projection imaging data to be processed. This can be achieved by triggering the industrial camera to capture the current wall projection image and transmit it to the processing unit. For example, this operation can be achieved through single-frame high-resolution shooting or multi-frame averaging and noise reduction shooting, thereby obtaining two-dimensional observation data for texture separation and avoiding reliance on three-dimensional sensing. Texture feature extraction is then performed using a texture separation model. This can be achieved by inputting the projection imaging data to be processed into the texture separation model and outputting the separation result of the wall background texture and the projected content. In a specific embodiment, this operation can be achieved by directly outputting the separation map using an end-to-end neural network or by using an iterative optimization method to alternately estimate the background and foreground, thereby effectively decoupling the inherent wall interference from the projected content and providing clean features for anchor point positioning. Determining the locations of multiple key texture distortion anchor points can be achieved by detecting the center or extreme points of significantly distorted regions on the separated texture feature map. Furthermore, this operation can be accomplished by detecting curvature extreme points based on the Hessian matrix or by locating distortion-sensitive regions using a saliency map, thereby obtaining spatial reference points characterizing the influence of the wall's microstructure.

[0027] Step S300: Based on the positions of multiple key texture distortion anchor points, the non-flat wall surface is divided into distortion transmission areas to determine multiple texture distortion transmission areas.

[0028] The texture distortion propagation region can be a sub-region of the wall surface with similar distortion propagation characteristics, divided based on the location of key texture distortion anchor points. This supports independent optimization of projection parameters for different regions, enabling site-specific correction strategies. In an exemplary embodiment, the texture distortion propagation region can be expanded into connected regions centered on anchor points using clustering, graph segmentation, or region growing algorithms, ensuring consistency in distortion patterns within the region. For example, the texture distortion propagation region may include convexity-dominated regions, concave-dominated regions, and regions with drastic texture changes. Splitting the distortion propagation region of an uneven wall surface based on multiple key texture distortion anchor point locations can be achieved by dividing the wall surface regions based on spatial proximity and distortion similarity, with anchor points as the core. Further, this operation can be implemented by using K-means clustering combined with spatial constraints for region division or by using graph cut algorithms to segment connected regions, thereby forming sub-regions with consistent internal distortion characteristics, supporting differentiated processing. Determining multiple texture distortion propagation regions can output the region splitting results, with each region accompanied by its corresponding anchor point and distortion type label, thus providing structured input for multi-region parameter optimization.

[0029] Step S400: Based on multiple texture distortion transmission regions, perform multi-region optimization on the preset projection imaging parameters to determine multiple region projection correction parameters.

[0030] The regional projection correction parameters can be a set of imaging adjustment parameters independently set for each texture distortion transmission region to control the output of the projection device. These parameters dynamically adjust the projection output to compensate for local distortions and solve problems of uneven brightness and focus drift. In one specific embodiment, the regional projection correction parameters can be optimized using algorithms (such as least squares or gradient descent) to fit the optimal geometric mapping, brightness gain, and focus compensation values ​​within a local region. For example, the regional projection correction parameters may include local geometric deformation mapping parameters, regional brightness gain coefficients, and local focus offsets. Multi-region optimization of preset projection imaging parameters based on multiple texture distortion transmission regions can involve independently adjusting the geometric mapping, brightness, and focus parameters within each region to achieve the best effect for the projected content in that region. Furthermore, this operation can be achieved by optimizing geometric parameters using local mesh deformation or by back-calculating the gain coefficient based on the regional average brightness, thereby enabling localized and refined correction and overcoming the limitations of a globally uniform strategy. Determining multiple regional projection correction parameters can involve encapsulating the optimization results into parameter sets corresponding to each region for the projection device to call, thus forming executable correction instructions to drive differentiated output from the projection device.

[0031] Step S500: Based on multiple area projection correction parameters, control the projection device to project an image onto the uneven wall surface and output an enhanced projection image.

[0032] The projection device can be a hardware unit that performs image projection, used to output an enhanced projected image based on regional projection correction parameters. The enhanced projected image can be an optimized image presented on a non-flat wall surface after regional correction, used to achieve high-definition, color-accurate, brightness-uniform, and focus-stable visual output. Controlling the projection device to project images onto the non-flat wall surface based on multiple regional projection correction parameters can be achieved by mapping regional parameters to pixel-level control signals of the projection device, enabling spatially variable image output. In an exemplary embodiment, this operation can be achieved through real-time geometric deformation using a GPU shader or by adjusting regional brightness using a projector's built-in LUT, thereby generating a high-quality image after distortion compensation on the non-flat wall surface. Outputting the enhanced projected image can be achieved by the projection device continuously outputting optimized images after correction, with an industrial camera optionally providing closed-loop feedback, thus presenting a clear, color-accurate, brightness-uniform, and focus-stable visual effect.

[0033] Taking an immersive art exhibition as an example, the image processing-based projection image optimization method in this embodiment can be used in a historical building exhibition hall that retains the original brick wall texture. The projection system needs to accurately project dynamic digital art content onto the uneven wall surface. The system first acquires an initial projection image using an industrial camera and extracts multiple image blocks from it. It then uses a texture separation model to distinguish between the brick joint texture and the art content, identifying key anchor points that cause image stretching due to brick protrusions. Based on this, the wall surface is divided into several texture distortion transmission areas, such as the protruding area on the top of the bricks, the recessed area of ​​the brick joints, and the mortar transition area. The geometric deformation and brightness parameters are independently optimized for each area. For example, inverse compression mapping is applied to the protruding area and brightness gain is increased in the recessed area. Finally, the projection device outputs an enhanced image based on the area correction parameters, so that the digital content is clearly presented while retaining the historical texture of the wall surface, without blurring or color distortion. Moreover, the system can be quickly deployed without the need to install a structured light scanner.

[0034] In one embodiment, reference Figure 2 The original projected images of the sample projected image patch set are obtained according to preset projection imaging parameters. An industrial camera is then used to acquire the projected imaging data to be processed. Texture features are extracted using a texture separation model to determine the locations of several key texture distortion anchor points, including: Step S201: Project the sample projection content onto the target uneven wall surface according to the preset projection imaging parameters, call the industrial camera to complete the acquisition of the original projection image, and obtain the original projection image to be processed. Projecting the sample content onto the target uneven wall surface according to preset projection imaging parameters can be achieved by using a projection device to project a known standard test pattern onto the actual wall surface according to the current configuration. Furthermore, projecting the sample content onto the target uneven wall surface according to preset projection imaging parameters can be achieved by projecting a mixed test pattern containing edges and color blocks, or by projecting a multi-frequency phase-shift fringe pattern, thereby generating an observation scene containing real wall surface interference, providing a basis for data acquisition. Acquiring the original projected image using an industrial camera can be achieved by triggering the industrial camera to capture an image of the current wall projection state. Furthermore, acquiring the original projected image using an industrial camera can be achieved by using synchronous triggering to ensure consistency between projection and acquisition timing, and multi-exposure fusion to improve dynamic range, thereby obtaining real two-dimensional observation data reflecting wall geometry and illumination interference. Obtaining the original projected image to be processed can be achieved by using the image acquired by the industrial camera as input for subsequent processing, thus providing an original data source containing mixed texture information.

[0035] Step S202: Call the texture separation model to decompose the original projection image to be processed, and perform texture weight correction by combining the original standard projection texture reference and the wall inherent texture reference to obtain the corrected wall concave-convex distortion texture component, projection content texture component and ambient light interference texture component. The wall surface unevenness distortion texture component can be a structured interference texture component separated from the original projection image to be processed, caused by the geometric shape of the wall (such as protrusions or depressions). It can be used to identify areas of image stretching, compression, or blurring caused by physical undulations of the wall, supporting the localization of geometric distortion anchor points. In this embodiment, the wall surface unevenness distortion texture component can be output after weight correction by combining the texture separation model with the inherent texture reference of the wall, reflecting the spatial modulation effect of the wall's micro-geometry on the projected content. For example, the wall surface unevenness distortion texture component can include, but is not limited to, one or more of the following: protrusion-dominated distortion component, depression-dominated distortion component, and composite undulation distortion component. The projection content texture component can be an ideal projection content texture restored from the original projection image to be processed, unaffected by wall interference. It can be used as a correction target reference to evaluate the degree of distortion and verify the separation accuracy, avoiding misjudging wall imperfections as image details. In an exemplary embodiment, the projection content texture component can be extracted during the texture separation process, guided by the original standard projection texture reference, and after suppressing wall and lighting interference through weight correction. For example, the texture components of the projected content may include, but are not limited to, edge structure components, smooth color block components, high-frequency texture components, etc.

[0036] Ambient lighting interference texture components can be brightness and color shift interference components separated from the original projected image to be processed, caused by uneven illumination or reflection of ambient light. They can be used to identify key anchor points in areas of light offset, supporting local compensation for brightness and color. In one specific embodiment, the ambient lighting interference texture components can be modeled using the difference between the inherent texture reference of the wall surface and the standard projection reference, and the influence of non-projection light sources can be isolated through texture weight correction. For example, ambient lighting interference texture components may include, but are not limited to, ambient diffuse reflection interference components, directional specular highlight interference components, and shadow occlusion interference components. The original standard projection texture reference can be a theoretical texture representation of a standard test pattern or content projected under ideal planar conditions according to preset projection imaging parameters. It can be used as a content prior in the texture separation process, guiding the model to distinguish between real projected content and wall interference. In this embodiment, the original standard projection texture reference can be pre-stored in a digital image template in the system, containing known geometric structures and color distributions. Furthermore, the original standard projection texture reference can collaborate with the texture separation model to provide content semantic constraints during the weight correction stage, improving the accuracy of restoring the texture components of the projected content.

[0037] The inherent texture reference of the wall surface can be a background texture reference image obtained by acquiring and processing the target non-flat wall surface in a non-projected state. It can be used as prior knowledge of wall surface interference to guide the texture separation model to identify and strip away the inherent characteristics of the wall surface. In an exemplary embodiment, the inherent texture reference of the wall surface can be generated by acquiring a wall image with an industrial camera when projection is off, and then performing denoising and illumination normalization processing. Furthermore, the inherent texture reference of the wall surface can work in conjunction with the texture separation model to suppress the interference of wall surface material clutter on content extraction during the weight correction stage. Texture weight correction can be a mechanism that dynamically adjusts the contribution of each component based on the original standard projected texture reference and the inherent texture reference of the wall surface during the texture separation process. It can be used to improve the accuracy of triple texture component decomposition, avoid component aliasing, and enhance the reliability of subsequent anchor point recognition. In a specific embodiment, texture weight correction can adaptively allocate the weights of the three components—wall surface, content, and illumination—by calculating the similarity or residual between the input image and the two references.

[0038] Applying a texture separation model to decompose the original projected image can be achieved by inputting the original projected image into a trained texture separation model and outputting preliminary component estimates. Further, texture decomposition of the original projected image can be achieved by using an end-to-end network to output three types of components at once, separating the wall surface, lighting, and content in stages, thus achieving preliminary texture decoupling and providing a basis for weight correction. Texture weight correction can be performed by combining the original standard projected texture benchmark and the wall's inherent texture benchmark, which can be achieved by using the two benchmarks to perform confidence weighting or residual optimization on the preliminary separation results. Further, texture weight correction can be achieved by dynamically adjusting the weights of each pixel component based on a similarity metric and iteratively optimizing to minimize the overall error with the benchmark, thereby improving the separation purity of the three texture components and reducing component leakage. The corrected texture components for wall surface unevenness distortion, projected content, and ambient lighting interference can be obtained. This can be output as a weighted, three-channel or three sets of independent texture maps, thus forming a clear and non-interfering component representation to support accurate anchor point recognition. For example, let's assume... The texture decomposition model decomposes the raw projection image acquired by the industrial camera into three components:

[0039] In the formula, The pixel value at pixel coordinates (x, y) of the original projected image to be processed; The texture component representing wall surface bump distortion represents the geometric structure and material texture of the wall surface and is the main basis for generating image distortion. The texture component of the projected content is the pure image content that the projection device originally intended to project. This is the ambient light interference texture component, used to eliminate the unintended effects of ambient light on color and brightness; For random noise; , as well as These are the weight correction coefficients for the wall surface unevenness distortion texture component, the projected content texture component, and the ambient light interference texture component, respectively. They are dynamically adjusted based on the original standard projected texture reference and the wall surface inherent texture reference, and are used to enhance effective features and suppress interference, such as wall surface unevenness and suppressing interference such as ambient light.

[0040] Step S203: Traverse the concave-convex distortion texture component, the projected content texture component, and the ambient lighting interference texture component to identify the key texture distortion anchor point positions, and obtain the key texture distortion anchor point position sets for the convex region, the concave region, and the lighting offset region. The identification of key texture distortion anchor points can be a process of analyzing various texture components to locate representative local distortion feature points. This can be used to generate a categorized set of anchor points, clearly distinguishing the spatial locations of geometric distortion and illumination disturbance. In one specific embodiment, key texture distortion anchor point identification can detect curvature extrema in the wall surface concavity / convexity distortion texture component and gradient abrupt change regions in the ambient illumination disturbance texture component. The set of key texture distortion anchor points for raised areas can be the set of anchor point locations corresponding to raised parts of the wall surface identified in the wall surface concavity / convexity distortion texture component. This can be used to identify areas of the image that are locally compressed or focused, for subsequent geometric and optical parameter correction. In an exemplary embodiment, obtaining the set of key texture distortion anchor points for raised areas can be achieved by filtering anchor points corresponding to positive curvature or local maxima in the concavity / convexity distortion component, thereby clearly identifying the compression distortion area caused by the wall surface protrusion. The set of key texture distortion anchor points in the recessed area can be the set of anchor point locations corresponding to the recessed parts of the wall surface identified in the wall surface concavity-convexity distortion texture components. This set can be used to identify areas of the image that are locally stretched or have reduced brightness, guiding inverse deformation and gain compensation. In one specific embodiment, obtaining the set of key texture distortion anchor points in the recessed area can involve filtering anchor points corresponding to negative curvature or local minima in the concavity-convexity distortion components, thereby clearly identifying the stretching distortion areas caused by the wall surface concavity.

[0041] The set of key texture distortion anchor points in the illumination offset region can be a set of anchor point locations identified in the ambient lighting interference texture component that are affected by non-uniform ambient light. This set can be used to identify areas of color distortion or abnormal brightness for local white balance and brightness correction. In one embodiment, obtaining the set of key texture distortion anchor point locations in the illumination offset region can be achieved by locating the center of areas with significant brightness or chromaticity deviations in the ambient lighting interference texture component, thereby identifying areas severely affected by ambient light interference for color and brightness correction. Identifying key texture distortion anchor point locations by traversing the bump distortion texture component, the projection content texture component, and the ambient lighting interference texture component can be achieved by executing a feature detection algorithm on each component to locate points with significant distortion indication significance. Furthermore, identifying key texture distortion anchor point locations by traversing the bump distortion texture component, the projection content texture component, and the ambient lighting interference texture component can be achieved by using a Harris-Laplace detector on the bump component and DoG speckle detection on the illumination component, thereby generating a well-defined set of anchor point candidates that distinguishes between geometric and illumination-related distortions.

[0042] Step S204: Integrate the sets of key texture distortion anchor points in the raised region, the set of key texture distortion anchor points in the recessed region, and the set of key texture distortion anchor points in the lighting offset region to obtain multiple key texture distortion anchor point positions.

[0043] Integrating the sets of key texture distortion anchor points in convex, concave, and illumination-offset regions can be achieved by merging these three sets into a unified spatial point set, labeled with type tags. Furthermore, this integration can be accomplished by deduplicating spatial coordinates while preserving type priority, and constructing an anchor point graph with semantic tags. This results in a complete and clearly categorized set of key texture distortion anchor points. Obtaining multiple key texture distortion anchor point locations can output a list of integrated anchor points for subsequent region segmentation, providing a distortion representation benchmark that combines type information and spatial accuracy.

[0044] Taking a building facade projection show as an example, the projection image optimization method based on image processing in this embodiment can be used for a nighttime projection performance on the exterior wall of a historical building with complex stone texture and window recesses. The system first projects a standard checkerboard pattern, and an industrial camera simultaneously acquires the original projection image to be processed. Then, the texture separation model is called, and the image is triple-decomposed by combining the pre-acquired image of the unprojected wall (the inherent texture reference of the wall) and the ideal checkerboard template (the original standard projection texture reference) to obtain the wall surface unevenness distortion texture component, the projection content texture component, and the ambient light interference texture component. In the unevenness component, the compression anchor points at the stone protrusions and the stretch anchor points at the window frame recesses are identified, and in the illumination component, the high brightness offset anchor points caused by street light illumination are identified. After integrating the three types of anchor points, the system divides the texture distortion transmission area accordingly, and applies reverse stretching to the protruding area, compression compensation to the recessed area, and reduces the gain of the high brightness area. Finally, the dynamic image is accurately fitted and color-consistently presented on the complex facade, relying solely on the industrial camera and requiring no laser scanning.

[0045] In one embodiment, obtaining the set of key texture distortion anchor point locations for the raised region includes: Based on the 3D point cloud structure information of the target non-flat wall surface, obtain the set of geometric prior anchor point positions.

[0046] The 3D point cloud structure information of the target non-flat wall surface can be a set of discrete spatial points obtained from the target non-flat wall surface through sparse or lightweight 3D reconstruction methods. It represents the macroscopic geometric shape of the wall surface and can be used to provide preliminary geometric clues for protruding areas of the wall surface, generating initial anchor point candidates. In an exemplary embodiment, the 3D point cloud structure information of the target non-flat wall surface can be generated by a low-cost depth sensor, structure of motion (SfM), or stereo vision system, without the need for high-precision LiDAR or dense structured light scanning. For example, the 3D point cloud structure information of the target non-flat wall surface can include, but is not limited to, sparse SfM point clouds, monocular depth estimation point clouds, and binocular stereo matching point clouds. The set of geometric prior anchor point positions can be a set of potential protrusion position candidates extracted based on regions with significant changes in curvature or normal vectors in the 3D point cloud structure information. It can be used as a coarse screening anchor point before texture verification, narrowing the scope of subsequent analysis and improving efficiency. Furthermore, the set of geometric prior anchor point positions can be used to calculate the principal curvature or surface normal gradient by performing local surface fitting on the point cloud, screening positive protrusion extreme points. In one specific embodiment, the set of geometric prior anchor points can be coordinated with the bump distortion texture components to integrate geometric priors and actual projection distortion observations during the scoring stage, thereby avoiding purely geometric misjudgments.

[0047] Based on the 3D point cloud structure information of the target non-flat wall surface, a set of geometric prior anchor point locations can be obtained. This can be achieved by performing local geometric analysis on the 3D point cloud, extracting points with maximum curvature or abrupt changes in normal, and projecting them onto the image plane. Furthermore, this operation can be implemented by using principal component analysis (PCA) to estimate the local surface and extracting convex extreme points, and then using point cloud normal clustering to identify convex regions facing the projection direction. This allows for the generation of initial anchor point candidates covering potential convex regions, reducing the risk of missed detections associated with pure texture methods.

[0048] According to the preset distortion evaluation index, the set of geometric prior anchor points is scored by combining the concave and convex distortion texture components to obtain a set of candidate scores for geometric prior anchor points. A preset distortion evaluation index can be an assessment function used to quantify the severity of actual projection distortion at geometric prior anchor points. It can be used to correlate geometric candidate points with the distortion intensity of the real image, achieving a mapping from geometric probability to visual impact. In this embodiment, the preset distortion evaluation index can be constructed based on the statistical or structural features of the concave-convex distortion texture components in the neighborhood of the anchor point. For example, the preset distortion evaluation index can include, but is not limited to, local gradient magnitude indices, structural similarity loss indices, and texture energy concentration indices. According to the preset distortion evaluation index, candidate scoring is performed on the set of geometric prior anchor point locations in conjunction with the concave-convex distortion texture components. This can be achieved by calculating the distortion intensity index value of the concave-convex distortion texture components in the image neighborhood corresponding to each geometric prior anchor point. Further, this operation can be implemented by calculating the L2 norm of the texture gradient in the neighborhood of the anchor point as a score and calculating the SSIM loss with the ideal plane projection as a score, thereby fusing geometric priors and actual observed distortion to improve the correlation between anchor points and real visual impact. For example, suppose... The set of geometric prior anchor points for extracting 3D point cloud structural information, and the comprehensive score of the i-th anchor point. The calculation is as follows:

[0049] In the formula, For geometric prior anchor points The distortion candidate score indicates that the higher the score, the greater the probability that a critical distortion has occurred at that point. Texture offset, measuring The offset distance of the texture at this location in the component relative to the ideal plane reflects the degree of physical deformation; The peak value of the gradient distortion is used to calculate the magnitude of the image gradient at that location. Capture the dramatic edge changes caused by the unevenness of the wall surface; For spatial consistency, it is used to measure the difference between the features of a point and the features of points in its neighborhood, and is used to remove unstructured isolated noise points; , as well as Preset weighting coefficients for texture offset, gradient distortion peak, and spatial consistency are used to balance the importance of different evaluation metrics. The candidate score set for geometric prior anchor points can be a set of numerical confidence scores calculated for each geometric prior anchor point based on preset distortion evaluation metrics. This set reflects the likelihood of each candidate point causing visual distortion in actual projection, supporting subsequent screening. Obtaining the candidate score set for geometric prior anchor points can output structured data of each anchor point and its corresponding score, thus providing a quantitative basis for subsequent spatial screening.

[0050] The set of candidate scores for geometric prior anchor point positions is filtered based on a preset pixel spacing to determine the set of key texture distortion anchor point positions in the raised area.

[0051] The preset pixel spacing can be a minimum anchor point distance threshold set on the image plane to control spatial distribution density. This prevents excessive anchor point clustering, ensuring sufficient spatial resolution and manageable computational overhead for subsequent region segmentation. In a specific embodiment, the preset pixel spacing can include, but is not limited to, spacing for high-density display scenes, spacing for large-scale building projections, and robust spacing for dynamic environments. Filtering the candidate score set of geometric prior anchor point positions based on the preset pixel spacing can be achieved by performing non-maximum suppression or greedy clustering on high-scoring anchor points in the image coordinate system, retaining points with a spacing greater than the threshold. Further, this operation can be implemented by traversing in descending order of score, eliminating subsequent points with a distance less than the preset pixel spacing from the selected points, and using grid partitioning to retain the highest-scoring point in each region. This optimizes the spatial distribution of anchor points, avoids redundancy, and ensures the rationality of region segmentation and computational efficiency. Determining the set of key texture distortion anchor point positions for protruding regions can be achieved by outputting the final anchor point set after geometric-texture joint verification and spatial redundancy removal. This provides a high-confidence, reasonably distributed protrusion distortion localization benchmark, supporting subsequent accurate correction.

[0052] Taking immersive projection of cultural heritage as an example, the image processing-based projection image optimization method in this embodiment can be used for digital restoration projection within an ancient temple with relief decorations. The system first reconstructs a sparse 3D point cloud of the wall surface using a sequence of images captured by a mobile phone, extracting geometric prior anchor points for the raised areas of the relief. Then, combining the bump distortion texture components output by the texture separation model, it calculates the gradient energy of the neighborhood of each anchor point as a distortion score. Finally, high-scoring anchor points are selected at 30-pixel intervals to form a set of key texture distortion anchor point locations in the raised areas. This set accurately identifies the areas where the high points of the relief cause image compression, ensuring that subsequent correction preserves the texture of the cultural relic while ensuring the clarity and distortion-free overlay of the historical image. No professional 3D scanning equipment was used throughout the entire process.

[0053] In one embodiment, the preset distortion evaluation metrics include at least texture offset, gradient distortion peak, and spatial consistency.

[0054] Texture offset can be a quantitative indicator of the geometric displacement of the projected content in a local area of ​​an uneven wall surface relative to the ideal planar projection position. It can be used to reflect the intensity of geometric distortion such as image stretching and compression caused by wall protrusions or depressions, and to assess the degree of content distortion at the anchor point. In an exemplary embodiment, texture offset can be calculated by comparing the feature point or structural matching deviation between the concave-convex distortion texture component and the original standard projection texture reference in the corresponding area. Gradient distortion peak can be an abnormal extreme value of the image gradient amplitude detected in the distorted area, characterizing the degree of excessive sharpening or blurring attenuation of edge or structural details. It can be used to identify highly sensitive distortion points that significantly affect visual clarity, and to assist in determining whether focusing or sharpening compensation is needed. For example, gradient distortion peak can be obtained by extracting local maxima or residual peak values ​​with the ideal gradient distribution after performing gradient calculations on the concave-convex distortion texture component. Spatial consistency can be a measure of the smoothness and structural coherence of the distortion response change between adjacent pixels or neighborhoods within a local area. It can be used to suppress false detections caused by noise, isolated artifacts, or transient interference, and to ensure that the selected anchor point has real physical meaning and regional representativeness. In this embodiment, spatial consistency can be obtained by calculating the variance, autocorrelation coefficient, or graph smoothing energy of distortion indices (such as texture offset or gradient) within the neighborhood of the anchor point.

[0055] Taking the projection restoration of museum artifacts as an example, the projection image optimization method based on image processing in this embodiment can be used to dynamically restore the patterns within a display case of a bronze artifact with a complex surface undulation. The system calculates the texture offset at candidate anchor points to assess the degree of pattern misalignment, while simultaneously detecting gradient distortion peaks to identify edge blurring areas caused by copper rust protrusions, and verifying the spatial consistency around these areas to eliminate isolated artifacts caused by dust reflections. Finally, only anchor points with significant values ​​in all three indicators are retained as the set of key texture distortion anchor points for protruding areas, ensuring that the correction parameters are accurately applied to the actual deformed areas, making the restored animation seamlessly integrated with the artifact surface and with clear details.

[0056] In one embodiment, the non-flat wall surface is segmented into distortion transmission regions based on the locations of multiple key texture distortion anchor points, determining multiple texture distortion transmission regions, including: A preset standard projection is used to test the projection on the non-flat target wall surface. Textures are collected at multiple key texture distortion anchor point positions to obtain texture sequences at multiple key texture distortion anchor point positions. The preset standard projection can be a set of standardized test images with known content used for trial projection. This actively stimulates the distortion response of the wall surface, providing controllable and repeatable observation conditions. Trial projection, before formal calibration, involves projecting the preset standard projection onto the target non-flat wall surface to collect response data, generating benchmark observation data for texture acquisition and distortion modeling. The texture sequence of multiple key texture distortion anchor points can be a set of local texture responses collected during trial projection for each key texture distortion anchor point location as the content of different standard projections changes. This reflects the dynamic modulation effect of the wall surface's microstructure on the projection signal, supporting distortion trend extraction.

[0057] Trial projection of a non-flat target wall surface under a pre-defined standard projection can be achieved by sequentially projecting a set of standardized test patterns onto the wall surface, while an industrial camera simultaneously records the response images. Further, trial projection of a non-flat target wall surface under a pre-defined standard projection can be achieved by projecting a sequence of multi-directional stripe patterns or a combination of grayscale steps and color blocks, thereby actively inducing and recording the distortion performance of the wall surface under controlled input, establishing an input-response mapping foundation. Texture acquisition of multiple key texture distortion anchor point locations can be achieved by cropping local image patches around each anchor point from images captured by the industrial camera. For example, texture acquisition of multiple key texture distortion anchor point locations can be achieved by center-cropping a fixed window size and adjusting the size of the adaptive window according to the local gradient, thereby obtaining dynamic texture response data reflecting the local modulation characteristics of the wall surface. Obtaining a texture sequence of multiple key texture distortion anchor point locations can be achieved by organizing local images of the same anchor point in different trial projection frames into a sequence according to time or content order, forming structured data that can be used for time-series or cross-sample analysis.

[0058] Distortion feature analysis was performed on the texture sequences at multiple key texture distortion anchor points to obtain distortion trend features at these locations. The distortion trend features can be quantified attributes representing the direction and intensity of local distortion evolution, parsed from the texture sequences of key texture distortion anchor points. These attributes are used to transform the original image data into a physically meaningful distortion semantic description, supporting subsequent coupling relationship modeling. In an exemplary embodiment, distortion trend features can be extracted through temporal or cross-sample analysis of the texture responses of the same anchor point under different projection contents, yielding interpretable features such as the main stretching direction, blur kernel scale, and brightness attenuation gradient. The distortion trend features of multiple key texture distortion anchor points can be a set of corresponding distortion evolution attributes parsed from the texture sequences of each anchor point, forming the basic input for distortion coupling analysis. Distortion feature parsing of the texture sequences at multiple key texture distortion anchor points can be achieved by applying image analysis algorithms to extract quantifiable indicators related to distortion from the texture sequences. In a specific embodiment, distortion feature parsing of the texture sequences at multiple key texture distortion anchor points can be achieved by calculating the deformation direction using optical flow estimation and estimating the blur degree using frequency domain analysis, thereby transforming the original texture data into semantically meaningful distortion trend features. Obtaining distortion trend features of multiple key texture distortion anchor points can be achieved by outputting distortion trend feature vectors or maps corresponding to each anchor point, thus providing structured input for distortion coupling analysis.

[0059] Distortion coupling analysis is performed based on the distortion trend characteristics of multiple key texture distortion anchor points to determine the distortion coupling trend characteristics of multiple key texture distortion anchor points. The distortion coupling trend feature can be a comprehensive feature reflecting the mutual influence of distortion propagation, obtained through spatial correlation analysis based on the distortion trend features of multiple key texture distortion anchor points. This feature is used to reveal the superposition and propagation mechanism of nonlinear distortion caused by the wall's micro-morphology within a local area, providing a basis for reasonable area division. In this embodiment, the distortion coupling trend feature can employ graph neural networks, covariance analysis, or neighborhood correlation modeling methods to quantify the cooperative change law of distortion patterns between adjacent anchor points. The distortion coupling trend feature of multiple key texture distortion anchor points can be a feature set describing the distortion interaction relationship between anchor points obtained after coupling analysis, used to guide the boundary delineation of the texture distortion transmission area. Distortion coupling analysis based on the distortion trend features of multiple key texture distortion anchor points can analyze the similarity, propagation direction, or mutual reinforcement relationship of distortion trends between adjacent or functionally related anchor points. Furthermore, distortion coupling analysis based on the distortion trend characteristics of multiple key texture distortion anchor points can be achieved by constructing an anchor point relationship graph, performing graph convolution, and calculating the local covariance matrix to identify coupling patterns. This reveals the nonlinear superposition and propagation mechanism of distortion in local areas of the wall surface. Determining the distortion coupling trend characteristics of multiple key texture distortion anchor points can output a comprehensive feature representation after coupling analysis, thus forming the basis for region division.

[0060] Based on the distortion coupling trend characteristics of multiple key texture distortion anchor points, the texture distortion propagation region is split to determine multiple texture distortion propagation regions, where each texture distortion propagation region includes distortion integrated texture features.

[0061] The distortion-integrated texture feature can be a unified texture representation that integrates the distortion coupling trends of all anchor points within each texture distortion propagation region. This serves as input for optimizing region-level correction parameters, ensuring a high degree of matching between the compensation strategy and the local physical environment. In one specific embodiment, the distortion-integrated texture feature can generate a feature vector or map representing the overall distortion response mode of the region by weighted aggregation or embedding fusion of the coupling trend features of anchor points within the region. The texture distortion propagation region segmentation based on the distortion coupling trend features of multiple key texture distortion anchor points can be achieved by dividing the wall into several connected sub-regions based on the similarity of the coupling trend features. Furthermore, the segmentation of the texture distortion propagation region based on the distortion coupling trend features of multiple key texture distortion anchor points can be achieved by using spectral clustering combined with spatial constraints and a region growing algorithm to expand with highly coupled anchor points as the core. This ensures that the distortion propagation law within each region is consistent and the physical meaning of the boundaries is clear. Determining multiple texture distortion propagation regions can output the final region segmentation result, with each region accompanied by its distortion-integrated texture feature, providing refined spatial units for subsequent multi-region parameter optimization.

[0062] Taking a building facade projection show as an example, the image processing-based projection image optimization method in this embodiment can be used to implement a nighttime projection show on the exterior wall of a historical building whose surface is made of irregularly pieced stone. The system first projects a set of standard stripes and color block patterns for trial projection, and an industrial camera simultaneously captures the wall response; extracts texture sequences around the identified key texture distortion anchor points (such as stone joints and protruding edges); through analysis, it is found that the top of a certain protruding stone shows obvious vertical stretching under the horizontal stripes, while the adjacent concave area is accompanied by brightness attenuation, and the distortion trends of the two are strongly coupled; based on this, the system divides the stone and its surroundings into a texture distortion transmission area and generates distortion integrated texture features containing stretching-attenuation coupling characteristics; in the subsequent correction stage, reverse geometric compression and local gain compensation are applied to this area simultaneously, so that the dynamic image maintains accurate shape and consistent color on the complex stone surface, avoiding edge misalignment or brightness jumps caused by ignoring the coupling effect in traditional methods.

[0063] In one embodiment, distortion coupling analysis is performed based on the distortion trend characteristics of multiple key texture distortion anchor point positions to determine the distortion coupling trend characteristics of multiple key texture distortion anchor point positions, including: Based on the pixel spacing distance between multiple key texture distortion anchor points, key texture distortion anchor points whose distance to multiple key texture distortion anchor points is within a preset distortion distance threshold are added to their neighborhood to obtain the distortion trend feature neighborhood of multiple key texture distortion anchor points. Based on the distortion trend feature neighborhood of multiple key texture distortion anchor points, distortion coupling analysis is performed on the distortion trend features of multiple key texture distortion anchor points to obtain the distortion coupling trend features of multiple key texture distortion anchor points.

[0064] The preset distortion distance threshold can be the maximum pixel interval distance used to determine whether there is a potential distortion propagation association between two key texture distortion anchor points. It can control the scope of neighborhood construction, ensuring the inclusion of physically related anchor points and excluding irrelevant interference from distant points. In this embodiment, the preset distortion distance threshold can be set based on the typical microstructure scale of the wall or empirical priors, or it can be dynamically adjusted based on the overall anchor point density using an adaptive algorithm. The distortion trend feature neighborhood of a key texture distortion anchor point can be a local set centered on a key texture distortion anchor point, containing all other anchor points whose pixel interval distance does not exceed the preset distortion distance threshold, along with the distortion trend features of each member. This can be used to provide local contextual information for distortion coupling analysis and explicitly model the spatial correlation of the wall distortion field. In an exemplary embodiment, the distortion trend feature neighborhood of a key texture distortion anchor point can be formed by calculating the pairwise Euclidean distance based on the anchor point coordinates and filtering anchor points that satisfy the distance constraints to form an adjacency list or graph structure. Furthermore, the distortion trend feature neighborhood of the key texture distortion anchor point location can include, but is not limited to, spherical neighborhood, anisotropic elliptical neighborhood, topologically connected neighborhood, etc.

[0065] Based on the pixel spacing between multiple key texture distortion anchor points, key texture distortion anchor points whose distance to these anchor points is within a preset distortion distance threshold are added to their neighborhoods. This can be achieved by calculating the pixel Euclidean distance between any two anchor points in the image coordinate system; if the distance is less than or equal to the preset distortion distance threshold, the other anchor point is included in its neighborhood set. Furthermore, this operation can be accelerated by constructing a kd-tree to speed up nearest neighbor search and by using a sliding window for local scanning to reduce computation, thereby establishing a spatial association structure reflecting the continuity of the local wall morphology and identifying potential distortion propagation paths. Obtaining the distortion trend feature neighborhoods of multiple key texture distortion anchor points can be achieved by outputting the neighborhood members corresponding to each anchor point and their respective distortion trend features, thus forming a structured input that can be used for context-aware coupling analysis.

[0066] Based on the distortion trend feature neighborhoods of multiple key texture distortion anchor points, distortion coupling analysis is performed on the distortion trend features of these anchor points. This can be achieved by jointly modeling the distortion trend features of each anchor point and its neighbors within the neighborhood, and evaluating collaborative change patterns. In one specific embodiment, this operation can be implemented by calculating the covariance matrix of the feature vectors within the neighborhood to measure the coupling strength and using a graph attention mechanism to weighted aggregate neighbor information. This allows for the extraction of a comprehensive distortion coupling trend that integrates local context, overcoming the limitations of isolated point analysis. Obtaining the distortion coupling trend features of multiple key texture distortion anchor points can output an enhanced coupling feature representation of each anchor point obtained after neighborhood coupling analysis, thus providing a criterion with spatial consistency and physical rationality for subsequent region segmentation.

[0067] Taking the projection onto a curved rock wall in an immersive digital exhibition hall as an example, the image processing-based projection image optimization method in this embodiment can be to deploy a projection system on an arc-shaped exhibition wall made of natural rock. The system has identified dozens of key texture distortion anchor points, distributed in rock fissures, protrusions, and weathering pits. A preset distortion distance threshold of 120 pixels is set, corresponding to a physical distance of about 8 centimeters from the wall surface. For each anchor point, the system automatically includes other anchor points within its 120-pixel range into its neighborhood. For example, the neighborhood of a rock fissure anchor point contains three adjacent micro-protrusions. Subsequently, the system analyzes the distortion trend characteristics (such as stretching direction and blurring degree) of the anchor point and its neighbors, and finds that its stretching principal axis is consistent with the height of the adjacent protrusions, indicating the existence of a common geometric deformation source. The distortion coupling trend characteristics generated accordingly strengthen the overall deformation semantics of the area. Finally, the region division classifies these strongly coupled anchor points into the same texture distortion transmission area, and a uniform reverse bending mapping is applied during correction, so that the projected dynamic water flow image is smooth and continuous on the complex rock surface, without breaks or distortions.

[0068] In one embodiment, based on the distortion trend feature neighborhood of multiple key texture distortion anchor point positions, distortion coupling analysis is performed on the distortion trend features of the multiple key texture distortion anchor point positions to obtain the distortion coupling trend features of the multiple key texture distortion anchor point positions, including: Calculate the feature similarity set between the neighborhood of the distortion trend feature of multiple key texture distortion anchor points and the corresponding distortion trend feature of multiple key texture distortion anchor points, and calculate the mean to obtain multiple feature similarity mean sets. Normalize the multiple feature similarity mean sets to construct multiple coupling analysis vectors, and call the convolutional network to analyze the distortion trend features of multiple coupling analysis vectors and multiple key texture distortion anchor points to obtain the distortion coupling trend features of multiple key texture distortion anchor points.

[0069] The feature similarity set can be a set of similarity measures calculated between the distortion trend features of a key texture distortion anchor point and the corresponding features of all other anchor points in its neighborhood. This set can be used to quantify the consistency or difference in distortion patterns among anchor points within a local region, providing raw data for coupling strength assessment. In this embodiment, the feature similarity set can be obtained by comparing the distortion trend feature vectors of the central anchor point and its neighbors pairwise using measurement methods such as cosine similarity, inverse Euclidean distance, or correlation coefficient. Calculating the feature similarity set between the neighborhood of the distortion trend features of multiple key texture distortion anchor points and the corresponding distortion trend features of multiple key texture distortion anchor points can be achieved by performing pairwise similarity calculations between each anchor point's own distortion trend features and the features of each member in its neighborhood. For example, this operation can be implemented by using cosine similarity to measure directional consistency and using a Gaussian kernel function to calculate weighted similarity, thereby transforming geometric proximity into a quantitative basis for semantic coupling strength.

[0070] The feature similarity mean set can be a scalar or vector set obtained by arithmetically averaging the feature similarity sets corresponding to each anchor point. It can be used to compress neighborhood similarity information, forming a concise coupling strength index. In an exemplary embodiment, the feature similarity mean set is obtained by averaging all similarity values ​​within the neighborhood of the same anchor point, reflecting the overall coupling level between the anchor point and its local environment. Calculating the mean of the feature similarity sets to obtain multiple feature similarity mean sets can be achieved by performing an arithmetic mean operation on the similarity set of each anchor point. In a specific embodiment, this operation can be achieved by calculating a simple average of all neighborhood similarities or a distance-weighted average, thereby refining the overall level of local coupling and reducing noise interference. For example, for anchor points... Define the neighborhood set within its distortion distance threshold. The average feature similarity between it and its neighbors for:

[0071] In the formula, anchor point The neighborhood set; The number of anchor points within the neighborhood; as well as These represent anchor points. and its neighboring points The distortion trend feature vector; Similarity calculation functions, such as cosine similarity, are used to measure the degree of similarity in distortion trends between two points; The mean value of feature similarity reflects the consistency of distortion features in a local area. The larger the value, the more uniform the distortion characteristics in that area.

[0072] The coupling analysis vector can be a standardized numerical vector composed of a set of normalized feature similarity mean values. It can be used to transform spatial coupling relationships into structured, scale-consistent numerical representations for processing by convolutional networks. In this embodiment, the coupling analysis vector is obtained by performing min-max normalization or Z-score normalization on the feature similarity mean values ​​to adapt them to the neural network input range. Normalizing multiple sets of feature similarity mean values ​​can map the values ​​in the mean sets to a unified interval (e.g., 0-1) or a standard distribution. Furthermore, this operation can be achieved by applying min-max normalization and using Z-score normalization, thereby eliminating dimensional differences and improving the stability of subsequent neural network training. Constructing multiple coupling analysis vectors can be achieved by organizing the normalized features into fixed-dimensional vectors, thus forming a structured input adapted to the deep learning model. The convolutional network can be a deep neural network model used to jointly analyze the coupling analysis vector and distortion trend features. It can be used to automatically learn the nonlinear propagation and superposition mechanism of wall distortion in space, generating highly semantic distortion coupling trend features. In one specific embodiment, the convolutional network employs a stacked structure of one-dimensional or two-dimensional convolutional layers to extract local patterns and nonlinearly combine input features, outputting an enhanced representation of coupling trends. For example, the convolutional network may include, but is not limited to, a one-dimensional CNN coupling analyzer, a graph convolutional network, a lightweight MobileNet variant, etc.

[0073] The convolutional network is invoked to analyze the distortion trend features of multiple coupling analysis vectors and multiple key texture distortion anchor points. This can be achieved by concatenating or fusing the coupling analysis vectors with the original distortion trend features and then inputting them into a pre-trained or online-learned convolutional network. Furthermore, this operation can be achieved by using one-dimensional convolution to process sequential features and employing an attention mechanism to dynamically fuse the two types of inputs, thereby automatically modeling the spatial coupling rules of distortion in complex walls through end-to-end learning. Obtaining the distortion coupling trend features of multiple key texture distortion anchor points can be achieved by outputting the activation result of the last layer of the convolutional network as an enhanced coupling feature, thus generating a region partitioning criterion with high semantic expressiveness and spatial consistency.

[0074] For example, in the scenario of projecting onto curved glass inside a museum artifact display case, the image processing-based projection image optimization method in this embodiment can project narration animations onto the inner wall of a precious artifact display case covered with curved anti-glare glass. The system has identified multiple key texture distortion anchor points located where the glass curvature changes drastically. For each anchor point, the cosine similarity between its distortion trend features (such as blur direction and brightness gradient) and those of other anchor points in its neighborhood is calculated to form a feature similarity set; then, the average value is calculated to obtain the overall coupling level of the anchor point, and a coupling analysis vector is constructed by normalization. This vector, along with the original distortion trend features, is input into a lightweight one-dimensional convolutional network. The network automatically learns that in the convex area of ​​the glass, adjacent anchor points, although close in distance, have opposite distortion directions due to abrupt changes in the refraction angle, resulting in low similarity; while in the smooth transition area, they are highly consistent. The final output distortion coupling trend features accurately distinguish between these two physical mechanisms. Based on this, the system classifies the highly coupled areas as the same texture distortion transmission area, and applies continuous deformation compensation during correction, making the text narration clear and readable on the curved glass without ghosting or distortion.

[0075] In one embodiment, multi-region optimization of preset projection imaging parameters is performed based on multiple texture distortion transmission regions to determine multiple region projection correction parameters, including: Based on the distortion-integrated texture features corresponding to multiple texture distortion transmission regions, the region correction optimizer is called to perform analysis and determine multiple initial region projection correction parameters. Neighborhood interference analysis was performed on multiple initial regional projection correction parameters, and the parameters were optimized based on the analysis results to obtain multiple regional projection correction parameters.

[0076] The region correction optimizer can be an algorithm module or computational unit used to generate corresponding region projection correction parameters based on the distortion-integrated texture features. It can be used to automatically convert local distortion characteristics of the wall surface into an initial correction strategy. In this embodiment, the region correction optimizer can map the distortion-integrated texture features to a combination of parameters such as geometric deformation, brightness gain, and focus shift, based on a physical imaging model or a data-driven method. For example, the region correction optimizer can include, but is not limited to, optimizers based on physical optics models, optimizers based on deep learning regression, and hybrid analytical-learning optimizers. The initial region projection correction parameters can be correction parameters independently generated by the region correction optimizer based on the distortion-integrated texture features of a single texture distortion propagation region, without neighborhood coordination. These parameters can reflect the optimal projection compensation strategy within the region, but may be discontinuous with other regions at the boundaries. Based on the distortion-integrated texture features corresponding to multiple texture distortion propagation regions, the region correction optimizer can be invoked for analysis. This can involve inputting the distortion-integrated texture features of each region into the region correction optimizer and performing parameter mapping calculations. Furthermore, this operation can be achieved by using a pre-trained neural network to regress the correction parameters and analyzing the required compensation amount based on a reverse optics model, thereby generating initial correction parameters that accurately match the local physical characteristics of each region. Determining multiple initial region projection correction parameters can be achieved by outputting the calculation results of the region correction optimizer, forming a parameter set that corresponds one-to-one with each texture distortion transmission region, thus establishing a basis for region-specific correction.

[0077] Neighborhood interference analysis can be a process of quantitatively evaluating the visual or optical discontinuities caused by the initial correction parameters of adjacent texture distortion transmission regions at their boundaries. It can be used to identify the risk of boundary artifacts caused by independent optimization, providing a basis for collaborative parameter optimization. In an exemplary embodiment, neighborhood interference analysis can construct an interference metric function by calculating indicators such as brightness gradient, color difference, and geometric mapping offset on both sides of the region boundary. Exemplarily, neighborhood interference analysis can include, but is not limited to, brightness jump interference analysis, geometric misalignment interference analysis, and chromaticity break interference analysis. Performing neighborhood interference analysis on multiple initial region projection correction parameters can involve traversing all adjacent region pairs and evaluating their visual continuity at the corrected boundary. Further, this operation can be achieved by calculating the second derivative of brightness of boundary pixels to detect jumps and comparing the consistency of the Jacobian matrix of adjacent region geometric mappings at the common boundary, thereby quantifying the degree of conflict between correction strategies between regions and locating potential artifact regions. Parameter optimization based on the analysis results can involve introducing smoothing constraints or energy minimization objectives while maintaining correction performance within the region, adjusting the parameters of each region to ensure consistency at the boundary. In one specific embodiment, this operation can be achieved by jointly optimizing all region parameters using graph Laplacian regularization and interpolating and fusing the correction functions of adjacent regions within the boundary neighborhood. This can eliminate brightness jumps, color breaks, or geometric misalignments, achieving a natural transition between regions. Multiple region projection correction parameters are obtained, which can be the final set of correction parameters output after neighborhood coordination. Each parameter satisfies both intra-region optimization and boundary continuity, resulting in a highly consistent and robust correction command that can be used for practical projection control.

[0078] Taking immersive museum artifact projection as an example, the image processing-based projection image optimization method in this embodiment can be used in an exhibition hall where dynamic narration needs to be accurately projected onto a replica of an ancient bronze artifact with a highly undulating surface. The system first divides the surface into multiple texture distortion transmission regions, each with its own distortion-integrated texture features. A region correction optimizer then generates initial correction parameters, such as applying enhanced geometric inverse deformation to raised decorative areas and increasing brightness in recessed inscription areas. However, after initial correction, a clear light-dark boundary is found at the junction of the decorative and inscription areas. The system then initiates neighborhood interference analysis, quantifies the brightness gradient and geometric mapping discontinuity on both sides of the boundary, and uses a boundary smoothing optimization strategy to fine-tune the parameters on both sides, ensuring a natural brightness gradient in the transition area and no misalignment of the text edges. The final projected image clearly presents details on the complex curved surface while maintaining overall visual coherence, and the entire process uses only an industrial camera to acquire two-dimensional images, eliminating the need for 3D scanning equipment.

[0079] Furthermore, to achieve the above objectives, the present invention also provides a projection image optimization device based on image processing, the device comprising: a memory, a processor, and a projection image optimization program based on image processing stored in the memory and executable on the processor, the projection image optimization program based on image processing being configured to implement the steps of the projection image optimization method based on image processing as described above.

[0080] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing an image processing-based projection image optimization program, which, when executed by a processor, implements the steps of the image processing-based projection image optimization method as described above.

[0081] Other embodiments or specific implementations of the projection image optimization device based on image processing described in this invention can be found in the above-described method embodiments, and will not be repeated here.

[0082] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A projection image optimization method based on image processing, characterized in that, The method includes: Extract a predetermined number of image blocks from the non-flat wall projection image of the target projection scene to obtain a sample projection image block set; The original projection images of the sample projection image block set are obtained according to the preset projection imaging parameters, and the projection imaging data to be processed is obtained by calling an industrial camera. The texture features are extracted by combining the texture separation model to determine the positions of multiple key texture distortion anchor points. Based on the locations of the multiple key texture distortion anchor points, the distortion transmission area of ​​the uneven wall surface is divided to determine multiple texture distortion transmission areas; Based on the multiple texture distortion transmission regions, the preset projection imaging parameters are optimized in multiple regions to determine multiple region projection correction parameters. Based on the multiple area projection correction parameters, the projection device is controlled to project an image onto the uneven wall surface and output an enhanced projection image.

2. The projection image optimization method based on image processing as described in claim 1, characterized in that, The original projected images of the sample projected image block set are obtained according to preset projection imaging parameters, and the projected imaging data to be processed is acquired by calling an industrial camera. Texture features are extracted by combining a texture separation model to determine the positions of multiple key texture distortion anchor points, including: The sample projection content is projected onto the target uneven wall surface according to the preset projection imaging parameters, and the industrial camera is called to complete the acquisition of the original projection image to obtain the original projection image to be processed. The texture separation model is called to perform texture decomposition on the original projection image to be processed. The texture weight is corrected by combining the original standard projection texture reference and the inherent texture reference of the wall surface, and the corrected wall surface unevenness distortion texture component, projection content texture component and ambient light interference texture component are obtained. The key texture distortion anchor point positions are identified by traversing the concave-convex distortion texture components, the projected content texture components, and the ambient lighting interference texture components, thereby obtaining the key texture distortion anchor point position sets for the convex region, the concave region, and the lighting offset region. By integrating the sets of key texture distortion anchor points in the raised region, the set of key texture distortion anchor points in the recessed region, and the set of key texture distortion anchor points in the illumination offset region, multiple key texture distortion anchor point positions are obtained.

3. The projection image optimization method based on image processing as described in claim 2, characterized in that, The set of key texture distortion anchor point locations for the raised region includes: Based on the three-dimensional point cloud structure information of the target non-flat wall surface, obtain the set of geometric prior anchor point positions; According to the preset distortion evaluation index, the set of geometric prior anchor points is scored by combining the concave and convex distortion texture components to obtain a set of candidate scores for geometric prior anchor points. The set of candidate scores for geometric prior anchor point positions is filtered based on a preset pixel spacing to determine the set of key texture distortion anchor point positions in the raised area.

4. The projection image optimization method based on image processing as described in claim 3, characterized in that, The preset distortion evaluation indicators include at least texture offset, gradient distortion peak, and spatial consistency.

5. The projection image optimization method based on image processing as described in claim 1, characterized in that, The distortion transmission region of the uneven wall surface is divided based on the positions of the multiple key texture distortion anchor points, and multiple texture distortion transmission regions are determined, including: A preset standard projection is performed on the target non-flat wall surface, and texture acquisition is performed on the multiple key texture distortion anchor point positions to obtain the texture sequence of multiple key texture distortion anchor point positions. Distortion feature analysis is performed on the texture sequences of the multiple key texture distortion anchor points to obtain distortion trend features of the multiple key texture distortion anchor points. Distortion coupling analysis is performed based on the distortion trend characteristics of multiple key texture distortion anchor points to determine the distortion coupling trend characteristics of multiple key texture distortion anchor points. Based on the distortion coupling trend characteristics of the multiple key texture distortion anchor points, the texture distortion propagation region is split to determine the multiple texture distortion propagation regions, wherein each texture distortion propagation region includes distortion integrated texture features.

6. The projection image optimization method based on image processing as described in claim 5, characterized in that, The distortion coupling analysis is performed based on the distortion trend features of multiple key texture distortion anchor point positions to determine the distortion coupling trend features of multiple key texture distortion anchor point positions, including: Based on the pixel spacing distance between the multiple key texture distortion anchor points, the key texture distortion anchor points whose distance to the multiple key texture distortion anchor points is within a preset distortion distance threshold are added to their neighborhood, thereby obtaining the distortion trend feature neighborhood of the multiple key texture distortion anchor points. Based on the distortion trend feature neighborhood of the multiple key texture distortion anchor points, distortion coupling analysis is performed on the distortion trend features of the multiple key texture distortion anchor points to obtain the distortion coupling trend features of the multiple key texture distortion anchor points.

7. The projection image optimization method based on image processing as described in claim 6, characterized in that, The distortion trend feature neighborhood based on the positions of the multiple key texture distortion anchor points is used to perform distortion coupling analysis on the distortion trend features of the multiple key texture distortion anchor point positions to obtain the distortion coupling trend features of the multiple key texture distortion anchor point positions, including: Calculate the feature similarity set between the neighborhood of the distortion trend feature of multiple key texture distortion anchor points and the corresponding distortion trend feature of multiple key texture distortion anchor points, and calculate the mean to obtain multiple feature similarity mean sets. The multiple feature similarity mean sets are normalized respectively to construct multiple coupling analysis vectors. Then, a convolutional network is called to analyze the distortion trend features of the multiple coupling analysis vectors and multiple key texture distortion anchor point positions to obtain the distortion coupling trend features of multiple key texture distortion anchor point positions.

8. The projection image optimization method based on image processing as described in claim 5, characterized in that, The step of optimizing the preset projection imaging parameters based on the multiple texture distortion transmission regions to determine multiple region projection correction parameters includes: Based on the distortion-integrated texture features corresponding to the multiple texture distortion transmission regions, the region correction optimizer is called to perform analysis and determine multiple initial region projection correction parameters. Neighborhood interference analysis is performed on the multiple initial region projection correction parameters, and the parameters are optimized based on the analysis results to obtain the multiple region projection correction parameters.

9. A projection image optimization device based on image processing, characterized in that, The device includes: a memory, a processor, and an image processing-based projection image optimization program stored in the memory and executable on the processor, the image processing-based projection image optimization program being configured to implement the steps of the image processing-based projection image optimization method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an image processing-based projection image optimization program, which, when executed by a processor, implements the steps of the image processing-based projection image optimization method as described in any one of claims 1 to 8.

Images