Projection curtain image reconstruction method and system combined with distortion correction, and medium

Abstract

The invention discloses a projection curtain image reconstruction method and system combined with distortion correction and a medium, and relates to the technical field of image processing, and the method comprises the steps: projecting a calibration image to a curtain through a target projection device, and collecting a projection result through a sensing device group; geometric distortion is identified based on the calibration pattern library, and a screen distortion state is acquired; inputting the distortion state into a multi-channel distortion correction model, and making geometric, color and brightness correction decisions; therefore, the real-time projection image is reconstructed, and the reconstructed image is projected to the curtain again. Therefore, the technical effects of real-time, accurate and comprehensive reconstruction of the projection image and improvement of the image quality and the display stability are achieved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a projection screen image reconstruction method, system and medium combined with distortion correction. Background Art

[0002] Typically, projection equipment projects images onto a projection screen for audience viewing. However, due to the complexity of the projection environment, such as variations in projection angle, uneven screen surfaces, and limitations of optical components, the projected image often exhibits geometric distortion, color deviation, and uneven brightness. These distortions not only affect the visual quality of the image but also severely impact the accuracy of the projected image and the information conveyed, reducing the audience's viewing experience and understanding of the projected content.

[0003] Most existing projection distortion correction technologies only correct for a single type of distortion, such as a simple linear transformation correction for geometric distortion or a rough adjustment of color or brightness uniformity. These technologies are unable to simultaneously and effectively correct the geometric, color, and brightness distortions of the projection screen. In practical applications, even if the geometric distortion is corrected, the image color may deviate and the brightness may be uneven, and vice versa. Moreover, the correction methods used in existing technologies often require manual parameter adjustment, which is a cumbersome process and difficult to precisely control. This makes it difficult to adapt to the diverse needs of different projection environments and screen conditions, resulting in unsatisfactory correction results and an inability to meet user requirements for high-quality projected images. Summary of the Invention

[0004] The present invention provides a projection screen image reconstruction method, system and medium combined with distortion correction to solve the technical problems in the prior art such as single correction dimension, cumbersome operation and unsatisfactory correction effect, thereby achieving real-time, accurate and comprehensive reconstruction of the projected image, and improving the image quality and display stability.

[0005] In a first aspect, the present invention provides a method for reconstructing a projection screen image in combination with distortion correction, wherein the method for reconstructing a projection screen image in combination with distortion correction comprises: The calibration image is projected onto the target projection screen through the target projection device, and the calibration image projection result is collected through the sensing device group.

[0006] Based on a preset calibration pattern library, the calibration image projection result is analyzed to perform geometric distortion identification and obtain geometric distortion status information of the projection screen.

[0007] The distortion state information is input into a pre-built multi-channel distortion correction model to make a distortion correction decision, wherein the multi-channel distortion correction model includes a geometry correction channel, a color correction channel, and a brightness correction channel.

[0008] Based on the distortion correction decision result, the received real-time projection image is reconstructed, and the real-time reconstructed image obtained through image reconstruction is projected onto the target projection screen through the target projection device.

[0009] In a feasible implementation, projecting a calibration image onto a target projection screen by a target projection device includes: Obtain the physical shape parameters of the target projection screen, the pixel density parameters of the target projection device, and the size parameters of the target projection area.

[0010] Access a preset calibration pattern library, and use the screen shape parameter, pixel density parameter, and projection size parameter as index conditions to match and extract the corresponding standard calibration pattern from the calibration pattern library.

[0011] A calibration image is constructed in combination with the standard calibration pattern, wherein the calibration image includes structured pattern information for geometric calibration.

[0012] The calibration image is projected onto a target projection screen through a target projection device.

[0013] In a feasible implementation, collecting calibration image projection results through a sensing device group includes: A multimodal perception device is deployed in a target projection environment, wherein the multimodal perception device includes a brightness test device for brightness measurement, a chromaticity test device for color detection, and a high-resolution image acquisition device for image acquisition.

[0014] The multimodal sensing device is activated to respectively collect geometric image information, color distribution information, and brightness distribution information of the projection calibration image projected on the target projection screen.

[0015] The collected geometric image information, the color distribution information, and the brightness distribution information are spatially remapped and uniformly converted into an ideal projection screen space coordinate system to form the calibration image projection result of the calibration image.

[0016] In a feasible implementation, based on a preset calibration pattern library, the calibration image projection result is analyzed to perform geometric distortion recognition and obtain the geometric distortion state information of the projection screen, including: Pattern recognition and matching are performed on the calibration image projection result to obtain a set of matching image pairs, wherein the matching image pairs include the standard calibration pattern and the corresponding distortion calibration pattern.

[0017] The set of matching image pairs is traversed to perform geometric deformation recognition, including scaling, rotation, translation, and perspective distortion, to obtain a first recognition result.

[0018] The matching image pair set is traversed to perform brightness consistency evaluation and chromaticity deviation evaluation to obtain a second recognition result.

[0019] Feature extraction is performed on the first recognition result and the second recognition result respectively, and the feature extraction results are fused to generate the geometric distortion state information of the projection screen.

[0020] In a feasible implementation, performing feature extraction on the first recognition result and the second recognition result respectively, and fusing the feature extraction results to generate the geometric distortion parameters of the target projection screen further includes: Primary geometric distortion recognition is performed on the first recognition result, including extracting telescopic features, rotation features, translation features, and perspective deformation features related to the flatness of the curtain.

[0021] Based on the second recognition result, depth correction is performed on the primary geometric distortion recognition result to obtain a distortion depth feature. The depth correction includes estimating the directionality of the physical distortion of the target projection screen based on the brightness and chromaticity deviation information.

[0022] The distortion depth feature and the distortion depth feature are subjected to feature fusion, and a feature fusion result is output as a geometric distortion parameter of the target projection screen.

[0023] In one possible implementation, obtaining a pre-built multi-channel distortion correction model includes: Based on the inverse transformation algorithm, the geometric correction channel in matrix form is defined and constructed.

[0024] Sample color deviation status information is obtained as training data, and a color mapping network based on a machine learning model is constructed and trained, and the output is the color correction channel, wherein the sample color deviation status information includes sample projection device characteristic parameters, sample light incident angle, sample projection color value, and sample color deviation index.

[0025] The sample brightness deviation state information is obtained, and a brightness mapping model is constructed using a function fitting method, wherein the sample brightness deviation state information includes the sample calibration viewing position, the sample light incident angle offset, and the sample brightness attenuation.

[0026] The geometric correction channel, the color correction channel, and the brightness correction channel are integrated to obtain the multi-channel distortion correction model.

[0027] In a feasible implementation, the distortion state information is input into a pre-built multi-channel distortion correction model to make a distortion correction decision, including: The distortion state information is input into the geometric correction channel to perform channel initialization, and a geometric correction vector is calculated based on the initialized geometric correction channel to obtain a geometric correction decision matrix.

[0028] According to the geometric correction decision matrix and the distortion state information, derived distortion points are located to obtain a derived distortion point set.

[0029] A mapping relationship between the derived distortion point set and the distortion state information is established, and derived distortion state parameters are extracted.

[0030] The derived distortion state parameters are input to the color correction channel and the brightness correction channel respectively, a color correction vector and a brightness correction vector are calculated, and a color correction decision matrix and a brightness correction decision matrix are constructed accordingly.

[0031] The geometric correction decision matrix, the color correction decision matrix, and the brightness correction decision matrix are output as the distortion correction decision result.

[0032] In a feasible implementation, the inverse transformation algorithm includes at least one of a homography matrix inverse transformation, an affine inverse transformation, and a perspective inverse transformation.

[0033] In a second aspect, the present invention further provides a projection screen image reconstruction system combined with distortion correction, wherein the projection screen image reconstruction system combined with distortion correction comprises: The calibration image projection and acquisition module is used to project the calibration image onto the target projection screen through the target projection device, and to acquire the calibration image projection results through the sensing device group.

[0034] The geometric distortion recognition module is used to analyze the calibration image projection result based on a preset calibration pattern library to perform geometric distortion recognition and obtain geometric distortion status information of the projection screen.

[0035] The distortion correction decision module is used to input the distortion state information into a pre-built multi-channel distortion correction model to make a distortion correction decision, wherein the multi-channel distortion correction model includes a geometric correction channel, a color correction channel and a brightness correction channel.

[0036] The image reconstruction and real-time projection module is used to reconstruct the received real-time projection image based on the distortion correction decision result, and project the real-time reconstructed image obtained through image reconstruction onto the target projection screen through the target projection device.

[0037] In a third aspect, the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the projection screen image reconstruction method combined with distortion correction provided by the present invention.

[0038] The present invention discloses a projection screen image reconstruction method, system and medium combined with distortion correction, including: projecting a calibration image onto a target projection screen through a target projection device, and using a sensing device group to collect the projection effect of the image; analyzing the collected image in combination with a preset calibration pattern library, identifying and extracting the geometric distortion state information of the projection screen; inputting the distortion state information into a pre-built multi-channel distortion correction model, which includes three channels of geometric correction, color correction and brightness correction, for generating corresponding distortion correction decisions; performing image reconstruction processing on the received real-time projection image based on the correction decision, and re-projecting the reconstructed image onto the target screen through the target projection device to achieve dynamic optimization of the projection effect. The projection screen image reconstruction method, system and medium combined with distortion correction disclosed by the present invention solve the technical problems of single correction dimension, cumbersome operation and unsatisfactory correction effect, and achieve real-time, accurate and comprehensive reconstruction of the projection image, thereby improving the image quality and display stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Schematic diagram of the flow of the projection screen image reconstruction method combined with distortion correction according to the present invention.

[0040] Figure 2 Schematic diagram of the structure of the projection screen image reconstruction system combined with distortion correction of the present invention.

[0041] Description of reference numerals: calibration image projection and acquisition module 11 , geometric distortion recognition module 12 , distortion correction decision module 13 , image reconstruction and real-time projection module 14 . DETAILED DESCRIPTION

[0042] The above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods of the specification to better understand the above technical solution. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention. It should be understood that the present invention is not limited to the example embodiments used only to explain the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In addition, it should be noted that, for the convenience of description, only the parts related to the present invention, rather than all, are shown in the drawings.

[0043] Example 1, as Figure 1 : This is a flow chart of a projection screen image reconstruction method combined with distortion correction according to the present invention, wherein the projection screen image reconstruction method combined with distortion correction includes: S100: Projecting a calibration image onto a target projection screen through a target projection device, and collecting calibration image projection results through a sensing device group.

[0044] Specifically, a target projection device (e.g., a DLP projector or LCD projector) first projects a preset calibration image onto the target projection screen for subsequent distortion detection and modeling. Calibration images can include regular grid patterns (e.g., rectangular grids, orthogonal grids); diagonal cross patterns; multi-scale checkerboard patterns; and specific coded patterns (e.g., gray coding or structured light coding).

[0045] Specifically, the calibration image on the projection screen is synchronously collected by a group of sensing devices deployed at a predetermined position. The acquired image data should include: deformation information of the calibration image after projection, and distortion features caused by local concavity, curvature, texture disturbance, etc. on the surface of the projection screen.

[0046] The purpose of this step is to provide accurate raw data for subsequent geometric distortion identification. By projecting the calibration image and collecting the projection results, the image information actually presented on the projection screen can be obtained, and then compared and analyzed with the standard calibration pattern to identify the geometric distortion status information, laying the foundation for subsequent distortion correction decisions.

[0047] During the acquisition process, in some embodiments, projecting a calibration image onto a target projection screen via a target projection device includes: The method includes obtaining physical shape parameters of a target projection screen, pixel density parameters of a target projection device, and size parameters of a target projection area; accessing a preset calibration pattern library, and using the screen shape parameters, pixel density parameters, and projection size parameters as index conditions to match and extract corresponding standard calibration patterns from the calibration pattern library; constructing a calibration image based on the standard calibration pattern, wherein the calibration image includes structured pattern information for geometric calibration; and projecting the calibration image onto the target projection screen through the target projection device.

[0048] Specifically, the calibration pattern library is a database that stores a variety of standard calibration patterns. These standard calibration patterns have known characteristic information such as geometric shape, size, and color, and are used as a comparison benchmark to identify distortion.

[0049] Specifically, the system first obtains parameter information related to the projection environment, including the physical shape parameters of the target projection screen, such as the screen's curvature radius, bend angle, and surface contours; the target projection device's pixel density, such as the number of pixels per unit area (dpi) or the actual physical size of a single pixel; and the target projection area's dimensions, such as its width, height, and diagonal dimensions. It then accesses a pre-set calibration pattern library, which contains a variety of standard calibration patterns optimized for different screen shapes, pixel densities, and dimensions. Each standard calibration pattern corresponds to a set of index parameters, including the applicable screen shape type; the recommended pixel density range; and the supported projection size range. Next, the system uses the screen's physical shape parameters, pixel density parameters, and projection size parameters as index conditions to extract the standard calibration pattern that best matches the current projection environment from the calibration pattern library. Matching strategies include exact matching and nearest neighbor matching based on minimum distance.

[0050] Furthermore, the extracted standard calibration pattern is combined with a calibration image suitable for the current projection environment. This calibration image includes at least structured pattern information used for geometric calibration. This structured pattern information may include a regular grid, checkerboard, coded lines, or feature point arrays. Optionally, the pattern is scaled, stretched, or otherwise adapted to the actual projection size, ensuring that the scale and distribution of the calibration pattern on the projection screen conform to the intended design. The resulting calibration image is projected onto the target projection screen surface via the target projection device for subsequent acquisition and processing by the sensing device group.

[0051] Through the above implementation, the most suitable calibration pattern is dynamically selected according to the actual projection environment, which helps to improve the matching degree between the calibration image and the surface characteristics of the screen, ensure the structural integrity and geometric accuracy of the calibration pattern in the projection area, and improve the accuracy of subsequent distortion modeling and correction.

[0052] In some embodiments, collecting calibration image projection results through a sensing device group includes: A multimodal sensing device is deployed in a target projection environment, the multimodal sensing device including a brightness test device for brightness measurement, a chromaticity test device for color detection, and a high-resolution image acquisition device for image acquisition. The multimodal sensing device is activated to respectively acquire geometric image information, color distribution information, and brightness distribution information of a projected calibration image projected onto a target projection screen. The acquired geometric image information, color distribution information, and brightness distribution information are spatially remapped and uniformly converted into an ideal projection screen spatial coordinate system to form the calibration image projection result of the calibration image.

[0053] Specifically, a multimodal sensing device is a device that integrates multiple different types of sensors and is used to simultaneously capture multiple types of image information. Luminance testing equipment measures the brightness distribution of a projected image, specifically the luminous intensity of each image region. Chroma testing equipment detects the color distribution of an image, specifically the type and saturation of the color. High-resolution image acquisition equipment acquires high-definition geometric image information to capture image details and shape features.

[0054] Specifically, the ideal projection screen spatial coordinate system is a virtual, reference-based three-dimensional coordinate system that represents the spatial model of the projection screen under perfect projection conditions. In this coordinate system, the projection screen is assumed to be a perfectly flat, ideal plane located in a specific position and orientation, and the projected image is displayed perfectly on this plane without any distortion. Its origin is typically set at a specific location on the screen (such as the top left corner or the center), and the coordinate axes (such as the X-axis, Y-axis, and Z-axis) correspond to the horizontal, vertical, and depth directions of the screen, respectively.

[0055] Specifically, a multimodal sensing device is deployed in the target projection environment. The multimodal sensing device includes: a luminance test device for measuring the luminance distribution of the projected image in different areas; a chromaticity test device for detecting the color distribution characteristics of the projected image, such as color temperature, color gamut coverage, and color cast; and a high-resolution image acquisition device for capturing geometric information of the projected image, including pattern outlines and feature point locations. The aforementioned sensing devices can exist as a single integrated device (such as a multimodal sensor module) or as multiple independent devices working together.

[0056] Specifically, the multimodal sensing device is activated to collect different modal information of the calibration image projected on the target projection screen, including: Geometric image information: including the actual projection shape, edge contour, feature point coordinates, etc. of the calibration pattern; color distribution information: including the color value of each area (such as RGB, Lab or XYZ color space data); brightness distribution information: including the brightness value of each area (for example, measurement data in cd / m² (candela per square meter)).

[0057] Optionally, multiple sampling and data fusion are performed during the acquisition process to improve the accuracy and stability of the data.

[0058] Furthermore, spatial remapping processing is performed on the acquired geometric image information, color distribution information, and brightness distribution information, that is, each modal data is uniformly converted to a preset ideal projection screen spatial coordinate system, and the spatial distortion caused by factors such as the position offset of the sensing device, the difference in viewing angle, and the curvature of the screen is corrected, thereby ensuring the spatial alignment relationship between the information of different modalities and ensuring the one-to-one correspondence between the geometric, color, and brightness information of the same spatial position, forming a calibration image projection result data set corresponding to the ideal coordinate system, which serves as the basic data for subsequent distortion modeling, color correction, and brightness compensation.

[0059] S200: Based on a preset calibration pattern library, analyzing the calibration image projection result to perform geometric distortion recognition and obtain geometric distortion state information of the projection screen.

[0060] Specifically, the calibration pattern library is a database that stores a variety of standard geometric patterns and their corresponding feature information. By comparing the calibration image projection results with the corresponding calibration patterns in the standard calibration patterns, geometric deformations of the image on the projection screen, such as expansion, rotation, translation, and perspective distortion, can be detected. The acquired geometric distortion status information of the projection screen refers to the specific parameters and characteristics that describe the image distortion on the projection screen, including the type, degree, and location of the distortion, and is used for subsequent distortion correction decisions.

[0061] This step accurately and comprehensively identifies geometric distortion on the projection screen, providing reliable data support for subsequent corrections. Using a calibration pattern library as a reference allows for effective detection of multiple types of geometric distortion, improving the accuracy and comprehensiveness of distortion identification. Integrating the recognition results of luminance and chromaticity further enhances the richness and reliability of distortion status information, helping the subsequent correction model make more precise correction decisions.

[0062] In some embodiments, based on a preset calibration pattern library, analyzing the calibration image projection result to perform geometric distortion identification and obtain geometric distortion state information of the projection screen includes: Pattern recognition and matching are performed in the calibration image projection result to obtain a set of matching image pairs, wherein the matching image pairs include the standard calibration pattern and the corresponding distortion calibration pattern; geometric deformation recognition, including scaling, rotation, translation and perspective distortion, is traversed through the matching image pair set to obtain a first recognition result; brightness consistency evaluation and chromaticity deviation evaluation are traversed through the matching image pair set to obtain a second recognition result; feature extraction is performed on the first recognition result and the second recognition result respectively, and the feature extraction results are fused to generate the geometric distortion state information of the projection screen.

[0063] Specifically, first, pattern recognition and matching operations are performed in the calibration image projection results, that is, based on the preset standard calibration pattern, the corresponding distortion calibration pattern is extracted from the projection result to form a set of matching image pairs, where each matching image pair includes a standard calibration pattern and its corresponding distortion calibration pattern; the matching process can adopt feature point matching (such as SIFT, ORB), template matching or deep learning assisted recognition methods.

[0064] Next, the set of matching image pairs is traversed, and geometric deformation analysis is performed on each matching image pair. The system identifies deformations including, but not limited to, scaling, rotation, translation, and perspective distortion. A corresponding first recognition result is generated to describe the spatial distribution and extent of each type of geometric distortion. Optionally, geometric deformation recognition can be implemented using methods such as homography estimation and deformation field modeling.

[0065] At the same time, the set of matching image pairs is traversed, and each matching image pair is evaluated for brightness consistency (comparing the brightness difference between the corresponding areas of the standard calibration pattern and the distorted calibration pattern) and chromaticity deviation (analyzing the degree of color deviation between the corresponding areas of the standard calibration pattern and the distorted calibration pattern). This is because the deformation of the screen surface causes light to refract and scatter, which in turn affects the accuracy of different colors and the brightness at different locations. Based on these evaluations, a corresponding second recognition result is generated to describe the spatial distribution of brightness and color consistency. Luminance and chromaticity evaluation can be achieved using metrics such as mean square error, structural similarity, and color difference formulas (such as CIEDE2000).

[0066] Furthermore, feature extraction is performed on the first and second recognition results, and the extracted geometric deformation features, brightness consistency features, and chromaticity deviation features are fused to generate geometric distortion state information that comprehensively describes the geometric distortion state of the projection screen. The geometric deformation features serve as the primary information, while the brightness consistency features and chromaticity deviation features serve as auxiliary information, supplementing deformation information that cannot be clearly identified by the geometric deformation features. This helps accurately identify various geometric distortion phenomena of the projection screen during actual projection, improving adaptability and calibration accuracy in complex projection environments.

[0067] In some implementations, performing feature extraction on each of the first recognition result and the second recognition result, and fusing the feature extraction results to generate geometric distortion parameters of the target projection screen further includes: Performing primary geometric distortion recognition on the first recognition result, including extracting telescopic features, rotational features, translational features, and perspective deformation features related to the flatness of the screen; performing depth correction on the primary geometric distortion recognition result based on the second recognition result to obtain distortion depth features; wherein the depth correction includes inferring the directionality of the physical distortion of the target projection screen based on brightness and chromaticity deviation information; performing feature fusion on the distortion depth features and the distortion depth features, and outputting the feature fusion results as geometric distortion parameters of the target projection screen.

[0068] Specifically, the first recognition result is subjected to primary geometric distortion identification. Geometric features related to screen flatness, such as scaling, rotation, translation, and perspective deformation, are extracted to determine whether the projection screen exhibits geometric distortion and roughly determine the type and extent of the distortion. Then, based on the second recognition results, namely, luminance inconsistency and chromaticity deviation, the primary geometric distortion recognition result is subjected to depth correction. This correction utilizes the luminance inconsistency and chromaticity deviation information from the second recognition result to further refine the description of the geometric distortion by obtaining a more accurate distortion depth feature to infer the directionality of the physical distortion. For example, based on luminance deviation information, the physical concavity or convexity trend of a local area is inferred. Based on chromaticity deviation information, the direction of the screen surface normal change (i.e., color deviation may indicate changes in reflected light at different angles) is used to assist in inferring the directionality and extent of the projection screen's physical distortion. This results in a distortion depth feature that describes the directionality and extent of the projection screen's physical distortion.

[0069] Optionally, depth correction can be inferred by combining a lighting model (such as a Lambertian reflectance model) with color space analysis.

[0070] Furthermore, the distorted depth features are fused, and the fused feature results are output as the geometric distortion parameters of the target projection screen. First, the primary geometric distortion features and the distorted depth features (such as the concave-convex direction) are normalized to eliminate dimensional differences and facilitate subsequent feature fusion. The two types of features are then aligned according to their spatial location (pixel coordinates, block index, etc.) to ensure that features at the same location can be fused together. Feature fusion is then performed using strategies such as feature concatenation (directly concatenating the primary feature vector with the depth feature vector to form a new fused feature vector), weighted summation (assigning different weights to different features and then performing a weighted summation; the weights can be set empirically or obtained through training), or feature encoding (encoding the concatenated features using a small neural network (such as an MLP) to extract higher-level fused features). The fused features are then mapped into the final geometric distortion parameters.

[0071] Optionally, the geometric distortion parameters may include: screen part expansion / contraction ratio distribution, screen part rotation angle distribution, screen part translation amount distribution, screen part perspective distortion intensity, screen surface normal vector offset distribution (direction and degree), and screen part concave / convex depth estimation value.

[0072] The purpose of the above steps is to more accurately identify and quantify geometric distortion by comprehensively utilizing geometric, luminance, and chromaticity information, providing a more comprehensive and accurate parameter basis for the correction decisions of the subsequent multi-channel distortion correction model, thereby achieving higher-quality image reconstruction. Among them, primary geometric distortion identification can preliminarily determine the type and degree of geometric distortion; depth correction further utilizes luminance and chromaticity information to accurately infer the directionality of distortion, making up for the possible shortcomings of simple geometric identification and making the distortion characteristics more complete and accurate. The final feature fusion process integrates multiple features into a unified geometric distortion parameter, which helps to achieve more accurate geometric correction and thus improve the quality and effect of projected image reconstruction.

[0073] S300: Inputting the distortion state information into a pre-built multi-channel distortion correction model to make a distortion correction decision, wherein the multi-channel distortion correction model includes a geometry correction channel, a color correction channel, and a brightness correction channel.

[0074] Specifically, the distortion state information obtained in the previous step (including geometric distortion parameters, color offset information, and brightness non-uniformity information) is fed into a multi-channel distortion correction model, where it is processed in three different correction channels: geometric correction, color correction, and brightness correction. The correction decisions of each channel are combined to generate the final distortion compensation instructions or correction configuration.

[0075] The significance of this approach lies in the fact that dedicated channels are independently modeled for different types of distortion (shape, color cast, and uneven brightness). This helps improve correction results and ensures that correction decisions are targeted and controllable. For example, the geometry correction channel can address image shape distortion, the color correction channel can restore the image's true color, and the brightness correction channel can even out the image's brightness distribution. This multi-channel correction approach avoids the additional distortion issues that may be caused by a single correction, improving the stability and reliability of the correction results.

[0076] In some embodiments, obtaining a pre-built multi-channel distortion correction model includes: Based on the inverse transformation algorithm, the geometric correction channel in matrix form is defined and constructed; the sample color deviation status information is obtained as training data, and a color mapping network based on a machine learning model is constructed and trained, and the output is the color correction channel, wherein the sample color deviation status information includes the sample projection device characteristic parameters, the sample light incident angle, the sample projection color value, and the sample color deviation index; the sample brightness deviation status information is obtained, and a brightness mapping model is constructed by a function fitting method, wherein the sample brightness deviation status information includes the sample calibrated viewing position, the sample light incident angle offset, and the sample brightness attenuation; the geometric correction channel, the color correction channel, and the brightness correction channel are integrated to obtain the multi-channel distortion correction model.

[0077] Specifically, the geometric correction channel refers to a module that uses an inverse transformation algorithm and matrix transformation to correct the geometric deformation of the projected image. The color correction channel uses a color mapping network trained using machine learning methods to correct color deviation. The brightness correction channel uses a brightness mapping model built using function fitting methods to compensate for brightness attenuation or unevenness.

[0078] Specifically, sample color deviation status information includes: sample projection device optical characteristic parameters, sample light incident angle, sample projected color value, sample color deviation indicators (such as ΔE, color temperature drift, etc.). Sample brightness deviation status information includes: sample calibration viewing position (such as a specific area or angle on the screen), sample light incident angle offset, and sample brightness attenuation (such as the percentage decrease relative to the center brightness).

[0079] Specifically, first, based on the inverse transformation algorithm, a mathematical model of geometric distortion is defined, such as affine transformation, perspective transformation or free deformation model, and the relationship between distortion and correction is expressed in matrix form, that is, an inverse transformation matrix from the distorted image coordinates to the ideal image coordinates is established as the core output of the geometric correction channel.

[0080] Specifically, the projected color values ​​under different projection devices and different light incident angles are recorded, and the color deviation indicators (such as ΔE value) between the actual projection results and the standard color are measured to obtain a large amount of sample color deviation status information. The sample color deviation status information is used as input and the standard color value is used as a label to construct and train a color mapping network based on machine learning.

[0081] Exemplarily, a shallow or deep fully connected neural network (MLP) is used, and a feature vector such as [device parameters, incident angle, projection color value, color deviation index] is input; the output is a color correction parameter (such as a color gain matrix or a color gamut mapping matrix).

[0082] Specifically, using the same idea, we record the brightness values ​​corresponding to different viewing positions (such as center, edge, and corner), and measure the corresponding relationship between the light incident angle offset and the brightness attenuation. Then, based on the collected data, we use function fitting methods (such as polynomial fitting and spline interpolation) to establish a brightness mapping model as a brightness correction channel.

[0083] Furthermore, the geometric correction channel, color correction channel and brightness correction channel constructed above are integrated to form a complete multi-channel distortion correction model, wherein the integration method adopts inter-channel dependency modeling, that is, geometric correction is performed first, and then color and brightness compensation is performed.

[0084] Through the above process, it is possible to independently model geometric deformation, color deviation and uneven brightness problems in a modular manner, so that the formed multi-channel distortion correction model has good scalability and real-time performance, thereby improving the overall image quality and user experience.

[0085] In some embodiments, inputting the distortion state information into a pre-built multi-channel distortion correction model to make a distortion correction decision includes: The distortion state information is input into the geometric correction channel for channel initialization, and a geometric correction vector is calculated based on the initialized geometric correction channel to obtain a geometric correction decision matrix; based on the geometric correction decision matrix and the distortion state information, the derived distortion points are located to obtain a derived distortion point set; a mapping relationship between the derived distortion point set and the distortion state information is established to extract derived distortion state parameters; the derived distortion state parameters are respectively input into the color correction channel and the brightness correction channel to calculate the color correction vector and the brightness correction vector, and a color correction decision matrix and a brightness correction decision matrix are correspondingly constructed; the geometric correction decision matrix, the color correction decision matrix, and the brightness correction decision matrix are output as the distortion correction decision result.

[0086] Specifically, the geometric correction decision matrix refers to the transformation matrix used to restore the distorted image to its standard geometric form, typically an affine matrix, perspective matrix, or freeform mesh matrix. Derived distortion points are image points that, after preliminary geometric correction, are mapped to the locations of the original distorted areas. These points require pre-calibration of color and brightness to overcome potential anomalies. Derived distortion state parameters are parameters such as color deviation and brightness offset extracted from the derived distortion points and are used for subsequent color and brightness correction.

[0087] Specifically, the color correction decision matrix refers to a mapping matrix or gain matrix used to correct color deviation, and the brightness correction decision matrix refers to a gain matrix or compensation map used to compensate for brightness unevenness or brightness attenuation.

[0088] Specifically, first, the distortion state information is input into the geometric correction channel for channel initialization. During the initialization process, the geometric features in the distortion state information (such as grid deformation, angle offset, translation, etc.) are analyzed, the geometric correction vector is calculated based on the inverse transformation algorithm, and a geometric correction decision matrix is ​​generated. The geometric correction decision matrix is ​​used to restore the standard geometric form of the image.

[0089] Specifically, an inverse mapping relationship is then established based on the geometric correction decision matrix and the distortion state information. That is, the image points that will be mapped to the distorted positions on the original screen after geometric correction are identified as derived distortion points, and all identified derived distortion points are grouped into a derived distortion point set; further, based on the derived distortion point set, the color deviation parameters and brightness offset parameters corresponding to each point are extracted to form a derived distortion state parameter set.

[0090] Furthermore, the derived distortion state parameters are input into the color correction channel and the brightness correction channel respectively; the color correction vector is calculated in the color correction channel, and a color correction decision matrix is ​​generated; the brightness correction vector is calculated in the brightness correction channel, and a brightness correction decision matrix is ​​generated, and the above-generated geometric correction decision matrix, color correction decision matrix and brightness correction decision matrix are output as the distortion correction decision results.

[0091] In the above method steps, geometric distortion is accurately located and corrected through initialization and vector calculation of the geometric correction channel. Simultaneously, by extracting the derived distortion state parameters and inputting them into the color and brightness correction channels, precise correction of color and brightness distortion is achieved. This multi-channel collaborative processing approach ensures comprehensive and accurate correction decisions, laying the foundation for high-quality image reconstruction.

[0092] In some embodiments, the inverse transformation algorithm includes at least one of an inverse homography matrix transformation, an inverse affine transformation, and an inverse perspective transformation.

[0093] S400: Based on the distortion correction decision result, the received real-time projection image is reconstructed, and the real-time reconstructed image obtained through image reconstruction is projected onto the target projection screen through the target projection device.

[0094] Specifically, the distorted real-time projection image is processed according to the distortion correction decision result to generate new image data, so that the reconstructed image meets the expected effects in terms of geometry, color, brightness, etc.

[0095] Specifically, first, the real-time projection image frame output from the image source (such as an image server, camera, sensor or graphics processing unit GPU) is received and used as the input image; then, the pre-generated distortion correction decision result is called to perform geometric correction, color correction and brightness correction on the image in sequence; then, after completing the image reconstruction, the reconstructed image is projected onto the target projection screen through the target projection device and displayed to the audience.

[0096] Optionally, during the image reconstruction process, if there are sparse areas in the reconstructed image due to geometric transformation (i.e., missing pixels or mapping holes), fast interpolation processing is performed, including but not limited to: nearest neighbor interpolation, bilinear interpolation, weighted average interpolation, and convolution filling interpolation, to fill in the sparse areas and improve image coherence and visual quality.

[0097] In summary, the projection screen image reconstruction method combined with distortion correction provided by the present invention has the following technical effects: The calibration image is projected onto the target projection screen through the target projection device, and the projection effect of the image is captured by the sensing device group; the captured image is analyzed in combination with the preset calibration pattern library to identify and extract the geometric distortion state information of the projection screen; the distortion state information is input into a pre-built multi-channel distortion correction model, which includes three channels: geometric correction, color correction, and brightness correction, and is used to generate corresponding distortion correction decisions; based on the correction decision, image reconstruction processing is performed on the received real-time projection image, and the reconstructed image is re-projected onto the target screen through the target projection device to achieve dynamic optimization of the projection effect, thereby achieving real-time, accurate, and comprehensive reconstruction of the projection image, and improving the technical effect of image quality and display stability.

[0098] Example 2, as Figure 2 Schematic diagram of the structure of the projection screen image reconstruction system combined with distortion correction of the present invention. For example, Figure 1 The flowchart of the projection screen image reconstruction method combined with distortion correction in the present invention can be shown as follows: Figure 2 The structure shown is implemented.

[0099] Based on the same concept as the projection screen image reconstruction method combined with distortion correction in the above embodiment, the present invention also provides a projection screen image reconstruction system combined with distortion correction, including: The calibration image projection and acquisition module 11 is used to project the calibration image onto the target projection screen through the target projection device, and to acquire the calibration image projection result through the sensing device group.

[0100] The geometric distortion recognition module 12 is configured to analyze the calibration image projection result based on a preset calibration pattern library to perform geometric distortion recognition and obtain geometric distortion status information of the projection screen.

[0101] The distortion correction decision module 13 is used to input the distortion state information into a pre-built multi-channel distortion correction model to make a distortion correction decision, wherein the multi-channel distortion correction model includes a geometric correction channel, a color correction channel, and a brightness correction channel.

[0102] The image reconstruction and real-time projection module 14 is used to reconstruct the received real-time projection image based on the distortion correction decision result, and project the real-time reconstructed image obtained through image reconstruction onto the target projection screen through the target projection device.

[0103] In some embodiments, the calibration image projection and acquisition module 11 includes: The parameter acquisition unit is used to acquire the physical shape parameters of the target projection screen, the pixel density parameters of the target projection device, and the size parameters of the target projection area.

[0104] The standard calibration pattern extraction unit is used to access a preset calibration pattern library and extract the corresponding standard calibration pattern from the calibration pattern library using the screen shape parameter, pixel density parameter and projection size parameter as index conditions.

[0105] A calibration image construction unit is configured to construct a calibration image in combination with the standard calibration pattern, wherein the calibration image includes structured pattern information for geometric calibration.

[0106] The calibration image projection unit is used to project the calibration image onto a target projection screen through a target projection device.

[0107] In some embodiments, the calibration image projection and acquisition module 11 further includes: A multimodal sensing device deployment unit is used to deploy a multimodal sensing device in a target projection environment. The multimodal sensing device includes a brightness test device for brightness measurement, a chromaticity test device for color detection, and a high-resolution image acquisition device for image acquisition.

[0108] The calibration image information acquisition unit is used to activate the multimodal sensing device and respectively acquire geometric image information, color distribution information and brightness distribution information of the projected calibration image projected on the target projection screen.

[0109] The spatial remapping processing unit is used to perform spatial remapping processing on the collected geometric image information, the color distribution information, and the brightness distribution information, and uniformly convert them into an ideal projection screen spatial coordinate system to form the calibration image projection result of the calibration image.

[0110] In some embodiments, the geometric distortion identification module 12 includes: The pattern recognition and matching unit is used to perform pattern recognition and matching in the calibration image projection result to obtain a set of matching image pairs, wherein the matching image pairs include the standard calibration pattern and the corresponding distortion calibration pattern.

[0111] The geometric deformation recognition unit is used to traverse the set of matching image pairs to perform geometric deformation recognition, including scaling, rotation, translation and perspective distortion, to obtain a first recognition result.

[0112] The brightness and chromaticity evaluation unit is used to traverse the set of matching image pairs to perform brightness consistency evaluation and chromaticity deviation evaluation to obtain a second recognition result.

[0113] The feature extraction and fusion unit is used to extract features from the first recognition result and the second recognition result respectively, and fuse the feature extraction results to generate the geometric distortion state information of the projection screen.

[0114] In some implementations, the feature extraction and fusion unit in the geometric distortion identification module 12 includes: The primary geometric distortion recognition unit is used to perform primary geometric distortion recognition on the first recognition result, including extracting the telescopic feature, rotation feature, translation feature and perspective deformation feature related to the flatness of the curtain.

[0115] A depth correction unit is used to perform depth correction on the primary geometric distortion recognition result based on the second recognition result to obtain a distortion depth feature; wherein the depth correction includes inferring the directionality of the physical distortion of the target projection screen based on the brightness and chromaticity deviation information.

[0116] The feature fusion and distortion parameter output unit is used to perform feature fusion on the distortion depth feature and the distortion depth feature, and output the feature fusion result as the geometric distortion parameter of the target projection screen.

[0117] In some embodiments, the distortion correction decision module 13 includes: The geometric correction channel construction unit is used to define and construct the geometric correction channel in matrix form based on an inverse transformation algorithm.

[0118] A color correction channel training unit is used to obtain sample color deviation status information as training data, build and train a color mapping network based on a machine learning model, and output the color correction channel, wherein the sample color deviation status information includes sample projection device characteristic parameters, sample light incident angle, sample projection color value, and sample color deviation index.

[0119] The brightness correction channel construction unit is used to obtain sample brightness deviation state information and construct a brightness mapping model through a function fitting method, wherein the sample brightness deviation state information includes the sample calibration viewing position, the sample light incident angle offset, and the sample brightness attenuation.

[0120] The multi-channel distortion correction model integration unit is used to integrate the geometric correction channel, the color correction channel and the brightness correction channel to obtain the multi-channel distortion correction model.

[0121] In some embodiments, the distortion correction decision module 13 includes: The geometric correction channel initialization and decision matrix acquisition unit is used to input the distortion state information into the geometric correction channel for channel initialization, and perform geometric correction vector calculation based on the initialized geometric correction channel to obtain a geometric correction decision matrix.

[0122] The derived distortion point positioning and point set acquisition unit is used to locate the derived distortion points and acquire the derived distortion point set according to the geometric correction decision matrix and the distortion state information.

[0123] The mapping relationship establishment and derived distortion state parameter extraction unit is used to establish a mapping relationship between the derived distortion point set and the distortion state information, and extract derived distortion state parameters.

[0124] The color and brightness correction decision matrix construction unit is used to input the derived distortion state parameters into the color correction channel and the brightness correction channel respectively, calculate the color correction vector and the brightness correction vector, and construct the color correction decision matrix and the brightness correction decision matrix accordingly.

[0125] The distortion correction decision result output unit is configured to output the geometric correction decision matrix, the color correction decision matrix, and the brightness correction decision matrix as the distortion correction decision result.

[0126] In some embodiments, the inverse transformation algorithm includes at least one of an inverse homography matrix transformation, an inverse affine transformation, and an inverse perspective transformation.

[0127] In a third embodiment, the present invention further provides a computer-readable storage medium that can be used to store software programs, computer executable programs, and modules, such as program instructions / modules corresponding to the projection screen image reconstruction method combined with distortion correction in the embodiment of the present invention, thereby realizing the above-mentioned projection screen image reconstruction method combined with distortion correction.

[0128] It should be understood that the embodiments mentioned in this specification focus on their differences from other embodiments. The specific embodiments in the aforementioned embodiment 1 are also applicable to the projection screen image reconstruction system combined with distortion correction described in embodiment 2. For the sake of brevity of the specification, no further elaboration is given here.

[0129] It should be understood that the embodiments disclosed in the present invention and the above description can enable those skilled in the art to use the present invention to implement the present invention. At the same time, the present invention is not limited to the embodiments mentioned above. It should be understood that those skilled in the art can still modify the technical solutions described in the above embodiments or replace some of the technical features therein with equivalents; and such modifications or replacements do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention and are all included in the scope of protection of the present invention.

Claims

1. A projection screen image reconstruction method combined with distortion correction, characterized in that: include: Projecting the calibration image onto the target projection screen through the target projection device, and collecting the calibration image projection result through the sensing device group; Based on a preset calibration pattern library, the calibration image projection result is analyzed to perform geometric distortion identification and obtain geometric distortion state information of the projection screen; Inputting the distortion state information into a pre-built multi-channel distortion correction model to make a distortion correction decision, wherein the multi-channel distortion correction model includes a geometry correction channel, a color correction channel, and a brightness correction channel; Based on the distortion correction decision result, the received real-time projection image is reconstructed, and the real-time reconstructed image obtained through image reconstruction is projected onto the target projection screen through the target projection device.

2. The projection screen image reconstruction method combined with distortion correction according to claim 1, wherein: Projecting a calibration image onto a target projection screen through a target projection device includes: Obtaining the physical shape parameters of the target projection screen, the pixel density parameters of the target projection device, and the size parameters of the target projection area; Accessing a preset calibration pattern library, and using the screen shape parameter, pixel density parameter, and projection size parameter as index conditions, matching and extracting a corresponding standard calibration pattern from the calibration pattern library; constructing a calibration image in combination with the standard calibration pattern, wherein the calibration image includes structured pattern information for geometric calibration; The calibration image is projected onto a target projection screen through a target projection device.

3. The projection screen image reconstruction method combined with distortion correction according to claim 2, wherein: The calibration image projection results are collected through the perception device group, including: Deploying a multimodal sensing device in a target projection environment, the multimodal sensing device comprising a brightness test device for brightness measurement, a colorimetry test device for color detection, and a high-resolution image acquisition device for image acquisition; activating the multimodal sensing device to respectively collect geometric image information, color distribution information, and brightness distribution information of the projection calibration image projected on the target projection screen; The collected geometric image information, the color distribution information, and the brightness distribution information are spatially remapped and uniformly converted into an ideal projection screen space coordinate system to form the calibration image projection result of the calibration image.

4. The projection screen image reconstruction method combined with distortion correction according to claim 3, wherein: Based on a preset calibration pattern library, the calibration image projection result is analyzed to perform geometric distortion recognition and obtain geometric distortion status information of the projection screen, including: Performing pattern recognition matching in the calibration image projection result to obtain a set of matching image pairs, wherein the matching image pairs include the standard calibration pattern and the corresponding distortion calibration pattern; Traversing the set of matching images to perform geometric deformation recognition, including scaling, rotation, translation, and perspective distortion, to obtain a first recognition result; Traversing the set of matching images to perform brightness consistency evaluation and chromaticity deviation evaluation to obtain a second recognition result; Feature extraction is performed on the first recognition result and the second recognition result respectively, and the feature extraction results are fused to generate the geometric distortion state information of the projection screen.

5. The projection screen image reconstruction method combined with distortion correction according to claim 4, wherein: Performing feature extraction on the first recognition result and the second recognition result respectively, and fusing the feature extraction results to generate geometric distortion parameters of the target projection screen further includes: Performing primary geometric distortion recognition on the first recognition result, including extracting telescopic features, rotational features, translational features, and perspective deformation features related to the flatness of the curtain; Performing depth correction on the primary geometric distortion recognition result based on the second recognition result to obtain a distortion depth feature; wherein the depth correction includes estimating the directionality of the physical distortion of the target projection screen based on the brightness and chromaticity deviation information; The distortion depth feature and the distortion depth feature are subjected to feature fusion, and a feature fusion result is output as a geometric distortion parameter of the target projection screen.

6. The projection screen image reconstruction method combined with distortion correction according to claim 4, wherein: Get pre-built multi-channel distortion correction models, including: Based on the inverse transformation algorithm, defining and constructing the geometric correction channel in matrix form; Obtaining sample color deviation status information as training data, constructing and training a color mapping network based on a machine learning model, and outputting the color correction channel, wherein the sample color deviation status information includes sample projection device characteristic parameters, sample light incident angle, sample projection color value, and sample color deviation index; Obtaining sample brightness deviation state information and constructing a brightness mapping model using a function fitting method, wherein the sample brightness deviation state information includes the sample calibration viewing position, the sample light incident angle offset, and the sample brightness attenuation; The geometric correction channel, the color correction channel, and the brightness correction channel are integrated to obtain the multi-channel distortion correction model.

7. The projection screen image reconstruction method combined with distortion correction according to claim 6, wherein: Inputting the distortion state information into a pre-built multi-channel distortion correction model to make a distortion correction decision includes: Inputting the distortion state information into the geometric correction channel to initialize the channel, and performing geometric correction vector calculation based on the initialized geometric correction channel to obtain a geometric correction decision matrix; Locating derived distortion points according to the geometric correction decision matrix and the distortion state information, and obtaining a derived distortion point set; Establishing a mapping relationship between the derived distortion point set and the distortion state information, and extracting derived distortion state parameters; Inputting the derived distortion state parameters into the color correction channel and the brightness correction channel respectively, calculating a color correction vector and a brightness correction vector, and constructing a color correction decision matrix and a brightness correction decision matrix accordingly; The geometric correction decision matrix, the color correction decision matrix, and the brightness correction decision matrix are output as the distortion correction decision result.

8. The projection screen image reconstruction method combined with distortion correction according to claim 6, wherein: The inverse transformation algorithm includes at least one of homography matrix inverse transformation, affine inverse transformation and perspective inverse transformation.

9. A projection screen image reconstruction system combined with distortion correction, characterized in that: The method for reconstructing a projection screen image combined with distortion correction according to any one of claims 1 to 8 comprises: The calibration image projection and acquisition module is used to project the calibration image onto the target projection screen through the target projection device and acquire the calibration image projection result through the sensing device group; A geometric distortion recognition module is used to analyze the calibration image projection result based on a preset calibration pattern library to perform geometric distortion recognition and obtain geometric distortion status information of the projection screen; a distortion correction decision module, configured to input the distortion state information into a pre-built multi-channel distortion correction model to make a distortion correction decision, wherein the multi-channel distortion correction model includes a geometric correction channel, a color correction channel, and a brightness correction channel; The image reconstruction and real-time projection module is used to reconstruct the received real-time projection image based on the distortion correction decision result, and project the real-time reconstructed image obtained through image reconstruction onto the target projection screen through the target projection device.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the projection screen image reconstruction method combined with distortion correction according to any one of claims 1 to 8 is implemented.

Images