Projection screen correction method and electronic device

Abstract

The application relates to the technical field of projectors, and discloses a projection picture correction method and electronic equipment. The method comprises the following steps: acquiring a to-be-projected picture, collecting multi-modal sensing data of a projection surface; performing feature extraction on obstacles in the projection surface according to the multi-modal sensing data, to obtain position features and semantic category features of the obstacles; performing saliency analysis on the to-be-projected picture, to obtain importance distribution features for representing the importance of each region in the to-be-projected picture in vision; determining a target projection region in the projection surface according to the importance distribution features and the position features and the semantic category features of the obstacles; and mapping the to-be-projected picture to the target projection region by using a curved surface deformation method, and controlling the deformation degree of each region in the to-be-projected picture in the mapping process according to the importance distribution features. The embodiment of the application can improve the projection picture coverage integrity and visual comfort.

Description

Technical Field

[0001] This application relates to the field of projector technology, and in particular to a method for correcting projected images and an electronic device. Background Technology

[0002] In practical applications of smart projection devices, projection walls commonly contain various physical obstacles, such as power switch panels, decorative picture frames, air conditioning vent grilles, edges of wall-mounted shelves, and door and window frames. These obstacles not only occupy the projection area, partially obscuring the image content, but also cause light scattering, uneven reflection, and shadow projection due to differences in surface materials and geometric shapes, resulting in localized brightness imbalances, color shifts, and geometric distortions in the image. Especially when obstacles have three-dimensional convex characteristics, the projected light undergoes nonlinear refraction at the boundary of the obstacle, further exacerbating the image distortion and making it difficult to clearly present key visual information. Existing projection correction technologies mainly rely on four-corner trapezoidal correction or automatic obstacle avoidance functions based on preset rules. Their limitation is that they can only handle overall projection surface tilt or simple geometric distortions, and cannot perform refined perception and dynamic adaptation to localized obstacles within the projection area. Summary of the Invention

[0003] The purpose of this application is to provide a projection image correction method and electronic device, which can dynamically adapt to obstacles in the projection surface, thereby improving the coverage integrity and visual comfort of the projection image.

[0004] This application provides a projection image correction method, including: Acquire the image to be projected and collect multimodal sensing data of the projection surface; Based on the multimodal sensing data, features are extracted from the obstacles in the projection surface to obtain the location features and semantic category features of the obstacles; A saliency analysis is performed on the image to be projected to obtain the importance distribution characteristics that characterize the visual importance of each region in the image to be projected. Based on the importance distribution characteristics, as well as the location and semantic category characteristics of the obstacles, the target projection area in the projection plane is determined; The surface deformation method is used to map the image to be projected onto the target projection area, and the degree of deformation of each area in the image to be projected is controlled during the mapping process according to the importance distribution characteristics.

[0005] In some embodiments, the step of extracting features from obstacles in the projection surface based on the multimodal sensing data includes: Based on the multimodal sensing data, the projection surface is subjected to difference feature extraction to obtain the corresponding depth difference features, color difference features and texture difference features; The contour boundary of the obstacle is identified based on the depth difference feature, the color difference feature, and the texture difference feature; The positional features of the obstacle are determined based on the positional features of the contour boundary; Based on the geometric morphological features of the outline boundary, the semantic category features of the obstacle are determined.

[0006] In some embodiments, identifying the contour boundary of the obstacle based on the depth difference feature, the color difference feature, and the texture difference feature includes: Based on the depth difference characteristics, determine the depth boundary position of the obstacle and the depth deviation region relative to the projection surface; Based on the color difference characteristics, the local color deviation area of ​​the obstacle is determined; Based on the texture difference characteristics, the regions of abnormal texture complexity of the obstacle are determined; An energy function is constructed based on the depth boundary location, the depth deviation region, the color deviation region, and the texture complexity anomaly region; The energy function is solved using the graph cut segmentation method to obtain the outline boundary of the obstacle.

[0007] In some embodiments, the saliency analysis of the image to be projected includes: Multi-scale spectral residual significance detection is performed on the image to be projected to obtain significance maps at each scale. The significance maps are then fused to obtain a fused significance map. The image to be projected is semantically segmented to determine the semantic category of each pixel in the image to be projected and the semantic weight corresponding to the semantic category. The importance distribution features are obtained by combining the fusion saliency map and the semantic weights.

[0008] In some embodiments, determining the target projection region in the projection plane based on the importance distribution features and the location and semantic category features of the obstacles includes: An optimization model is constructed with the optimization objectives of maximizing the effective projection area, maximizing the coverage area of ​​each region in the image to be projected according to the importance distribution characteristics, and minimizing geometric distortion. The distribution location of the obstacles is determined based on their location characteristics; Based on the semantic category features of the obstacle, determine the safe avoidance distance corresponding to the obstacle; The distribution locations of the obstacles and their corresponding safe avoidance distances are used as constraints on the optimization model. The optimization model is solved on the projection surface to determine the target projection area. The constraints require that the distance between the candidate projection area and the distribution locations of the obstacles be no less than the safe avoidance distance.

[0009] In some embodiments, controlling the degree of deformation of each region in the image to be projected during the mapping process based on the importance distribution characteristics includes: A curved surface control mesh is established on the target projection area; Based on the boundary of the target projection area, determine the control points in the control grid; The influence weights of the control points are configured according to the importance distribution characteristics; the importance of the region where the control point is located in the importance distribution characteristics is positively correlated with the influence weights. Based on the influence weight, the image to be projected is mapped to the target projection area.

[0010] In some embodiments, the projection image correction method further includes: The anisotropic deformation region generated during the mapping process of the image to be projected is detected, and the anisotropic deformation region is sharpened and compensated along the compression direction. Based on the projection distance and light incident angle corresponding to each pixel in the target projection area, a brightness attenuation compensation factor is calculated to compensate for the brightness of the target projection area; and / or Based on the surface reflection characteristics of the obstacle, the reflection influence range and shadow influence range of the obstacle's outline boundary on the projected light are determined. Based on the reflection influence range and shadow influence range, brightness pre-compensation is performed on the area in the image to be projected that is adjacent to the outline boundary of the obstacle.

[0011] In some embodiments, the projection image correction method further includes: Based on the position and size characteristics of the obstacles in the projection plane, a cost function is constructed; Based on the cost function, inter-frame tracking is performed on the obstacles in the projection plane to determine the motion state variables of the obstacles; When the state change exceeds a preset state change threshold, the control point is locally updated. When the obstacle is detected to have disappeared, the projection area of ​​the projection surface is gradually expanded according to a preset transition duration parameter, so as to reclaim the area occupied by the obstacle on the projection surface.

[0012] In some embodiments, the projection image correction method further includes: Acquire the pose changes of the projection device detected by the inertial measurement unit; When the pose change exceeds a preset pose change threshold, the projected position of the obstacle in the current pose is recalculated based on the pre-stored three-dimensional spatial coordinates of the obstacle, and the target projection area is updated based on the projected position of the obstacle in the current pose.

[0013] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described projection image correction method.

[0014] The beneficial effects of this application are as follows: Based on multimodal sensing data of the projection surface, the positional and semantic category features of obstacles in the projection surface are actively perceived. By performing saliency analysis on the image to be projected, the importance distribution features representing the visual importance of each region in the image to be projected are determined, which can improve the precision of obstacle perception and the visual priority of understanding the image content. By combining the positional features, semantic category features, and importance distribution features of obstacles, the target projection area is determined, and the image to be projected is mapped to the target projection area. The degree of deformation of each region in the image to be projected is controlled according to the importance distribution features during the mapping process, which can realize intelligent obstacle avoidance and content protection. This ensures that the corrected image can adapt to irregular projection areas and dynamically adapt to obstacles in the projection surface, while preserving the visual quality of important content to the greatest extent, thereby improving the coverage integrity and visual comfort of the projected image. Attached Figure Description

[0015] Figure 1 This is a flowchart of the projection image correction method provided in the embodiments of this application.

[0016] Figure 2 This is a flowchart of a method for feature extraction of obstacles in a projection plane provided in an embodiment of this application.

[0017] Figure 3 This is a flowchart of a method for performing saliency analysis on a projected image, provided in an embodiment of this application.

[0018] Figure 4 This is a flowchart of a method for determining a target projection area in a projection plane, provided in an embodiment of this application.

[0019] Figure 5 This is a flowchart of a method for controlling the degree of deformation of each region in the image to be projected during the mapping process based on the importance distribution characteristics provided in this application embodiment.

[0020] Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0022] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and drawings are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application. Furthermore, the information, data, and signals involved in the embodiments of this application are all authorized by relevant parties or have been fully authorized by all parties, and the collection, use, and processing of related data comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0024] In the application of smart projection devices, existing projection image correction technologies are limited by their inability to perceive local obstacles within the projection surface. They can only perform correction operations based on the overall geometry of the projection surface, failing to effectively identify the location and semantic category characteristics of obstacles. This results in content occlusion and geometric distortion when the projected image covers obstacles, thus affecting the complete presentation of visual information and user interaction efficiency. For example, in a home living room scenario, the projection wall may contain power switches and decorative picture frames. The raised structure of the power switch creates areas of depth difference, while the border of the picture frame creates color and texture boundary features. When the projected image covers these areas, the switch position causes local pixel loss, and the picture frame border causes edge stretching and distortion. Users need to manually adjust the projection position or zoom the image to avoid obstacles. This process requires multiple trials and errors and is difficult to accurately match the outline of obstacles, causing interruptions in the continuity of the projected content.

[0025] If this problem is not solved, the projected image will frequently show areas with missing content and non-uniform deformation, the recognizability of visual information will continue to decline, users will need to repeatedly perform position adjustment operations, increasing the interactive burden, and the automation level of the projection equipment will be limited, making it unable to adapt to the stable output requirements of complex wall environments.

[0026] Based on this, embodiments of this application provide a projection image correction method and electronic device, which extracts the position features and semantic category features of obstacles based on the multimodal sensing data of the projection surface, determines the target projection area in the projection surface by combining the image saliency analysis, and adaptively controls the degree of deformation by surface deformation, thereby realizing dynamic adaptation to obstacles in the projection surface.

[0027] See Figure 1 In one embodiment, a projection image correction method is provided, which includes, but is not limited to, steps S101 to S105.

[0028] Step S101: Obtain the image to be projected and collect multimodal sensing data of the projection surface.

[0029] The image to be projected refers to the digital image or video content output by the projection device that needs to be projected onto a certain surface. This image is the visual information that the user wants to present on the projection surface.

[0030] The projection surface refers to the physical surface on which the image to be projected is actually projected. It can be a wall, a screen, or any other medium suitable for receiving projections.

[0031] Multimodal sensing data refers to information about the projection surface collected by multiple sensors, including but not limited to depth information, color information, texture information, infrared information, etc.

[0032] The image to be projected can be acquired in various ways, such as reading digital image files from a storage device or capturing frames in real time from a video stream. Multimodal sensing data of the projection surface can be acquired using one or more sensors configured in or near the projection device. For example, a depth camera can be used to acquire depth information of the projection surface, while a color camera can be used to acquire color and texture information. Infrared sensors can be used to acquire thermal distribution data, or ultrasonic sensors can be used to detect the presence of obstacles. These sensors can work independently or collaboratively to provide comprehensive perception data about the projection surface.

[0033] Step S102: Based on the multimodal sensing data, feature extraction is performed on the obstacles in the projection plane to obtain the location features and semantic category features of the obstacles.

[0034] Location features refer to the spatial coordinate information of an obstacle on the projection plane, such as its center point coordinates and bounding box coordinates, which are used to describe the geometric position of the obstacle.

[0035] Semantic category features refer to the type or attribute information of obstacles, such as whether the obstacle is a switch, picture frame, or shelf. This feature helps to understand the nature of the obstacle and its impact on the projection.

[0036] Feature extraction of obstacles on the projection surface can begin by preprocessing the acquired multimodal sensor data, such as denoising and calibration. Then, image processing algorithms, such as edge detection, region growing, or cluster analysis, are used to identify regions on the projection surface that show significant differences from the background; these regions are initially identified as obstacles. For each identified obstacle, its geometric center coordinates and bounding box on the projection surface are calculated to obtain its positional features. Simultaneously, based on visual attributes such as shape, size, and color, obstacles can be categorized into predefined semantic categories, such as "switch," "picture frame," or "shelf," using preset rules or classifiers, thus obtaining their semantic category features.

[0037] Step S103: Perform saliency analysis on the image to be projected to obtain the importance distribution characteristics used to characterize the visual importance of each region in the image to be projected.

[0038] Importance distribution characteristics refer to the distribution information obtained through saliency analysis, used to characterize the visual importance of various regions in a projected image. This characteristic indicates, in numerical or graphical form, which parts of the image are core content and which parts are secondary content.

[0039] Saliency analysis can be implemented using various algorithms. For example, contrast-based algorithms can be used to assess the saliency of each pixel by calculating the color or brightness difference between each pixel and its surrounding pixels. Frequency-domain-based algorithms can also be used to identify visual focal points by analyzing the spectral characteristics of the image. Alternatively, a pre-trained neural network model can be used, taking the image to be projected as input and outputting a saliency map where brightness values ​​or color intensities represent the visual importance of the corresponding regions. For instance, in an image containing both people and a background, the area containing the people is assigned a higher importance value, while the background area has a relatively lower importance value.

[0040] Step S104: Determine the target projection area in the projection plane based on the importance distribution characteristics, the location characteristics and semantic category characteristics of the obstacles.

[0041] The target projection area refers to the effective area determined for the actual projection of the image after considering obstacles on the projection surface and the importance distribution of the image to be projected. This area is optimized to retain the most important content of the image while avoiding obstacles.

[0042] Based on the importance distribution characteristics, as well as the location and semantic category characteristics of obstacles, the target projection area in the projection plane is determined. First, the location and semantic category characteristics of obstacles can be visualized on the projection plane to clarify their occupancy. Then, combining the importance distribution characteristics of the image to be projected, the coverage of important content and obstacle avoidance are evaluated under different candidate projection areas. For example, a simple rule can be set: prioritize avoiding all obstacles and select an area from the remaining available area that can accommodate the image to be projected and maximize the coverage of important content as the target projection area. Alternatively, an iterative search can be used to find the area that provides the best projection effect for important areas of the image, while satisfying obstacle avoidance conditions. For example, if the central area of ​​the image has high importance, the determination of the target projection area will prioritize ensuring that this central area is not obscured by obstacles.

[0043] Step S105: The surface deformation method is used to map the image to be projected onto the target projection area, and the degree of deformation of each area in the image to be projected is controlled during the mapping process according to the importance distribution characteristics.

[0044] The surface deformation method maps the image to be projected onto the target projection area. A mesh-based deformation method can be used, dividing the image into multiple small meshes and then adjusting the vertex positions of these meshes according to the shape of the target projection area to achieve overall image deformation. Alternatively, a physics-based simulation method can be used, treating the image as an elastic film and applying a force field to adapt it to the shape of the target area. Controlling the degree of deformation of different areas in the image during the mapping process based on their importance distribution characteristics allows for differentiated processing of deformation in different areas. For example, for highly important areas, the degree of deformation can be limited to maintain their original proportions and shape as much as possible to avoid visual distortion. Conversely, less important background areas can be allowed greater deformation to better adapt to the boundaries of the target projection area.

[0045] Therefore, the projection image correction method provided in this embodiment demonstrates significant technical contributions. By introducing multimodal sensing data acquisition and obstacle feature extraction, this embodiment enables the projection device to actively perceive and identify local obstacles on the projection surface. This contrasts sharply with existing technologies that correct obstacles solely based on the overall geometry of the projection surface, greatly improving the precision and accuracy of obstacle perception. Furthermore, this embodiment introduces saliency analysis of the projection image and obtains importance distribution characteristics, enabling an understanding of the visual priority of the image content and thus allowing for intelligent decision-making during image correction. This overcomes the shortcomings of existing methods that indiscriminately deform all areas of the image, easily leading to distortion of important content. This embodiment introduces a method that comprehensively considers the location characteristics, semantic category characteristics, and importance distribution characteristics of obstacles to determine the target projection area, achieving a balance between intelligent obstacle avoidance and content protection, effectively solving problems of image occlusion, distortion, and cumbersome operation. Finally, a surface deformation method is adopted, and the degree of deformation is controlled according to the distribution characteristics of importance. This ensures that the corrected image can adapt to irregular projection areas while preserving the visual quality of important content to the greatest extent, avoiding the discomfort caused by image distortion or obstruction in existing technologies. Therefore, the technical solution of this embodiment has significant progress in improving the intelligence, precision, and user experience of projection image correction.

[0046] See Figure 2 In one embodiment, the method for extracting features of obstacles in the projection plane includes, but is not limited to, steps S201 to S204.

[0047] Step S201: Based on the multimodal sensing data, extract the difference features of the projection surface to obtain the corresponding depth difference features, color difference features and texture difference features.

[0048] Step S202: Identify the outline boundaries of obstacles based on depth difference features, color difference features, and texture difference features.

[0049] Step S203: Determine the location features of the obstacle based on the location features of the contour boundary.

[0050] Step S204: Determine the semantic category features of the obstacle based on the geometric morphological features of the contour boundary.

[0051] When extracting difference features from a projection surface, depth, color, and texture information from multimodal sensing data can be utilized. Depth difference features can be calculated using depth maps or point cloud data acquired by depth sensors (e.g., structured light sensors, Time-of-Flight (ToF) cameras, or LiDAR), obtained by comparing the depth values ​​of local regions with the depth values ​​of the surrounding background or a pre-defined planar model. Color difference features can be extracted from color images acquired by RGB cameras, for example, by analyzing the distribution of pixel color values ​​in a specific color space (such as HSV, Lab) or significant deviations from the background color. Texture difference features can be obtained by converting the color image to grayscale and then calculating texture descriptors (such as Local Binary Pattern (LBP), Gabor features, or Gray-Level Co-occurrence Matrix (GLCM)) to quantify the roughness, directionality, or contrast of the texture, thereby identifying regions with textures different from the projection surface background.

[0052] When identifying the contour boundaries of obstacles, the extracted depth difference features, color difference features, and texture difference features can be comprehensively utilized. For example, these difference feature maps can be fused, and then image segmentation algorithms, such as threshold-based segmentation, region growing algorithms, level set methods, or graph cut methods, can be applied to accurately delineate the contours of obstacles. These methods can leverage the complementarity of different modal information to improve the accuracy and robustness of contour recognition.

[0053] When determining the positional characteristics of an obstacle, calculations can be performed based on the identified contour boundary. For example, the center coordinates of the smallest bounding rectangle or smallest bounding circle of the contour boundary can be calculated as the two-dimensional position of the obstacle on the projection plane. Furthermore, the dimensions of the bounding rectangle can also serve as an indication of the obstacle's size.

[0054] When determining the semantic category features of obstacles, the geometric features of the contour boundary and the convex height features of the obstacle relative to the projection surface can be analyzed. Geometric features can include the area features of the region enclosed by the contour boundary (e.g., by calculating the number of pixels within the contour), aspect ratio features (e.g., by calculating the aspect ratio of the minimum bounding rectangle), and roundness features (e.g., by calculating the ratio of the contour perimeter to the area). The convex height features of the obstacle relative to the projection surface can be obtained by combining depth difference features, calculating the difference between the average depth value of the region inside the obstacle contour and the background depth value of the projection surface. These geometric features and convex height features can be used as inputs and analyzed by a pre-trained classifier (such as a Support Vector Machine (SVM), decision tree, or neural network) to classify obstacles into different semantic categories, such as "book," "water cup," and "hand."

[0055] This application's solution comprehensively extracts depth, color, and texture difference features from multimodal sensing data, and accurately identifies the contour boundaries of obstacles based on this. Furthermore, by analyzing the geometric morphological features of the contour boundaries and the protrusion height features of the obstacles, the location and semantic category of the obstacles can be accurately determined. This multi-dimensional and refined feature extraction method enables the system to perceive obstacles on the projection surface more accurately and robustly, overcoming the limitations of single-modal or generalized feature extraction. Accurate identification of obstacle location and semantic category provides reliable and detailed input for subsequent intelligent planning of the target projection area and surface deformation mapping, thereby ensuring that the projected image can effectively avoid obstacles and adopt more reasonable avoidance strategies based on obstacle type.

[0056] In some embodiments, identifying the contour boundary of an obstacle based on depth difference features, color difference features, and texture difference features includes: determining the depth boundary position of the obstacle and the depth deviation region relative to the projection plane based on the depth difference features; determining the local color deviation region of the obstacle based on the color difference features; determining the texture complexity anomalous region of the obstacle based on the texture difference features; constructing an energy function based on the depth boundary position, depth deviation region, color deviation region, and texture complexity anomalous region; and solving the energy function using a graph cut segmentation method to obtain the contour boundary of the obstacle.

[0057] Determining the depth boundary locations of obstacles and depth deviation regions relative to the projection surface based on depth difference characteristics can be achieved by performing gradient calculations or edge detection algorithms (such as Sobel and Canny operators) on the depth image of the projection surface. Simultaneously, by setting a depth threshold, depth values ​​are compared with the reference depth of the projection surface; regions exceeding the threshold range can be identified as depth deviation regions. Alternatively, a region-growing-based depth segmentation method can be used to identify regions in the depth image that are significantly different from the depth of the projection surface; the boundaries of these regions are the depth boundary locations, and the regions themselves constitute the depth deviation regions.

[0058] Local color deviation regions of obstacles can be identified based on color difference characteristics. This can be achieved by calculating the color distance (e.g., Euclidean distance in RGB, HSV, and Lab color spaces) between pixels in the color image of the projection surface and their neighboring pixels or with the background color model of the projection surface. Regions with color distances exceeding a preset threshold are marked as local color deviation regions. Alternatively, color clustering algorithms (e.g., K-means) can be used to perform color segmentation on the image, identifying regions that are inconsistent with the background color clusters of the projection surface as local color deviation regions.

[0059] To identify regions with abnormal texture complexity in obstacles, texture difference features can be used. This can be achieved by calculating the texture features of local regions in the color image of the projection surface (e.g., Gray-Level Co-occurrence Matrix (GLCM), Gabor filter response, Local Binary Pattern (LBP)) and comparing them with the background texture features of the projection surface. Regions with significant differences in texture features or abnormal complexity indicators (e.g., entropy, energy) are identified as regions with abnormal texture complexity. Alternatively, frequency domain analysis methods based on wavelet transform or Fourier transform can be used to detect the frequency component distribution of local image regions, identifying regions with significantly different frequency distributions from the background texture as regions with abnormal texture complexity.

[0060] Energy functions typically include a data term (measuring the likelihood of a pixel belonging to a certain category) and a smoothing term (encouraging adjacent pixels to have the same label). The data term can be defined based on a pixel's behavior at depth boundary locations, depth deviation regions, color deviation regions, and texture complexity anomaly regions; for example, if a pixel is located in these regions, its classification as an obstacle is less costly. Constructing an energy function based on depth boundary locations, depth deviation regions, color deviation regions, and texture complexity anomaly regions can employ a Markov Random Field (MRF)-based energy function model. These discrepancies are used as observation data for the MRF, and the energy function is constructed by defining node potentials and edge potentials. Node potentials reflect the probability of a pixel belonging to an obstacle, while edge potentials reflect the tendency for adjacent pixels to have consistent labels.

[0061] This application's solution involves in-depth analysis of multimodal sensing data. First, it extracts the depth boundary location and depth deviation region of obstacles from depth difference features, local color deviation regions from color difference features, and texture complexity anomaly regions from texture difference features. These different modal differences provide clues for distinguishing obstacles from the projection surface from their respective dimensions. To overcome the limitations of single features, this application uses these multimodal differences as input to construct a comprehensive energy function. This energy function quantifies the probability that each pixel belongs to an obstacle or background and considers the smoothness constraints between adjacent pixels, thus mathematically describing the optimization problem of obstacle contour boundaries. Subsequently, a graph cut segmentation method is used to solve the constructed energy function. The graph cut method can effectively find the minimum value of the energy function globally, thus obtaining an optimal segmentation result, i.e., an accurate obstacle contour boundary. This method effectively fuses multimodal information and avoids local optima through global optimization, significantly improving the accuracy and robustness of obstacle contour boundary recognition. It is particularly suitable for obstacle recognition in complex environments, providing a reliable foundation for subsequent target projection region determination.

[0062] See Figure 3In one embodiment, the method for performing saliency analysis on the projected image includes, but is not limited to, steps S301 to S303.

[0063] Step S301: Perform multi-scale spectral residual significance detection on the image to be projected to obtain significance maps at each scale, and fuse the significance maps to obtain a fused significance map.

[0064] Step S302: Perform semantic segmentation on the image to be projected to determine the semantic category of each pixel in the image to be projected and the semantic weight corresponding to the semantic category.

[0065] Step S303: Integrate the saliency map and semantic weights to obtain the importance distribution features.

[0066] To perform multi-scale spectral residual saliency detection on a projected image, a Fourier transform can be used to convert the image from the spatial domain to the frequency domain, calculate its logarithmic spectrum, and then estimate the background spectrum using methods such as Gaussian filtering. Subtracting the background spectrum from the original spectrum yields the spectral residual, which is then converted to a saliency map using an inverse Fourier transform. At multiple scales, the original image can be downsampled to different degrees or processed using filters of different sizes to generate multiple saliency maps. These saliency maps can be merged into a single fused saliency map using methods such as weighted averaging, maximum value fusion, or nonlinear fusion.

[0067] Determining the semantic category and corresponding semantic weight of each pixel in the image to be projected can be achieved using deep learning models, such as fully convolutional networks, to perform pixel-level classification of the image, thus obtaining the semantic category to which each pixel belongs. Semantic weights can be pre-set according to the application scenario; for example, faces, text, and key objects can be assigned higher weights, while backgrounds and less important areas can be assigned lower weights. Another approach is to combine traditional image segmentation algorithms (such as superpixel segmentation and region growing) with machine learning classifiers (such as support vector machines and random forests) to semantically label the segmented regions and define the weights of different semantic categories based on expert knowledge or user preferences.

[0068] By combining the saliency map and the semantic weights, the pixel values ​​in the fused saliency map can be multiplied or weighted and summed with their corresponding semantic weights to obtain the final importance distribution features. For example, if a pixel has a high value in the saliency map and its semantic category is assigned a high weight, then that pixel will have a higher value in the importance distribution features. Another approach is to design a fusion model that can generate importance distribution features based on the fused saliency map and semantic weights using nonlinear mapping or learning algorithms (such as neural networks) to better balance the contributions of both.

[0069] The proposed solution first performs multi-scale spectral residual saliency detection on the image to be projected, capturing its visual saliency at different scales from the perspective of low-level visual features, and generating a fused saliency map to reflect which areas in the image are visually more prominent. Simultaneously, semantic segmentation is performed on the image to be projected, identifying the semantic category of each pixel and assigning different importance to different semantic categories according to preset semantic weights, thereby understanding the importance of the image content from a high-level semantic perspective. Subsequently, the low-level visual saliency information (fused saliency map) and the high-level semantic importance information (semantic weights) are integrated to obtain a more comprehensive and accurate importance distribution feature. This importance distribution feature not only considers the visual saliency of the image but also incorporates the semantic value of the content, making the identification of important areas of the image more accurate. This comprehensive importance distribution feature is then used to determine the target projection area in the projection surface and control the degree of deformation of the image to be projected during the mapping process, ensuring that while avoiding obstacles, the display effect of important content is preserved and optimized to the greatest extent.

[0070] See Figure 4 In one embodiment, the method for determining the target projection area in the projection plane includes, but is not limited to, steps S401 to S404.

[0071] Step S401: An optimization model is constructed with the optimization objectives of maximizing the effective projection area, maximizing the coverage area of ​​each region in the image to be projected by weighting the distribution characteristics of importance, and minimizing geometric distortion.

[0072] Step S402: Determine the distribution location of the obstacles based on their location characteristics.

[0073] Step S403: Determine the safe avoidance distance for the obstacle based on its semantic category features.

[0074] Step S404: Using the distribution location of obstacles and the corresponding safe avoidance distance as constraints of the optimization model, solve the optimization model on the projection surface to determine the target projection area.

[0075] The constraint requires that the distance between the candidate projection area and the location of the obstacles be no less than the safe avoidance distance.

[0076] The optimization model can be a multi-objective optimization model, balancing various objectives through methods such as weighted summation or Pareto optimization; it can also be a constrained optimization model, transforming some objectives into constraints. Maximizing the effective projected area can be achieved by calculating the geometric area of ​​the candidate projection region or the number of pixels it contains. Maximizing the weighted coverage area of ​​each region in the image to be projected, based on its importance distribution characteristics, can be achieved by summing the importance values ​​of each pixel within the projection region, using the importance distribution characteristics as weights, or by calculating the coverage ratio of important regions within the projection region. Minimizing geometric distortion can be quantified by evaluating the relative positional changes between pixels before and after projection, the curvature of lines, or the local deformation rate of the image.

[0077] The distribution location of obstacles refers to the specific spatial coordinates or area occupied by the obstacles on the projection surface. It can be represented as the center point coordinates of the obstacles, the set of boundary points, or a mask area composed of pixels.

[0078] The distribution locations of obstacles and their corresponding safe avoidance distances are used as constraints in the optimization model. These constraints require that the distance between the candidate projection area and the location of the obstacles be no less than the safe avoidance distance. When solving the optimization model on the projection surface, gradient descent, genetic algorithms, simulated annealing, interior point methods, etc., can be used. The target projection area is the final result of solving the optimization model, that is, obtaining the optimal projection area on the projection surface that avoids obstacles while also considering image importance and geometric quality. This area can be a polygon, an irregular closed curve region, or a set of discrete pixels.

[0079] This application's solution transforms the problem of determining the target projection area into a multi-objective optimization problem, introducing the location and semantic category features of obstacles as key constraints to achieve intelligent planning of the projection area. First, the system utilizes the acquired obstacle location and semantic category information to set reasonable safe avoidance distances for different types of obstacles, making the avoidance strategy more refined and intelligent. Next, combining the importance distribution characteristics of the image to be projected, an optimization model is constructed that comprehensively considers the effective projection area, coverage of important content, and minimization of geometric distortion. During the solution process, this model strictly adheres to the constraint of maintaining a safe avoidance distance from obstacles, ensuring that the projected image does not encroach on the obstacle area. In this way, the solution can adaptively determine the optimal projection area in a complex and dynamic environment that effectively avoids obstacles while maximizing the visual effect and projection quality. This enables the projection system to provide a clearer, more complete, and user-expected projection experience when facing irregular projection surfaces and obstacles.

[0080] See Figure 5In one embodiment, the method for controlling the degree of deformation of each region in the image to be projected during the mapping process based on the importance distribution characteristics includes, but is not limited to, steps S501 to S504.

[0081] Step S501: Establish a surface control mesh on the target projection area.

[0082] Step S502: Determine the control points in the control grid based on the boundary of the target projection area.

[0083] Step S503: Configure the influence weights of control points according to the importance distribution characteristics.

[0084] Step S504: Map the image to be projected to the target projection area according to the influence weight.

[0085] A surface control mesh is established on the target projection area. This can be done using a parametric mesh, such as a B-spline surface or a NURBS surface, by defining control points and basis functions. Alternatively, a polygonal mesh can be used, discretizing the target projection area into a series of interconnected triangular or quadrilateral patches, with each vertex serving as a control point.

[0086] Control points are key nodes in the surface control mesh; their location and number directly affect the accuracy and flexibility of surface deformation. Control points are determined based on the boundary of the target projection region. This can be achieved by sampling points uniformly or non-uniformly along the boundary as boundary control points, and then generating internal control points within the region according to certain rules (such as uniform distribution or curvature-based distribution). Alternatively, image processing techniques can be used to identify the contour of the target projection region and determine control points based on contour features (such as corner points or points with large curvature changes).

[0087] Assigning influence weights based on importance distribution characteristics can be achieved by normalizing the importance distribution characteristics and then directly or through a mapping function assigning their values ​​to the influence weights of the corresponding control points. The importance of the region where a control point is located, as indicated by the importance distribution characteristics, is positively correlated with its influence weight; for example, the higher the importance value, the greater the weight. The influence weight can also be determined based on the average or maximum importance value of the area covered by the control point.

[0088] Mapping the image to be projected onto the target projection area can be achieved using a free-form deformation (FFD) method. The image is embedded into a control mesh, and the final mapped position of each pixel in the image is calculated based on the displacement of the control points and their influence weights. Alternatively, a thin-plate spline (TPS) interpolation method can be used, with control points as base points and influence weights as factors contributing to deformation, to calculate the global deformation function.

[0089] This application's solution transforms the complex problem of surface deformation into the manipulation of control points within a surface control mesh on the target projection area. Specifically, an operable surface control mesh is first constructed on the target projection area, providing the basic structure for subsequent deformation operations. Next, control points within the mesh are precisely determined based on the geometric boundaries of the target projection area; these control points are crucial for local or global deformation adjustments. To achieve deformation control over important areas of the image, this application introduces an importance distribution feature and assigns an influence weight to each control point based on this feature. This configuration ensures that the importance of the area where the control point is located, as indicated by the importance distribution feature, is positively correlated with the influence weight of that control point; that is, the more important the area, the greater the "resistance" to deformation of its corresponding control point during deformation. Finally, when mapping the image to be projected onto the target projection area, the deformation algorithm calculates based on these influence weights, ensuring that visually important areas maintain minimal deformation during mapping, while allocating larger deformations to less important areas, thereby optimizing the overall visual quality of the projected image. This mechanism cleverly combines the visual importance of the image content with geometric distortion control, enabling the projected image to adapt to complex projection surfaces while maximizing the clarity and readability of key information.

[0090] In some embodiments, the projection image correction method further includes: detecting anisotropic deformation regions generated during the mapping process of the image to be projected, and sharpening the anisotropic deformation regions along the compression direction; calculating a brightness attenuation compensation factor based on the projection distance and light incident angle corresponding to each pixel in the target projection region to compensate for the brightness of the target projection region; and / or determining the reflection influence range and shadow influence range of the obstacle's contour boundary on the projected light based on the surface reflection characteristics of the obstacle, and performing brightness pre-compensation on the region in the image to be projected adjacent to the contour boundary of the obstacle based on the reflection influence range and shadow influence range.

[0091] Detecting anisotropic deformation regions generated during the mapping process of the image to be projected can be achieved by analyzing the geometric transformation matrix between pixels before and after mapping, or by using image processing algorithms to identify non-uniform changes in local texture. Sharpening compensation for anisotropic deformation regions can be performed using unsharp masking algorithms or gradient-based edge enhancement algorithms, applying specific sharpening filters along the compression direction of the deformation to restore image sharpness. Another approach is to analyze the local deformation tensor of the deformed region to determine the principal compression direction and apply an adaptive convolution kernel for sharpening along this direction.

[0092] The brightness attenuation compensation factor is calculated based on the projection distance and light incidence angle of each pixel in the target projection area. This can be achieved by establishing a physical model, such as based on Lambert's cosine law and the inverse square law of distance, to estimate the theoretical brightness attenuation of each pixel. Alternatively, the projection system can be pre-calibrated to obtain brightness attenuation curves at different distances and angles, and then the compensation factor can be obtained through table lookup or interpolation. Brightness compensation for the target projection area can be achieved by adjusting the projector's light output intensity or by performing gamma correction or multiplicative adjustment on pixel values ​​during digital image processing.

[0093] Based on the surface reflection characteristics of obstacles, the influence range of the obstacle's outline boundary on the reflection and shadow effects of projected light is determined. This can be achieved by simulating the interaction between the projected light and the obstacle using ray tracing algorithms, or by estimating the influence range based on the shape of the obstacle and the relative position of the projector using geometric calculations. Based on the reflection and shadow influence ranges, brightness pre-compensation is performed on the areas of the image to be projected that are adjacent to the outline boundary of the obstacle. Within the reflection influence range, the projection brightness can be appropriately reduced to avoid overexposure; within the shadow influence range, the projection brightness can be appropriately increased to offset the brightness loss caused by shadows. Alternatively, based on the material properties of the obstacle (such as specular reflection and diffuse reflection coefficients) and the projector's lighting model, the intensity distribution of reflected light and shadows is accurately calculated, and the pixel brightness of adjacent areas is adjusted accordingly.

[0094] This application, based on the aforementioned projection image correction method, further focuses on optimizing the visual quality of the projected image. First, after the image to be projected is mapped onto the target projection area, anisotropic distortion may occur in localized areas, leading to blurred image details. This application detects these distorted areas and performs sharpening compensation along their compression direction, effectively restoring image clarity and ensuring the integrity of image details. Second, since the projection surface is not an ideal plane, the distance from each pixel to the projector and the angle of light incidence differ, resulting in uneven brightness in the projected image. This application calculates the projection distance and angle of light incidence corresponding to each pixel to accurately estimate brightness attenuation, and calculates a brightness attenuation compensation factor accordingly, thereby compensating for brightness across the entire target projection area to achieve uniform image brightness. Furthermore, obstacles on the projection surface not only obstruct the projection area, but their surface characteristics also reflect projected light or create shadows, further affecting image quality. This application analyzes the surface reflection characteristics of obstacles to accurately determine their reflection and shadow influence ranges, and performs brightness pre-compensation on these affected areas. For example, reducing brightness in reflective areas to avoid glare and increasing brightness in shadow areas to reduce dark areas eliminates visual interference from obstacles on the projected image. Through the above series of post-processing optimizations, this application ensures that even in complex projection environments, the final projected image maintains high definition, uniform brightness, and good visual effects, significantly improving the user experience.

[0095] In some embodiments, the projection image correction method further includes: constructing a cost function based on the position and size characteristics of obstacles in the projection surface; performing inter-frame tracking of obstacles in the projection surface based on the cost function to determine the motion state of the obstacles; locally updating control points when the state change exceeds a preset state change threshold; and gradually expanding the projection area of ​​the projection surface according to a preset transition duration parameter when the obstacle is detected to disappear, so as to reclaim the area occupied by the obstacle on the projection surface.

[0096] The cost function is constructed based on the positional and dimensional features of obstacles in the projection plane. It can be based on factors such as the degree of overlap between the obstacle and the target projection area, and the degree of occlusion of important areas by the obstacle. Pixels within the obstacle area are assigned a higher cost, while pixels at the obstacle's edge are assigned a medium cost. Alternatively, the size features (such as area and perimeter) and positional features (such as center coordinates and bounding box) of the obstacle can be used as input, and the cost can be calculated using a weighted sum or a nonlinear function (such as a Gaussian function), so that larger obstacles and those closer to important areas have higher costs.

[0097] Based on the cost function, obstacles in the projection plane are tracked between frames to determine their motion state variables. State estimation algorithms such as Kalman filtering and particle filtering can be used, treating changes in the cost function as observations to predict and update the obstacle's motion state (e.g., position, velocity, acceleration). Alternatively, visual tracking techniques based on feature point matching or optical flow can be used, combined with the cost function, to assist in obstacle identification and tracking. A significant change in the cost function distribution of an obstacle across consecutive frames indicates that it has moved.

[0098] When an obstacle moves significantly, the control points are locally updated to adapt to the new environment. This local update can be performed only on the area affected by the obstacle. It can involve recalculating the weights or positions of control points near the obstacle, or employing a region-based update strategy that updates only control points adjacent to or overlapping with the moving obstacle. The update process may include recalculating the influence weights of the control points or adjusting their positions on the target projection area to ensure the projected image avoids the new obstacle location.

[0099] When an obstacle is detected to have disappeared, the projection area of ​​the projection surface is gradually expanded according to a preset transition duration parameter to reclaim the area occupied by the obstacle. When the obstacle is removed from the projection surface, the projection space it occupied is smoothly restored to avoid abrupt screen changes. Obstacle disappearance can be detected by a significant decrease or disappearance of the obstacle cost function in consecutive frames. The preset transition duration parameter can be a time period (e.g., 1 second, 2 seconds), during which the system gradually incorporates the original obstacle area into the target projection area in a linear or non-linear manner. Alternatively, during the transition, the boundary of the target projection area can be gradually adjusted, or the cost of the original obstacle area can be gradually reduced, making it gradually considered a projectable area. For example, by using interpolation or smoothing functions, the projection area can be gradually expanded over multiple frames, making the screen transition natural and avoiding noticeable area jumps for the user.

[0100] This application's solution quantifies the impact of obstacles on projection by constructing a cost function and uses this cost function to perform inter-frame tracking of obstacles, thereby acquiring the motion state of obstacles in real time. When an obstacle moves significantly, the system can promptly update the control points of the target projection area locally, ensuring that the projected image dynamically adapts to changes in the obstacle's position. Furthermore, when an obstacle disappears, the system does not immediately reclaim the area it occupied, but rather gradually expands the projection area according to a preset transition duration parameter, thus achieving a smooth and natural image transition. This dynamic tracking, adaptive updating, and smooth reclamation mechanism enables the projection image correction method to effectively cope with dynamic changes in obstacles in the projection environment, ensuring the accuracy and continuity of the projection. In some embodiments, the projection image correction method further includes: acquiring the pose change of the projection device detected by the inertial measurement unit; when the pose change exceeds a preset pose change threshold, recalculating the projection position of the obstacle in the current pose based on the pre-stored three-dimensional spatial coordinates of the obstacle, and updating the target projection area based on the projection position of the obstacle in the current pose.

[0101] The system recalculates the projected position of the obstacle in the current pose based on the pre-stored 3D spatial coordinates of the obstacle. Using this projected position, an optimization model can be reconstructed to maximize the effective projected area, weighted coverage area, and minimize geometric distortion. The new obstacle projected position is then used as a constraint for the solution. Alternatively, the system can maintain a dynamic map of the obstacle projection area. When the obstacle's projected position is updated, the corresponding unprojectable area is directly adjusted on the map, and then the target projection area is replanned based on the remaining projectable area.

[0102] The solution proposed in this application acquires the pose change of the projection device detected by the inertial measurement unit and compares it with a preset pose change threshold. When the detected pose change exceeds the threshold, instead of relying on the old obstacle projection position, it uses pre-stored three-dimensional spatial coordinates of the obstacle, combined with the precise pose information of the current projection device, to recalculate the accurate position of the obstacle on the current projection plane. Based on this updated obstacle position information, the target projection area can be dynamically redefined, thereby ensuring that the projected image always accurately avoids obstacles and makes full use of the projectable space. This dynamic adaptation mechanism enables the projection system to maintain the accuracy and integrity of the projection even when the pose of the projection device changes, effectively solving the problem of overlapping or deviation between the projected image and obstacles caused by device movement.

[0103] This application also provides an electronic device. (See appendix.) Figure 6 , Figure 6 This is a schematic diagram of the main structure of an electronic device according to an embodiment of this application. Figure 6 As shown, the electronic device in this embodiment mainly includes a processor 601 and a memory 602. The memory 602 can be configured to store a program for executing the methods of the above-described method embodiments, and the processor 601 can be configured to execute the program in the memory 602. This program includes, but is not limited to, a program for executing the methods of the above-described method embodiments. For ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application.

[0104] In some possible embodiments of this application, the electronic device may include multiple processors 601 and multiple memories 602. The program executing the methods of the above method embodiments may be divided into multiple subroutines, each of which may be loaded and run by a processor 601 to perform different steps of the methods of the above method embodiments. Specifically, each subroutine may be stored in a different memory 602, and each processor 601 may be configured to execute programs in one or more memories 602 to jointly implement the methods of the above method embodiments; that is, each processor 601 executes different steps of the methods of the above method embodiments to jointly implement the methods of the above method embodiments.

[0105] The aforementioned multiple processors 601 can be processors deployed on the same device. For example, the aforementioned electronic device can be a high-performance device composed of multiple processors, and the aforementioned multiple processors 601 can be processors configured on that high-performance device. Alternatively, the aforementioned multiple processors 601 can also be processors deployed on different devices. For example, the aforementioned electronic device can be a server cluster, and the aforementioned multiple processors 601 can be processors on different servers within the server cluster.

[0106] This application also provides a computer-readable storage medium. In one embodiment of the computer-readable storage medium according to this application, the computer-readable storage medium can be configured to store a program that performs the methods of the above-described method embodiments, the program of which can be loaded and run by a processor to implement the above-described methods. For ease of explanation, only the parts related to the embodiments of this application are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of this application. The computer-readable storage medium can be a memory formed by various electronic devices. Optionally, in the embodiments of this application, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0107] The projection image correction method and electronic device provided in this application actively perceive the positional and semantic category features of obstacles on the projection surface based on multimodal sensing data of the projection surface. By performing saliency analysis on the image to be projected, the importance distribution features representing the visual importance of each region in the image to be projected are determined, which can improve the precision of obstacle perception and the visual priority of understanding the image content. The target projection area is determined by combining the positional features, semantic category features, and importance distribution features of the obstacles, and the image to be projected is mapped to the target projection area. The degree of deformation of each region in the image to be projected is controlled according to the importance distribution features during the mapping process, which can realize intelligent obstacle avoidance and content protection. This ensures that the corrected image can adapt to irregular projection areas and dynamically adapt to obstacles on the projection surface, while preserving the visual quality of important content to the greatest extent, thereby improving the coverage integrity and visual comfort of the projection image.

[0108] Exemplary embodiments of this disclosure have been specifically shown and described above. It should be understood that this disclosure is not limited to the detailed structures, arrangements, or implementations described herein; rather, this disclosure is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. A method for correcting projected images, characterized in that, include: Acquire the image to be projected and collect multimodal sensing data of the projection surface; Based on the multimodal sensing data, features are extracted from the obstacles in the projection surface to obtain the location features and semantic category features of the obstacles; A saliency analysis is performed on the image to be projected to obtain the importance distribution characteristics that characterize the visual importance of each region in the image to be projected. Based on the importance distribution characteristics, as well as the location and semantic category characteristics of the obstacles, the target projection area in the projection plane is determined; The surface deformation method is used to map the image to be projected onto the target projection area, and the degree of deformation of each area in the image to be projected is controlled during the mapping process according to the importance distribution characteristics.

2. The projection image correction method according to claim 1, characterized in that, The step of extracting features from obstacles in the projection surface based on the multimodal sensing data includes: Based on the multimodal sensing data, the projection surface is subjected to difference feature extraction to obtain the corresponding depth difference features, color difference features and texture difference features; The contour boundary of the obstacle is identified based on the depth difference feature, the color difference feature, and the texture difference feature; The positional features of the obstacle are determined based on the positional features of the contour boundary; Based on the geometric morphological features of the outline boundary, the semantic category features of the obstacle are determined.

3. The projection image correction method according to claim 2, characterized in that, The step of identifying the contour boundary of the obstacle based on the depth difference feature, the color difference feature, and the texture difference feature includes: Based on the depth difference characteristics, determine the depth boundary position of the obstacle and the depth deviation region relative to the projection surface; Based on the color difference characteristics, the local color deviation area of ​​the obstacle is determined; Based on the texture difference characteristics, the regions of abnormal texture complexity of the obstacle are determined; An energy function is constructed based on the depth boundary location, the depth deviation region, the color deviation region, and the texture complexity anomaly region; The energy function is solved using the graph cut segmentation method to obtain the outline boundary of the obstacle.

4. The projection image correction method according to claim 1, characterized in that, The saliency analysis of the image to be projected includes: Multi-scale spectral residual significance detection is performed on the image to be projected to obtain significance maps at each scale. The significance maps are then fused to obtain a fused significance map. The image to be projected is semantically segmented to determine the semantic category of each pixel in the image to be projected and the semantic weight corresponding to the semantic category. The importance distribution features are obtained by combining the fusion saliency map and the semantic weights.

5. The projection image correction method according to claim 1, characterized in that, The step of determining the target projection region in the projection plane based on the importance distribution characteristics and the location and semantic category characteristics of the obstacles includes: An optimization model is constructed with the optimization objectives of maximizing the effective projection area, maximizing the coverage area of ​​each region in the image to be projected according to the importance distribution characteristics, and minimizing geometric distortion. The distribution location of the obstacles is determined based on their location characteristics; Based on the semantic category features of the obstacle, determine the safe avoidance distance corresponding to the obstacle; The distribution locations of the obstacles and their corresponding safe avoidance distances are used as constraints on the optimization model. The optimization model is solved on the projection surface to determine the target projection area. The constraints require that the distance between the candidate projection area and the distribution locations of the obstacles be no less than the safe avoidance distance.

6. The projection image correction method according to claim 1, characterized in that, The step of controlling the degree of deformation of each region in the image to be projected during the mapping process based on the importance distribution characteristics includes: A curved surface control mesh is established on the target projection area; Based on the boundary of the target projection area, determine the control points in the control grid; The influence weights of the control points are configured according to the importance distribution characteristics; the importance of the region where the control point is located in the importance distribution characteristics is positively correlated with the influence weights. Based on the influence weight, the image to be projected is mapped to the target projection area.

7. The projection image correction method according to claim 1, characterized in that, Also includes: The anisotropic deformation region generated during the mapping process of the image to be projected is detected, and the anisotropic deformation region is sharpened and compensated along the compression direction. Based on the projection distance and light incident angle of each pixel in the target projection area, a brightness attenuation compensation factor is calculated to compensate the brightness of the target projection area. and / or Based on the surface reflection characteristics of the obstacle, the range of reflection and shadow influence of the obstacle's outline boundary on the projected light is determined. Based on the range of reflection and shadow influence, brightness pre-compensation is performed on the area in the image to be projected that is adjacent to the outline boundary of the obstacle.

8. The projection image correction method according to claim 1, characterized in that, Also includes: Based on the position and size characteristics of the obstacles in the projection plane, a cost function is constructed; Based on the cost function, inter-frame tracking is performed on the obstacles in the projection plane to determine the motion state variables of the obstacles; When the state change exceeds a preset state change threshold, the control point is locally updated. When the obstacle is detected to have disappeared, the projection area of ​​the projection surface is gradually expanded according to a preset transition duration parameter, so as to reclaim the area occupied by the obstacle on the projection surface.

9. The projection image correction method according to claim 1, characterized in that, Also includes: Acquire the pose changes of the projection device detected by the inertial measurement unit; When the pose change exceeds a preset pose change threshold, the projected position of the obstacle in the current pose is recalculated based on the pre-stored three-dimensional spatial coordinates of the obstacle, and the target projection area is updated based on the projected position of the obstacle in the current pose.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the projection image correction method according to any one of claims 1 to 9.

Images