Projection screen entering method for dynamically generating self-adaptive projection picture

Abstract

The invention provides a projection in-screen method for dynamically generating a self-adaptive projection picture, and the method comprises the steps: enabling a projection in-screen function based on a camera when the projection in-screen function is started, and enabling a projection device to obtain a projection background wall color or a screen frame color, and the contrast ratio of the projection background wall color or the screen frame color to the projection feature map background color is low. A projection feature map generation module is used for dynamically generating a projection feature map with a relatively large contrast ratio with a projection background wall color or a frame color, so that four vertexes of a curtain frame can be identified, and a homography matrix between an original projection image and the shot projection feature map is solved by using the four feature maps in the projection map; the homography matrix plays a key role in the projection picture screen entering operation module, and the zoom relation of the projection image is solved according to the relation between the recognized four vertexes of the screen frame and the homography matrix so as to achieve the projection screen entering function.

Description

Technical Field

[0001] The present invention relates to the field of projecting into a screen of a home direction projection device, and particularly to a method for projecting into a screen that dynamically generates an adaptive projection image. Background Art

[0002] In the existing projection into screen technology in a projection device system, especially in the method for projecting into a screen implemented based on a camera, a pure white color is used as the background color of the projection image, and a black and white checkerboard or a pure black feature map is superimposed as the projection image. When using the projection into screen function, the calibration map near the four vertices of the projected projection image should cover or exceed the screen border. Only under this condition can the projection into screen function be realized. There are the following problems: Currently, the indoor background walls in the whole-house smart home environment are not limited to the previous pure white style, but various colored background walls with high personalization. When the contrast between the background wall color or the screen border color and the background color of the projection feature map is relatively low, it is difficult to identify the four vertices of the screen border in the projection image, ultimately affecting the projection into screen function. In view of the above problems, a method for projecting into a screen that dynamically generates an adaptive projection image is provided. By using the method of dynamically generating an adaptive projection feature image in the automatic into screen function, the problem that it is difficult to detect the screen border when the contrast between the projection background wall color or the screen border and the background color of the projection feature map is relatively low, and ultimately the poor effect of the projection into screen function is solved. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for projecting into a screen that dynamically generates an adaptive projection image in order to solve the above problems.

[0004] The present invention realizes the above purpose through the following technical solutions:

[0005] The present invention provides a method for projecting into a screen that dynamically generates an adaptive projection image, including the following steps:

[0006] Step S1: The projection module first projects a default projection feature image with a pure white background color onto the projection screen, and the projection area should cover the four vertices of the screen border;

[0007] Step S2: The image acquisition module takes back the projection feature image in Step S1. The acquired projection feature image needs to include the background wall and the screen border. Using an image processing algorithm, the background wall area, the projection area, and the screen border area are segmented from the taken-back projection feature image, and the screen border area is roughly calculated;

[0008] Step S3: The dynamic projection feature map generation module first converts the collected regions into grayscale images, and then uses the structural similarity algorithm to compare the similarities between regions, and calculates the structural similarity coefficients between the background wall region and the projection region and between the projection region and the curtain border region respectively;

[0009] Step S4: Determine whether a new projection feature map needs to be generated according to the structural similarity coefficients of each region described in Step S3. If so, the dynamic projection feature map generation module will use the feature map generation model to generate a new projection feature map based on the background image and the curtain border color and project it onto the projection curtain;

[0010] Step S5: Project the new projection feature map described in Step S4 onto the projection curtain, and obtain the new projection feature map on the curtain through the image acquisition module. Detect the boundary of the projection region, the boundary of the curtain border region, the four vertices of the projection region, and the four vertices of the curtain border from the newly captured projection image;

[0011] Step S6: Calculate according to the pixel offset between the four vertices of the projection region and the four vertices of the curtain border to complete the function of projecting into the curtain.

[0012] As a preference of the present invention, the default projection feature map in Step S1 refers to a pure white or pure black image.

[0013] As a preference of the present invention, the specific steps of splitting the background wall region, the projection region, and the curtain border region from the projection image feature map in Step S2 are as follows:

[0014] Step S21: Convert the projection feature map into a grayscale image, use the contour detection algorithm to identify each region, save the coordinate information of the inflection points of each region, and record it as the vertex information of the region. High precision is not required when detecting the region contour here;

[0015] Step S22: Calculate the rectangular coordinate point information of the background wall region according to the vertex information of the region. The projection region and the curtain border region should be removed from the background wall region.

[0016] As a preference of the present invention, according to the similarity between regions in step S3 judged by the structural similarity coefficient, the closer the structural similarity coefficient of two regional images is to 1, the more similar the two images are. During the experiment, when the structural similarity coefficient of two images with a solid color background is less than 0.95, the two regions can be well distinguished. When the structural similarity coefficient between the background wall region and the projection region or between the projection region and the curtain border region is greater than 0.95, it is considered that the image similarity between the background wall region and the projection region or between the projection region and the curtain border region is relatively high, and the edge detection effect will be poor. The dynamic projection feature map generation module will use the feature map generation model to generate a new projection feature map based on the background wall color and the curtain border color.

[0017] As a preference of the present invention, the calculation of the pixel offset in step S6 is based on the projection ratio, resolution, and aspect ratio of the projection image of the projection device, and combines the coordinate information of the four vertices of the projection region and the four vertices of the available curtain region to obtain the scaling factor of the projection image, thereby completing the function of projecting into the curtain operation.

[0018] As a preference of the present invention, the calculation of the pixel offset in step S6 includes the following steps:

[0019] Step S61: Record the four vertices of each region in the captured image in a clockwise or counterclockwise order. The four vertices of the curtain border region are denoted as Pm1(x1, y1), Pm2(x2, y2), Pm3(x3, y3), Pm4(x4, y4), and the four vertices of the projection region are denoted as Pt1(x1, y1), Pt2(x2, y2), Pt3(x3, y3), Pt4(x4, y4);

[0020] Step S62: The projection ratio of the projector is T, the distance between the current projector and the curtain is measured by the distance sensor as L, the projection resolution is 1920*1080, and the aspect ratio of the projection device image is 16:9;

[0021] According to the projection ratio T of the projector and the distance L between the current projector and the curtain, the width Wwt and height Hwt of the projection region in the world coordinate system are obtained, where Wwt = T*L, Hwt = (Wwt*9) / 16. The number of pixels contained in each physical space is Ppix = 1920 / Wwt. Since the image is enlarged proportionally, the number of pixels contained in each physical space in the length and width is the same. Therefore, the coordinates of the four vertices in the world coordinate system can be obtained as P wt 1(x1, y1), P wt 2(x2, y2), P wt 3(x3, y3), P wt 4(x4, y4);

[0022] The four vertex coordinates of the projection area in the world coordinate system start from the upper left corner. Assuming that the Z coordinate in the world coordinate system is 0, they are P in counterclockwise order wt 1(x1, y1) = (-0.5 * T * L, T * L * 16 / 9), P wt 2(x2, y2) = (-0.5 * T * L, 0), P wt 3(x3, y3) = (0.5 * T * L, 0), P wt 4(x4, y4) = (0.5 * T * L, T * L * 16 / 9);

[0023] Step S63: According to the four vertex image coordinates of the obtained projection area and the coordinates of these four vertices in the world coordinate system, calculate the homography matrix H between the image coordinates and the world coordinates;

[0024] Step S64: According to the vertex coordinates Pm of the curtain border area and the homography matrix H, calculate the coordinates P of the curtain border area in the image in the world coordinate system w m = Pm * H; The projection area coordinates P wt and the coordinates P of the curtain border area in the world coordinate system w The difference between m is ΔP, and the pixel value on the original feature map corresponding to ΔP is ΔP * Ppix. The function of projecting into the curtain is realized through the scaling information of each coordinate vertex coordinate.

[0025] The present invention also provides a projection - into - curtain system for dynamically generating an adaptive projection screen, including the following modules:

[0026] Projection module: Used to project a default projection feature image, which is default to a solid - color background image, and the projection feature map should cover the four vertices of the curtain border

[0027] Image acquisition module: Used to acquire the projection feature map on the projection area

[0028] Dynamic generation of projection feature map module: Used to determine whether the current projection feature map meets the requirements. When it does not meet the requirements, a new projection feature map needs to be regenerated according to the feature map generation model

[0029] Image processing module: Used to detect the boundary and vertex coordinates of the curtain border area, the boundary and vertex coordinates of the projection area from the acquired projection feature map, and calculate the scaling value of the projection image according to the projection ratio of the projection device, the current distance between the projection device and the curtain, and the resolution of the projection device

[0030] Projection screen into - curtain operation module: Used to scale the image according to the projection image scaling value calculated by the image processing module to realize the function of the into - curtain operation

[0031] The beneficial effect of the present invention is that the projection-on-screen function of the present invention is not affected by ambient light and the projection background wall, and a projection feature map can be dynamically generated according to the color of the background wall and the screen frame, thereby realizing the projection-on-screen function under special circumstances. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram showing that a projection feature map corresponding to an embodiment of the present invention is close in similarity to a background wall.

[0033] Figure 2 This is a schematic diagram showing that an automatically generated projection feature map corresponding to an embodiment of the present invention has low similarity with a background wall.

[0034] Figure 3 The present invention is a schematic block diagram of a projection-on-screen method for dynamically generating an adaptive projection picture corresponding to an embodiment of the present invention.

[0035] Figure 4 The present invention is a flowchart of a projection-on-screen method for dynamically generating an adaptive projection picture corresponding to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the objectives, technical solutions and advantages of the present invention clearer, the implementation method of the present invention will be described in detail below in conjunction with the accompanying drawings and embodiments, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0037] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0038] As attached Figures 1-4 As shown, an embodiment of the present invention provides a projection screen method for dynamically generating an adaptive projection picture, comprising the following steps:

[0039] Step S10: The projection module first projects a default pure white background color projection image onto the projection screen, and the projection area must cover the four vertices of the screen frame;

[0040] Step S20: an image acquisition module uses a camera built into the projection device to capture the projection feature image in S10, wherein the captured projection feature image needs to include the background wall and the curtain frame. An image processing algorithm is used to segment the captured projection feature image into the background wall area, the projection area and the curtain frame area, and roughly calculate the curtain black frame area.

[0041] Step S30: Dynamically generate a projection feature map module. Since the colors of the background wall area, the curtain border area, and the projection area are relatively single, this module first converts the collected areas into grayscale images, and then uses the structural similarity algorithm to compare the similarities between the areas, and calculates the structural similarity coefficient between the background wall area and the projection area and the structural similarity coefficient between the projection area and the curtain border area respectively.

[0042] Step S40: Determine whether a new projection feature map needs to be generated according to the structural similarity coefficients of the respective areas. If so, this module will use the feature map generation model to generate a new projection feature map based on the background image and the curtain border color and project it onto the projection curtain.

[0043] Step S50: Project the new projection feature map onto the projection curtain, and obtain the new projection feature map on the curtain through the camera. Detect the projection area boundary, the curtain border area boundary, the four vertices of the projection area, and the four vertices of the curtain border from the newly captured projection image.

[0044] Step S60: Complete the function of projecting into the curtain according to the pixel offset between the four vertices of the projection area and the four vertices of the curtain border.

[0045] As a preference of the embodiment of the present invention, the default projection feature map in step S10 refers to a pure white or pure black image.

[0046] As a preference of the embodiment of the present invention, specifically including in step S20: segmenting the background wall area, the projection area, and the curtain border area from the projection image feature map taken

[0047] Since the background color of the projection feature map used in the function of projecting into the curtain is relatively single and the contours are less, this module uses the contour detection method in image segmentation to segment the background image area, the projection area, and the curtain border area in the feature map.

[0048] Convert the projection feature map into a grayscale image, use the contour detection algorithm to identify each area, save the coordinate information of the inflection points of each area, and record it as the vertex information of the area. Here, high precision is not required for area contour detection.

[0049] Calculate the rectangular coordinate point information of the background wall area according to the vertex information of the area. The background wall area should remove the projection area and the curtain border area.

[0050] Preferably, as an embodiment of the present invention, there are many methods for calculating the similarity between the images of each region according to the step S30, such as histogram comparison, hash algorithm, mean square error, structural similarity and other algorithms. In this embodiment, the structural similarity algorithm is used to calculate the similarity between two images, but it is not limited to this method, and other methods can also be used to calculate the similarity between two images. The similarity between each region in the step S30 is evaluated by the structural similarity coefficient. When the structural similarity coefficient between two regional images is closer to 1, the two images are more similar. During the experiment, when the structural similarity coefficient of two images with a pure color background is less than 0.95, the two regions can already be well distinguished. In this article, the threshold of the structural similarity coefficient is taken as 0.95. When the structural similarity coefficient between the background wall region and the projection region or between the projection region and the curtain border region is greater than 0.95, it is considered that the image similarity between the background wall region and the projection region or between the projection region and the curtain border region is relatively high, and the edge detection effect will be poor. Then, this module will use the feature map generation model to generate a new projection feature map based on the background wall color and the curtain border color.

[0051] Using the feature map generation model to generate a new projection feature map based on the background wall color and the curtain border color. Since the projection feature map is a pure color background map, the pixel superposition method can be used to generate a new feature map. First, calculate the pixel mean value of each channel of the curtain border region image. The pixel value of each channel of the newly generated image feature map is generated by adding a threshold to the mean value of each channel of the curtain border region image. In this article, the threshold is taken as 100, as shown in the appendix Figure 2 As the newly generated projection feature map, this projection feature map is easier to accurately detect the curtain border. In the actual implementation process, other methods can also be used to generate a new projection feature image, such as deep learning, AI large models, etc., and it is not limited to a certain implementation method.

[0052] Preferably, as an embodiment of the present invention, the calculation of the pixel offset in the step S6 is based on the projection ratio, resolution, and aspect ratio of the projection image of the projection device, and combines the coordinate information of the four vertices of the projection region and the four vertices of the curtain area to calculate the scaling factor of the projection image and complete the function of projecting into the curtain operation.

[0053] Preferably, as an embodiment of the present invention, the method for calculating the pixel offset according to the step S60 includes:

[0054] The four vertices of each region in the captured image can be recorded in a clockwise or counterclockwise direction. The four vertices of the curtain border region are denoted as Pm1(x1, y1), Pm2(x2, y2), Pm3(x3, y3), Pm4(x4, y4), and the four vertices of the projection region are denoted as Pt1(x1, y1), Pt2(x2, y2), Pt3(x3, y3), Pt4(x4, y4);

[0055] Assume that the transmittance of the projector is T, the distance between the current projector and the screen is measured by a distance sensor as L, the projection resolution is 1920*1080, and the aspect ratio of the projection device image is 16:9;

[0056] According to the transmittance T of the projector and the distance L between the current projector and the screen, the width Wwt and height Hwt of the projection area in the world coordinate system are obtained, where Wwt = T * L, Hwt = (Wwt * 9) / 16, and the number of pixels contained in each physical space is Ppix = 1920 / Wwt. Since the image is enlarged proportionally, the number of pixels contained in each physical space in the length and width is the same. Therefore, the coordinates of the four vertices in the world coordinate system can be obtained as P wt 1(x1, y1), P wt 2(x2, y2), P wt 3(x3, y3), P wt 4(x4, y4);

[0057] The coordinates of the four vertices of the projection area in the world coordinate system start from the upper left corner. Assuming that the Z coordinate in the world coordinate system is 0, they are arranged counterclockwise as P wt 1(x1, y1) = (-0.5 * T * L, T * L * 16 / 9), P wt 2(x2, y2) = (-0.5 * T * L, 0), P wt 3(x3, y3) = (0.5 * T * L, 0), P wt 4(x4, y4) = (0.5 * T * L, T * L * 16 / 9);

[0058] According to the four vertex image coordinates of the obtained projection area and the coordinates of these four vertices in the world coordinate system, the homography matrix H between the image coordinates and the world coordinates is obtained.

[0059] According to the vertex coordinates Pm of the screen border area and the homography matrix H, the coordinates P of the screen border area in the image in the world coordinate system are obtained w m = Pm * H. The projection area coordinates P wt and the coordinates P of the screen border area in the world coordinate system w m differ by ΔP, and the pixel value on the original feature map corresponding to ΔP is ΔP * Ppix. The function of projecting into the screen is realized through the scaling information of each coordinate vertex coordinate.

[0060] As shown in the Figure 3 attachment, the embodiment of the present invention also provides a system block diagram of a projection-into-screen method for dynamically generating an adaptive projection screen, including:

[0061] Projection module: Used to project a default projection feature image, which is a solid-color background image by default, and the projection feature image should cover the four vertices of the curtain border.

[0062] Image acquisition module: Used to acquire the projection feature image on the projection area, as shown in the appendix Figure 1 as follows.

[0063] Dynamic projection feature image generation module: Used to determine whether the current projection feature image meets the requirements. When it does not meet the requirements, a new projection feature image needs to be regenerated according to the feature image generation model. As shown in the appendix Figure 1 is the default projection feature image, and the appendix Figure 2 is the newly generated projection feature image.

[0064] Image processing module: Used to detect the boundary and vertex coordinates of the curtain border area, the boundary and four vertex coordinates of the projection area from the acquired projection feature image, and calculate the scaling value of the projection image based on the transmittance of the projection device, the current distance between the projection device and the curtain, and the resolution of the projection device.

[0065] Projection screen entry operation module: Used to scale the image according to the scaling value of the projection image calculated by the image processing module to implement the entry operation function.

[0066] Although the present invention has been described herein with reference to its explanatory embodiments, the above embodiments are only preferred embodiments of the present invention. The embodiments of the present invention are not limited by the above embodiments. It should be understood that those skilled in the art can design many other modifications and embodiments, and these modifications and embodiments will fall within the scope and spirit of the principles disclosed in this application.

Claims

1. A projection screen method for dynamically generating an adaptive projection picture, characterized in that: The steps include: Step S1: The projection module first projects a default pure white background color projection feature image onto the projection screen, and the projection area must cover the four vertices of the screen border; Step S2: The image acquisition module captures the projection feature image in step S1, wherein the captured projection feature image needs to include the background wall and the curtain frame. The image processing algorithm is used to segment the background wall area, the projection area and the curtain frame area from the captured projection feature image, and roughly calculate the curtain frame area. Step S3: The module for dynamically generating projection feature maps first converts the collected regions into grayscale images, and then uses a structural similarity algorithm to compare the similarities between the regions, and calculates the structural similarity coefficients between the background wall region and the projection region and between the projection region and the curtain frame region respectively; Step S4: judging whether it is necessary to generate a new projection feature map according to the structural similarity coefficients of each region described in step S3; if necessary, the dynamic projection feature map generation module generates a new projection feature map based on the background image and the screen border color using the feature map generation model and projects it onto the projection screen; Step S5: Projecting the new projection feature map according to step S4 onto the projection screen, acquiring the new projection feature map on the screen through the image acquisition module, and detecting the projection area boundary and the screen frame area boundary as well as the four vertices of the projection area and the four vertices of the screen frame from the newly captured projection map; Step S6: Calculate the pixel offset between the four vertices of the projection area and the four vertices of the screen frame to complete the projection onto the screen function.

2. A projection method for dynamically generating an adaptive projection image according to claim 1, characterized in that: The default projection feature image in step S1 refers to a pure white or pure black image.

3. A projection screen method for dynamically generating an adaptive projection image according to claim 2, characterized in that: In step S2, segmenting the background wall area, the projection area, and the curtain frame area from the captured projection image feature map specifically includes the following steps: Step S21: converting the projection feature map into a grayscale map, identifying each region using a contour detection algorithm, saving the coordinate information of the turning points of each region, and recording it as the vertex information of the region. Here, the region contour detection does not require high accuracy; Step S22: Calculate the rectangular coordinate point information of the background wall area according to the vertex information of the area. The background wall area should exclude the projection area and the curtain border area.

4. The method for dynamically generating an adaptive projection image onto a screen according to claim 3, characterized in that: According to the similarity between the regions in step S3, the structural similarity coefficient is used to judge. When the structural similarity coefficient of the two region images is closer to 1, the two images are more similar. During the experiment, when the structural similarity coefficient of the two images with a pure color background is less than 0.95, the two regions can be well distinguished. When the structural similarity coefficient of the background wall area and the projection area or the projection area and the curtain border area is greater than 0.95, it is considered that the image similarity between the background wall area and the projection area or the projection area and the curtain border area is high, and the edge detection effect will be poor. The module for dynamically generating a projection feature map will use the feature map generation model to generate a new projection feature map based on the background wall color and the curtain border color.

5. The method for dynamically generating an adaptive projection image onto a screen according to claim 4, characterized in that: The calculation of the pixel offset in step S6 is based on the projection ratio, resolution, and aspect ratio of the projection image of the projection device, and combined with the coordinate information of the four vertices of the projection area and the four vertices of the screen area to calculate the scaling amount of the projection image and complete the projection onto the screen operation function.

6. A projection method for dynamically generating an adaptive projection image according to claim 5, characterized in that: The calculation of the pixel offset in step S6 includes the following steps: Step S61: Record the four vertices of each area in the captured image in a clockwise or counterclockwise direction. The four vertices of the curtain frame area are recorded as Pm1 (x1, y1), Pm2 (x2, y2), Pm3 (x3, y3), and Pm4 (x4, y4). The four vertices of the projection area are recorded as Pt1 (x1, y1), Pt2 (x2, y2), Pt3 (x3, y3), and Pt4 (x4, y4). Step S62: The projector transmittance is T, and the distance between the current projector and the screen is measured by the distance sensor as L, the projection resolution is 1920*1080, and the aspect ratio of the projection device image is 16:9; According to the transmittance ratio T of the projector and the distance L between the current projector and the screen, the width Wwt and height Hwt of the projection area in the world coordinate system are calculated, where Wwt = T*L, Hwt = (Wwt*9) / 16, and the number of pixels contained in each physical space is Ppix = 1920 / Wwt. Since the image is enlarged in the same proportion, the number of pixels contained in each physical space in length and width is the same, so the coordinates of the four vertices in the world coordinate system can be calculated as P wt 1(x1,y1),P wt 2(x2,y2),P wt 3(x3,y3),P wt 4(x4,y4); The coordinates of the four vertices of the projection area in the world coordinate system start from the upper left corner. Assuming that the Z coordinate in the world coordinate system is 0, the coordinates in the counterclockwise direction are P wt 1(x1, y1)=(-0.5*T*L, T*L*16 / 9), P wt 2(x2, y2)=(-0.5*T*L, 0), P wt 3(x3, y3)=(0.5*T*L, 0), P wt 4(x4, y4)=(0.5*T*L, T*L*16 / 9); Step S63: Calculate the homography matrix H between the image coordinates and the world coordinates according to the acquired image coordinates of the four vertices of the projection area and the coordinates of the four vertices in the world coordinate system; Step S64: Calculate the coordinates P of the curtain border region in the image in the world coordinate system according to the vertex coordinates Pm of the curtain border region and the homography matrix H w m=Pm*H;Projection area coordinate P wt and the coordinates P of the curtain frame coordinate area in the world coordinate system w The difference m is ΔP, and the pixel value on the original feature map corresponding to ΔP is ΔP*Ppix. The function of projecting onto the screen is realized through the scaling information of the coordinates of each vertex.

7. A projection screen system for dynamically generating an adaptive projection image, characterized in that: Includes the following modules, Projection module: used to project the default projection feature image, which is a solid-color background image by default. The projection feature image should cover the four vertices of the curtain border. Image acquisition module: used to collect the projection feature map on the projection area, Dynamically generate projection feature map module: used to determine whether the current projection feature map meets the requirements. If it does not meet the requirements, a new projection feature map needs to be regenerated according to the feature map generation model. Image processing module: used to detect the border and vertex coordinates of the screen frame area, the border and vertex coordinates of the projection area from the acquired projection feature map, and calculate the scaling value of the projection image according to the projection ratio of the projection device, the distance between the current projection device and the screen, and the resolution of the projection device. Projection picture entry operation module: used to scale the image according to the projection image scaling value obtained by the image processing module to realize the entry operation function.

Images