Method, device and readable medium for aligning projection screen with projection image

Abstract

The present invention discloses a method, device and readable medium for aligning a projection screen with a projection picture. Before using the method, automatic trapezoidal correction is required to obtain a first projection quadrilateral. The first projection picture obtained after the trapezoidal correction is subjected to perspective transformation to obtain a second projection picture. The projectable area of ​​the screen is determined according to the positional relationship between the second projection picture and the screen. The second projection picture is proportionally scaled and centered within the projectable area of ​​the screen to obtain a third projection picture. The second projection quadrilateral is calculated according to the affine transformation relationship between the second projection picture and the third projection picture and the first projection quadrilateral. The second offsets of the four corresponding corner points are calculated according to the second projection quadrilateral. The first projection picture is moved according to the second offsets to obtain an aligned picture, so that the projection picture is centered with the screen at the same picture ratio when the screen is not fully photographed.

Description

Technical Field

[0001] The present invention relates to the field of projection, and in particular to a method, a device and a readable medium for aligning a projection screen with a projection image. Background Art

[0002] Projection screens are currently one of the most commonly used peripherals for projectors. Properly paired with a projector, they deliver excellent viewing quality. However, achieving optimal viewing often requires strict projector placement. Furthermore, when the entire screen cannot be captured due to factors such as ambient occlusion, achieving a good viewing experience is even more challenging. Therefore, current projectors offer keystone correction, allowing users to manually adjust the displayable area of ​​the projected image for optimal viewing. However, manual operation offers a poor user experience and is challenging in the presence of ambient occlusion. If the screen is not fully captured, it can be difficult to achieve center alignment of the projected image with equal aspect ratios, causing the projected image to extend beyond the screen's visible area, thus affecting the projection quality. Summary of the Invention

[0003] In response to the technical problems mentioned above, the purpose of the embodiments of the present application is to propose a method, device and readable medium for aligning a projection screen with a projection image, so as to solve the technical problems mentioned in the above background technology section, so that the projection image can be aligned with the center of the screen with the same aspect ratio even when the screen is not fully photographed.

[0004] In a first aspect, the present invention provides a method for aligning a projection screen with a projection image, comprising the following steps:

[0005] S1, performing automatic trapezoidal correction on the projected image, obtaining four-point offsets of the trapezoidal correction as first offsets, and obtaining a first projected quadrilateral based on the first offsets;

[0006] S2, obtaining an image of the feature map projected for screen alignment captured by the camera as a target image, wherein the target image presents the first projection image formed by the projected feature map and the surrounding screen;

[0007] S3, performing feature detection on the target image to obtain a feature detection result, and performing perspective transformation on the first projection image according to the feature detection result to obtain a second projection image and its corner point coordinates;

[0008] S4, in response to determining that the screen has a first edge located within the second projection screen and a second edge located outside the second projection screen, determining a projectable area of ​​the screen according to the first edge and an edge of the second projection screen adjacent to the second edge;

[0009] S5, maximizing the centering of the second projection image in the projectable area of ​​the screen according to the projection image ratio, to obtain a third projection image and its corner point coordinates;

[0010] S6, calculating an affine transformation matrix based on the translation and scaling relationship between the coordinates of the corner points of the second projection image and the coordinates of the corresponding corner points of the third projection image; obtaining a second projected quadrilateral based on the first projected quadrilateral and the affine transformation matrix;

[0011] S7, calculating second offsets of four corresponding corner points according to the second projected quadrilateral; and moving the first projected image according to the second offsets to obtain an aligned image.

[0012] Preferably, the feature image includes a feature rectangle and feature dots, the feature rectangle is located in the middle of the first projection screen, and the length and width of the feature rectangle are 1 / n of the length and width of the first projection screen, the feature dots are respectively located on the diagonal extension lines outside the four corners of the feature rectangle, and the center of the rectangle formed by connecting the centers of the four feature dots coincides with the center of the feature rectangle.

[0013] Preferably, in step S3, feature detection is performed on the target image to obtain feature detection results, specifically including:

[0014] Perform edge detection on the target image to obtain edge detection results;

[0015] Determine the characteristic points in the target image according to the edge detection result;

[0016] Determine a first area containing a characteristic rectangle according to the centers of the four characteristic points;

[0017] Corner point detection is performed on the first area to determine the coordinates of the corner points of the feature rectangle in the target image and use them as the coordinates of the first rectangle.

[0018] Preferably, in step S3, perspective transformation is performed on the first projection image according to the feature detection result to obtain the second projection image, which specifically includes:

[0019] A second rectangle is set according to the spatial position of the feature rectangle projected on the screen and the resolution of the target image captured by the camera, and the coordinates of the second rectangle are determined, wherein the length and width of the second rectangle are in proportional scaling relationship with the length and width of the feature rectangle;

[0020] A perspective transformation matrix is ​​determined according to the coordinates of the first rectangle and the second rectangle, and a perspective transformation is performed on the first projection image to obtain the second projection image and the coordinates of its corner points.

[0021] Preferably, step S5 specifically includes:

[0022] Calculate the aspect ratio of the projectable area of ​​the screen according to the coordinates of the corner points of the projectable area of ​​the screen;

[0023] In response to determining that the aspect ratio of the projectable area of ​​the screen is inconsistent with the projection resolution of the projection device, scaling the second projection image proportionally to the projection resolution of the projection device to within the projectable area of ​​the screen to obtain a scaled projection image;

[0024] The scaled projection image is centered in the projectable area of ​​the screen to obtain a third projection image;

[0025] The coordinates of the corner points of the third projection image are calculated according to the width and height of the scaled projection image.

[0026] Preferably, obtaining the second projected quadrilateral according to the first projected quadrilateral and the affine transformation matrix in step S6 specifically includes:

[0027] According to the affine transformation matrix and the four-point coordinates of the first projected quadrilateral, calculate the four-point coordinates of the second projected quadrilateral:

[0028]

[0029] in, Represent the four-point column and four-dimensional row vector of the row coordinates of the first projected quadrilateral, Represent the four-point column and four-dimensional row vector of the row coordinates of the second projected quadrilateral, is the affine transformation matrix, where m 00 、m 01 、m 10 、m 11 Indicates the coefficient of rotation and scaling, m 02 、m 12 Indicates the amount of translation.

[0030] Preferably, calculating the second offsets of the four corresponding corner points according to the second projected quadrilateral in step S7 specifically includes:

[0031] The offset of the corresponding corner point of the second projection quadrilateral is calculated according to the coordinates of the corner points of the second projection quadrilateral and the width or height of the projection resolution, and the offset is proportionally corrected according to the preset maximum projection image to obtain the second offset.

[0032] In a third aspect, the present invention provides a device for aligning a projection screen with a projection image, characterized in that it includes:

[0033] A trapezoidal correction module is configured to perform automatic trapezoidal correction on the projected image, obtain four-point offsets of the trapezoidal correction as first offsets, and obtain a first projected quadrilateral based on the first offsets;

[0034] a projection shooting module configured to obtain an image of the feature map projected for screen alignment captured by the camera as a target image, wherein the target image presents a first projection image formed by the projected feature map and the surrounding screen;

[0035] a perspective transformation module configured to perform feature detection on the target image to obtain a feature detection result, and perform perspective transformation on the first projection image according to the feature detection result to obtain a second projection image and its corner point coordinates;

[0036] a projectable area determination module configured to, in response to determining that the screen has a first edge located within the second projection screen and a second edge located outside the second projection screen, determine the projectable area of ​​the screen according to the first edge and an edge of the second projection screen adjacent to the second edge;

[0037] A proportional centering module is configured to maximize the centering of the second projection image in the projectable area of ​​the screen according to the projection image ratio to obtain a third projection image and its corner point coordinates;

[0038] An affine transformation module is configured to calculate an affine transformation matrix according to a translation and scaling relationship from the coordinates of the corner points of the second projection picture to the coordinates of the corner points of the corresponding third projection picture;

[0039] The offset module is configured to calculate second offsets of four corresponding corner points according to the second projected quadrilateral; and move the first projected image according to the second offsets to obtain an aligned image.

[0040] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0041] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] (1) The method for aligning a projection screen and a projection image proposed in the present invention first performs automatic trapezoidal correction to obtain a first projection quadrilateral, performs perspective transformation on the first projection image obtained after the trapezoidal correction to obtain a second projection image, determines the projectable area of ​​the screen according to the positional relationship between the second projection image and the screen, and scales the second projection image proportionally to be centered within the projectable area of ​​the screen to obtain a third projection image, calculates a second projection quadrilateral according to the affine transformation relationship between the second projection image and the third projection image and the first projection quadrilateral, calculates second offsets of the four corresponding corner points according to the second projection quadrilateral; and moves the first projection image according to the second offset to obtain an aligned image.

[0044] (2) The method for aligning the projection screen with the projection picture proposed in the present invention replaces the edge of the second projection picture with the second edge of the screen that is outside the second projection picture or is not photographed. Even if the screen is not fully photographed, the projectable area of ​​the screen can be obtained, thereby avoiding the projection picture from exceeding the projectable area of ​​the screen and achieving a better projection effect.

[0045] (3) The method for aligning the projection screen and the projected image proposed in the present invention can be used for aligning the projection screen in home scenes. When the user uses a projection device to project the screen and turns on the function of this method, the projection image on the screen can be aligned in the center with the same aspect ratio even if the screen is not fully photographed, so that the user can complete the screen alignment without manually adjusting the keystone or four-point correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 is a diagram of an exemplary device architecture to which an embodiment of the present application may be applied;

[0048] Figure 2 A schematic flow chart of a method for aligning a projection screen with a projection image according to an embodiment of the present application;

[0049] Figure 3 A schematic diagram of a projection situation when the third projection picture includes only three screen edges in the method for aligning the projection screen and the projection picture according to an embodiment of the present application;

[0050] Figure 4A schematic diagram of a projection situation when the third projection picture contains only two adjacent screen edges in the method for aligning the projection screen and the projection picture according to an embodiment of the present application;

[0051] Figure 5 A schematic diagram of a projection situation when the third projection picture contains only two parallel screen edges in the method for aligning the projection screen and the projection picture according to an embodiment of the present application;

[0052] Figure 6 A schematic diagram of a projection situation when the third projection picture includes only one screen edge in the method for aligning the projection screen and the projection picture according to an embodiment of the present application;

[0053] Figure 7 for Figure 3 Schematic diagram of the corresponding alignment screen;

[0054] Figure 8 for Figure 4 Schematic diagram of the corresponding alignment screen;

[0055] Figure 9 for Figure 5 Schematic diagram of the corresponding alignment screen;

[0056] Figure 10 for Figure 6 Schematic diagram of the corresponding alignment screen;

[0057] Figure 11 Schematic diagram of the relative positional relationship between the first projected quadrilateral and the second projected quadrilateral in an embodiment of the present application;

[0058] Figure 12 A schematic diagram of a device for aligning a projection screen and a projection image according to an embodiment of the present application;

[0059] Figure 13 It is a structural diagram of a computer device suitable for implementing the electronic device of the embodiment of the present application. DETAILED DESCRIPTION

[0060] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0061] Figure 1 An exemplary device architecture 100 is shown to which the method for aligning a projection screen with a projection image or the device for aligning a projection screen with a projection image according to an embodiment of the present application can be applied.

[0062] like Figure 1 As shown, the device architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0063] Users can use terminal devices 101, 102, 103 to interact with server 105 via network 104 to receive or send messages, etc. Various applications, such as data processing applications and file processing applications, can be installed on terminal devices 101, 102, 103.

[0064] Terminal devices 101, 102, and 103 can be hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software or software modules (for example, software or software modules used to provide distributed services), or they can be implemented as a single software or software module. No specific limitations are given here.

[0065] The server 105 may be a server that provides various services, such as a background data processing server that processes files or data uploaded by the terminal devices 101, 102, and 103. The background data processing server may process the acquired files or data and generate processing results.

[0066] It should be noted that the method for aligning the projection screen and the projection image provided in the embodiment of the present application can be executed by the server 105 or by the terminal devices 101, 102, and 103. Accordingly, the device for aligning the projection screen and the projection image can be set in the server 105 or in the terminal devices 101, 102, and 103.

[0067] It should be understood that Figure 1 The number of terminal devices, networks, and servers in the above description is merely illustrative. Any number of terminal devices, networks, and servers may be provided as needed. If the processed data does not need to be acquired remotely, the above-described apparatus architecture may not include a network, but only require servers or terminal devices.

[0068] Figure 2 A method for aligning a projection screen with a projection image provided by an embodiment of the present application is shown, comprising the following steps:

[0069] S1, automatically perform trapezoidal correction on the projected image, obtain four-point offsets of the trapezoidal correction as first offsets, and obtain a first projected quadrilateral according to the first offsets.

[0070] Specifically, when using a projection device, if it is improperly positioned or projected sideways, the image will be distorted, making it impossible to ensure a square image. In this case, the projected image needs to be keystone corrected. After keystone correction, the projection surface and the screen surface are aligned in the same plane, facilitating subsequent correction of the first projected image and ensuring more accurate alignment between the screen and the projected image. Keystone correction can be performed using various methods, such as using a camera to capture the projected image or using a distance sensor, and keystone correction methods are not limited to these.

[0071] S2, obtaining an image of the feature map projected for screen alignment captured by the camera as a target image, wherein the target image presents a first projection image formed by the projected feature map and the screen surrounding it.

[0072] In a specific embodiment, a feature map is projected onto a screen to obtain a first projection image, and the first projection image is captured by a camera. The offset of the four corner points of the first projection image obtained after trapezoidal correction, i.e., the first offset, can be read. Preferably, the first offset is a four-point offset set on the optical machine of the projection device. The feature map includes a feature rectangle and a feature circle. The feature rectangle is located in the middle of the second projection image, and the length and width of the feature rectangle are 1 / n of the length and width of the second projection image. The feature circles are located on the diagonal extension lines outside the four corners of the feature rectangle, and the center of the rectangle formed by connecting the centers of the four feature circles coincides with the center of the feature rectangle. By setting the feature map, it is convenient to perform an affine transformation in the subsequent process to align the projection images with equal aspect ratio and center alignment. The second offset in the optical machine is further calculated so that the final projection image can still be aligned with equal aspect ratio and center alignment even when the screen is not fully captured. In other embodiments, other suitable feature maps can also be used.

[0073] S3, performing feature detection on the target image to obtain a feature detection result, performing perspective transformation on the first projection image according to the feature detection result, and obtaining a second projection image and its corner point coordinates.

[0074] In a specific embodiment, step S3 performs feature detection on the target image to obtain feature detection results, which specifically includes:

[0075] Perform edge detection on the target image to obtain edge detection results;

[0076] Determine the characteristic points in the target image based on the edge detection results.

[0077] Determine a first area containing a characteristic rectangle according to the centers of the four characteristic points;

[0078] Corner point detection is performed on the first area to determine the coordinates of the corner points of the feature rectangle in the target image and use them as the coordinates of the first rectangle.

[0079] In a specific embodiment, step S3 performs perspective transformation on the first projection image according to the feature detection result to obtain the second projection image and its corner point coordinates, which specifically includes:

[0080] A second rectangle is set according to the spatial position of the feature rectangle projected on the screen and the resolution of the target image captured by the camera, and the coordinates of the second rectangle are determined, wherein the length and width of the second rectangle are in proportional scaling relationship with the length and width of the feature rectangle;

[0081] A perspective transformation matrix is ​​determined according to the coordinates of the first rectangle and the second rectangle, and a perspective transformation is performed on the first projection image to obtain the second projection image and the coordinates of its corner points.

[0082] In one embodiment, the findContours function in OpenCV can be used to perform edge detection on the target image. Based on information such as the area and perimeter of the edge enclosed in the edge detection results, four characteristic circular points are selected, and the center of mass of the area enclosed by the four characteristic circular points is used as the center coordinate. To reduce irrelevant background interference, the internal area where the characteristic rectangle is located is cut out based on the center coordinates of the four characteristic circular points, and the coordinates of the corner points of the characteristic rectangle are obtained using corner point detection. These coordinates are marked as the coordinates of the first rectangle. The coordinates of the first rectangle are measured in the coordinate system of the target image captured by the camera.

[0083] Specifically, a second rectangle is set based on the spatial position of the feature rectangle and the resolution of the captured image. The second rectangle is preferably close to the feature rectangle after geometric scaling. A perspective transformation is performed on the target image based on the coordinates of the first and second rectangles. The resulting corner coordinates of the second projected image are the coordinates of the second rectangle's corners magnified n times by the center of the second projected image. In one embodiment, the feature rectangle is set to 0.25 times the length and width of the first projected image and is centered in the image. The resulting corner coordinates of the second projected image are the coordinates of the second rectangle's corners magnified 4 times by the center of the second projected image. This magnified rectangle is used to simulate the second projected image to facilitate edge determination. Because the first projected image in the target image captured by the camera may be distorted, the first and second rectangles are used to perform a perspective transformation on the first projected image to obtain a second projected image whose corresponding edges are parallel to the edges of the screen. Based on the second projected image, the first and second edges are then searched for, and affine transformations such as translation, rotation, and scaling are performed on the two-dimensional plane to achieve more accurate alignment.

[0084] Generally, the edge of the second projection image is outside the edge of the screen. However, if the edge of the screen partially exceeds the second projection image due to factors such as the placement of the projection equipment, resulting in incomplete capture of the screen, the following will appear: Figure 3-6 In the projection situation shown, the interior of the gray box is divided into the second projection screen in the embodiment of the present application, wherein the black dots and black solid rectangles are the characteristic dots and characteristic rectangles in the second projection screen, and the black lines represent the edges of the screen. Figure 3 Indicates the projection situation when only the edges of the three screens are included in the second projection image; Figure 4 Indicates the projection situation when the second projection image contains only two adjacent screen edges; Figure 5 Indicates the projection situation when the second projection image contains only two parallel screen edges; Figure 6 This represents the projection situation when the second projection image only contains one screen edge. Therefore, the feature map cannot be guaranteed to be centered with the same aspect ratio in the above projection situation, which affects the projection effect.

[0085] S4, determining that the screen has a first edge located within the second projection screen and a second edge located outside the second projection screen, and determining a projectable area of ​​the screen according to the first edge and an edge of the second projection screen adjacent to the second edge.

[0086] Specifically, in the four projection scenarios above, if the screen has a first edge within the second projection image and a second edge outside the second projection image or not captured, the edge of the second projection image adjacent to and parallel to the second edge is used as a proxy for the second edge to determine the projectable area within the screen. Therefore, the projectable area within the screen is the area enclosed by the first edge and the edge of the second projection image adjacent to and parallel to the second edge. Contour detection is used to obtain the outline of the screen's projectable area, and line detection is used to obtain the equation of the line containing the edge of the screen or the second projection image. The intersection of the lines is calculated based on the equation, and the coordinates of the intersection are the coordinates of the corner points of the screen's projectable area.

[0087] S5, maximizing the centering of the second projection image in the projectable area of ​​the screen according to the projection image ratio, to obtain a third projection image and its corner point coordinates.

[0088] In a specific embodiment, step S5 specifically includes:

[0089] Calculate the aspect ratio of the projectable area of ​​the screen according to the coordinates of the corner points of the projectable area of ​​the screen;

[0090] In response to determining that the aspect ratio of the projectable area of ​​the screen is inconsistent with the projection resolution of the projection device, scaling the second projection image proportionally to the projection resolution of the projection device to within the projectable area of ​​the screen to obtain a scaled projection image;

[0091] The scaled projection image is centered in the projectable area of ​​the screen to obtain a third projection image;

[0092] The coordinates of the corner points of the third projection image are calculated according to the width and height of the scaled projection image.

[0093] Specifically, since the second projection image has a portion that exceeds the projectable area of ​​the screen, and the aspect ratio of the projectable area of ​​the screen is not necessarily the same as the projection screen ratio of the projection device, the second projection image needs to be scaled proportionally to conform to the projection screen ratio of the projection device and centered in the projectable area of ​​the screen to obtain a virtual third projection image. Figure 3-6 The corresponding third projection images are Figure 7-10 At this time, the third projection image is aligned and centered within the projectable area of ​​the screen, preventing the projection image from exceeding the projectable area of ​​the screen and achieving a good projection effect. The rectangular corner points of the third projection image are magnified n times and used as the corner point coordinates of the third projection image.

[0094] S6, calculating an affine transformation matrix based on the translation and scaling relationship from the corner coordinates of the second projection screen to the corresponding corner coordinates of the third projection screen; obtaining a second projection quadrilateral based on the first projection quadrilateral and the affine transformation matrix.

[0095] Specifically, the affine transformation matrix can be solved by substituting specific inputs and outputs. Since the translation and scaling relationship from the corner coordinates of the second projection screen to the corner coordinates of the third projection screen is consistent with the translation and scaling relationship from the first projection quadrilateral to the second projection quadrilateral, the same affine transformation matrix can be used. A matrix solution can be performed using the known corner coordinates of the third projection screen and the corner coordinates of the second projection screen to determine the values ​​of each element in the affine transformation matrix and obtain the affine transformation matrix.

[0096] In a specific embodiment, obtaining the second projected quadrilateral according to the first projected quadrilateral and the affine transformation matrix in step S6 specifically includes:

[0097] According to the calculated affine transformation matrix and the four-point coordinates of the first projected quadrilateral, calculate the four-point coordinates of the second projected quadrilateral:

[0098]

[0099] in, The four-point column and four-dimensional row vector of the row coordinates of the first projection quadrilateral, Represent the four-point column and four-dimensional row vector of the row coordinates of the second projected quadrilateral, is the affine transformation matrix, where m 00 、m 01 、m 10 、m 11 Indicates the coefficient of rotation and scaling, m 02 、m 12 Indicates the amount of translation.

[0100] Specifically, since there is an affine transformation relationship between the second projection screen and the third projection screen, an affine transformation matrix can be calculated based on the second projection screen and the third projection screen, and the first projection quadrilateral is affine transformed according to the affine transformation matrix to obtain the second projection quadrilateral.

[0101] S7, calculating second offsets of the four corresponding corner points according to the second projected quadrilateral; and moving the projected image according to the second offsets to obtain an aligned image.

[0102] In a specific embodiment, calculating the second offsets of the four corresponding corner points according to the second projected quadrilateral in step S7 specifically includes:

[0103] The offset of the corresponding corner point of the second projection quadrilateral is calculated according to the coordinates of the corner points of the second projection quadrilateral and the width or height of the projection resolution, and the offset is proportionally corrected according to the preset maximum projection image to obtain the second offset.

[0104] Specifically, the first projection image is a trapezoidal-corrected image, so it is necessary to calculate the first projection quadrilateral based on the first offset, calculate the second projection quadrilateral based on the first projection quadrilateral and the affine transformation matrix, and then calculate the second offset by calculating the four point coordinates in the second projection quadrilateral relative to the point coordinates of the original projection image. The second offset is the final offset that needs to be set on the optical machine.

[0105] Specifically, the second offset is calculated as follows:

[0106] refer to Figure 11 , taking the offset value calculation of the upper left corner as an example:

[0107] TopRatio=(WIDTH–x 0’ –x 1’ ) / top;

[0108] LeftRatio=(HEIGHT–y 0’ –y 3’ ) / left;

[0109] X=x0*TopRatio;

[0110] Y=y0*LeftRatio;

[0111] Among them, the outer quadrilateral is the edge of the default maximum projection screen, but due to the geometric scaling in the actual affine transformation, it is necessary to scale the four-point offset calculated in the second projection quadrilateral accordingly, and calculate the scaling ratio by the calculation formula of TopRatio and LeftRatio. In this embodiment, the minimum positive circumscribed moment of the second projection quadrilateral is obtained, top represents the width of the circumscribed moment, left represents the height of the circumscribed moment, WIDTH represents the pixel value of the width of the maximum screen that the optical machine can project, which is 1920 in this embodiment, HEIGHT represents the pixel value of the height of the maximum screen that the optical machine can project, which is 1080 in this embodiment, and x i ',y i ', i's value range is {0, 1, 2, 3}, representing the first offsets corresponding to the four corner points. X, Y, i.e., the second offsets, are calculated by scaling. The offsets for the four corner points of the projected image are set based on the second offsets. The four corner points of the second projected image after keystone correction are moved accordingly to achieve the desired aligned image.

[0112] Further references Figure 12 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a device for aligning a projection screen with a projection image. Figure 2Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0113] An embodiment of the present application provides a device for aligning a projection screen with a projection image, comprising:

[0114] The trapezoidal correction module 1 is configured to automatically perform trapezoidal correction on the projected image, obtain four-point offsets of the trapezoidal correction as first offsets, and obtain a first projected quadrilateral according to the first offsets;

[0115] The projection shooting module 2 is configured to obtain an image of the feature map projected for screen alignment captured by the camera as a target image, wherein the target image presents a first projection image formed by the projection feature map and the screen surrounding it;

[0116] The perspective transformation module 3 is configured to perform feature detection on the target image to obtain a feature detection result, and perform perspective transformation on the first projection image according to the feature detection result to obtain a second projection image and its corner point coordinates;

[0117] a projectable area determining module 4 configured to, in response to determining that the screen has a first edge located within the second projection picture and a second edge located outside the second projection picture, determine the projectable area of ​​the screen based on the first edge and an edge of the second projection picture adjacent to the second edge;

[0118] The proportional centering module 5 is configured to maximize the centering of the second projection image in the projectable area of ​​the screen according to the projection image ratio to obtain the third projection image and its corner point coordinates;

[0119] An affine transformation module 6 is configured to calculate an affine transformation matrix based on a translation and scaling relationship between the coordinates of the corner points of the second projection image and the coordinates of the corner points of the corresponding third projection image; and obtain a second projected quadrilateral based on the first projected quadrilateral and the affine transformation matrix.

[0120] The offset module 7 is configured to calculate second offsets of the four corresponding corner points according to the second projected quadrilateral; and move the first projected image according to the second offsets to obtain an aligned image.

[0121] Reference below Figure 13 , which shows an electronic device (eg Figure 1 Schematic diagram of the structure of a computer device 1300 (server or terminal device shown). Figure 13 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0122] like Figure 13As shown, the computer device 1300 includes a central processing unit (CPU) 1301 and a graphics processing unit (GPU) 1302, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1303 or the program loaded from the storage part 1309 to the random access memory (RAM) 1304. Various programs and data required for the operation of the device 1300 are also stored in the RAM 1304. The CPU 1301, GPU 1302, ROM 1303 and RAM 1304 are connected to each other via a bus 1305. An input / output (I / O) interface 1306 is also connected to the bus 1305.

[0123] The following components are connected to the I / O interface 1306: an input section 1307 including a keyboard, a mouse, and the like; an output section 1308 including, for example, a liquid crystal display (LCD), and a speaker; a storage section 1309 including, for example, a hard disk; and a communication section 1310 including, for example, a network interface card such as a LAN card or a modem. The communication section 1310 performs communication processing via a network such as the Internet. A drive 1311 may also be connected to the I / O interface 1306 as needed. Removable media 1312, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1311 as needed, so that computer programs read therefrom can be installed into the storage section 1309 as needed.

[0124] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1310, and / or installed from the removable medium 1312. When the computer program is executed by the central processing unit (CPU) 1301 and the graphics processing unit (GPU) 1302, the above-mentioned functions defined in the method of the present application are performed.

[0125] It should be noted that the computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable medium, or any combination thereof. Computer-readable media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, apparatuses, or components, or any combination thereof. More specific examples of computer-readable media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution device, apparatus, or component. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution apparatus, device, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, or any suitable combination thereof.

[0126] Computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions and operations of the devices, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based device that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0128] The modules involved in the embodiments described in this application may be implemented in software or hardware, and may also be set in a processor.

[0129] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or it may exist independently and not be assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device: performs automatic trapezoidal correction on the projection screen, obtains the four-point offset of the trapezoidal correction as the first offset, and obtains the first projected quadrilateral according to the first offset; obtains the image of the feature map projected for screen alignment captured by the camera as the target screen, and the target screen presents the first projection screen formed by the projection feature map and the screen around it; performs feature detection on the target screen to obtain feature detection results, performs perspective transformation on the first projection screen according to the feature detection results, and obtains the second projection screen and its corner coordinates; in response to determining that the screen has A first edge located within the second projection screen and a second edge located outside the second projection screen, and a projectable area of ​​the screen are determined according to the first edge and the edge of the second projection screen adjacent to the second edge; the second projection screen is centered in the projectable area of ​​the screen to maximize the projection screen ratio to obtain a third projection screen and its corner point coordinates; an affine transformation matrix is ​​calculated based on the translation and scaling relationship from the corner point coordinates of the second projection screen to the corresponding corner point coordinates of the third projection screen; a second projection quadrilateral is obtained based on the first projection quadrilateral and the affine transformation matrix; second offsets of the corresponding four corner points are calculated based on the second projection quadrilateral; and the first projection screen is moved according to the second offsets to obtain an aligned screen.

[0130] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for aligning a projection screen with a projection image, characterized in that: The following steps are involved: S1, performing automatic trapezoidal correction on the projected image, obtaining four-point offsets of the trapezoidal correction as first offsets, and obtaining a first projected quadrilateral based on the first offsets; S2, obtaining an image of the feature map projected for screen alignment captured by the camera as a target image, wherein the target image presents a first projection image formed by projecting the feature map and the surrounding screen; S3, performing feature detection on the target image to obtain a feature detection result, and performing perspective transformation on the first projection image according to the feature detection result to obtain a second projection image and its corner point coordinates; S4, in response to determining that the screen has a first edge located within the second projection screen and a second edge located outside the second projection screen, determining a projectable area of ​​the screen according to the first edge and an edge of the second projection screen adjacent to the second edge; S5, maximizing the projection ratio of the second projection image and centering it in the projectable area of ​​the screen to obtain a third projection image and its corner point coordinates; S6, calculating an affine transformation matrix based on the translation and scaling relationship between the coordinates of the corner points of the second projection image and the corresponding coordinates of the corner points of the third projection image; obtaining a second projected quadrilateral based on the first projected quadrilateral and the affine transformation matrix; S7, calculating second offsets of four corresponding corner points according to the second projected quadrilateral; and moving the first projected image according to the second offsets to obtain an aligned image.

2. The method for aligning a projection screen with a projection image according to claim 1, wherein: The feature image includes a feature rectangle and a feature dot. The feature rectangle is located in the middle of the first projection screen, and the length and width of the feature rectangle are 1 / n of the length and width of the first projection screen. The feature dots are respectively located on the diagonal extension lines outside the four corners of the feature rectangle, and the center of the rectangle formed by connecting the centers of the four feature dots coincides with the center of the feature rectangle.

3. The method for aligning a projection screen with a projection image according to claim 2, wherein: In step S3, feature detection is performed on the target image to obtain feature detection results, which specifically includes: Performing edge detection on the target image to obtain an edge detection result; Determining characteristic points in the target image according to the edge detection result; Determine a first area containing the characteristic rectangle according to the centers of the four characteristic points; Corner point detection is performed on the first area to determine the coordinates of the corner points of the feature rectangle in the target image, and use them as the coordinates of the first rectangle.

4. The method for aligning a projection screen with a projection image according to claim 3, wherein: The step S3 performs perspective transformation on the first projection image according to the feature detection result to obtain the second projection image, which specifically includes: A second rectangle is set according to the spatial position of the feature rectangle projected on the screen and the resolution of the target image captured by the camera, and the coordinates of the second rectangle are determined, wherein the length and width of the second rectangle are in proportional scaling relationship with the length and width of the feature rectangle; A perspective transformation matrix is ​​determined according to the coordinates of the first rectangle and the second rectangle, and a perspective transformation is performed on the first projection image to obtain the second projection image and the coordinates of its corner points.

5. The method for aligning a projection screen with a projection image according to claim 1, wherein: The step S5 specifically includes: Calculating the aspect ratio of the projection area of ​​the screen according to the coordinates of the corner points of the projection area of ​​the screen; In response to determining that the aspect ratio of the projectable area of ​​the screen is inconsistent with the projection resolution of the projection device, scaling the second projection image proportionally to the projection area of ​​the screen according to the projection resolution of the projection device to obtain a scaled projection image; Centering the scaled projection image in the projectable area of ​​the screen to obtain the third projection image; The coordinates of the corner points of the third projection image are calculated according to the width and height of the scaled projection image.

6. The method for aligning a projection screen with a projection image according to claim 1, wherein: Obtaining a second projected quadrilateral according to the first projected quadrilateral and the affine transformation matrix in step S6 specifically includes: Calculate the four-point coordinates of the second projected quadrilateral based on the affine transformation matrix and the four-point coordinates of the first projected quadrilateral: in, Represent the four-point column and four-dimensional row vector of the row coordinates of the first projected quadrilateral, Represent the four-point column and four-dimensional row vector of the row coordinates of the second projected quadrilateral, is the affine transformation matrix, where m 00 、m 01 、m 10 、m 11 Indicates the coefficient of rotation and scaling, m 02 、m 12 Indicates the amount of translation.

7. The method for aligning a projection screen with a projection image according to claim 1, wherein: Calculating the second offsets of the four corresponding corner points according to the second projected quadrilateral in step S7 specifically includes: The offset of the corresponding corner point of the second projection quadrilateral is calculated according to the coordinates of the corner points of the second projection quadrilateral and the width or height of the projection resolution, and the offset is proportionally corrected according to the preset maximum projection image to obtain the second offset.

8. A device for aligning a projection screen with a projection image, characterized in that: include: a trapezoidal correction module configured to perform automatic trapezoidal correction on the projected image, obtain four-point offsets of the trapezoidal correction as first offsets, and obtain a first projected quadrilateral based on the first offsets; a projection shooting module configured to obtain an image of the feature map projected for screen alignment captured by a camera as a target image, wherein the target image presents a first projection image formed by projecting the feature map and the screen surrounding it; a perspective transformation module configured to perform feature detection on the target image to obtain a feature detection result, and perform perspective transformation on the first projection image according to the feature detection result to obtain a second projection image and its corner point coordinates; a projectable area determination module configured to, in response to determining that the screen has a first edge located within the second projection screen and a second edge located outside the second projection screen, determine the projectable area of ​​the screen according to the first edge and an edge of the second projection screen adjacent to the second edge; a proportional centering module configured to maximize the centering of the second projection image within the projectable area of ​​the screen according to the projection image ratio, to obtain a third projection image and its corner point coordinates; an affine transformation module configured to calculate an affine transformation matrix according to a translation and scaling relationship from the coordinates of the corner points of the second projection screen to the corresponding coordinates of the corner points of the third projection screen; Obtaining a second projected quadrilateral according to the first projected quadrilateral and the affine transformation matrix; an offset module, configured to calculate second offsets of four corresponding corner points according to the second projected quadrilateral; The first projection picture is moved according to the second offset to obtain an aligned picture.

9. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images