Projection onto screen method and projector

Abstract

The present application relates to a method for projecting onto a screen and a projector. The method comprises: acquiring a captured image, the captured image including a screen and a screen projected from an effective display area based on an initial screen; determining a homography transformation relationship based on the captured image and the initial screen; acquiring each closed contour in the captured image and determining the frame of the effective screen area within each closed contour; obtaining the corrected coordinates of the frame of the effective screen area in the initial screen based on the frame of the effective screen area and the homography transformation relationship; and adjusting the effective display area in the initial screen based on the corrected coordinates. This method for projecting onto a screen enables the projector's projected screen to be accurately projected onto the screen.

Description

Technical Field

[0001] The present application relates to the technical field of projectors, and in particular to a method for projecting onto a screen and a projector. Background Art

[0002] As consumer-grade projectors become increasingly affordable, their features become more intelligent, and their user experience becomes easier and easier to use, projectors have become ubiquitous in households. While current projectors can project directly onto a plain wall, many people opt to add a screen to enhance the viewing experience, achieving a more realistic image and richer colors. However, when using the projector in a living room or bedroom, it can be difficult to position and adjust the projector's angle to perfectly fit the screen. Typically, the projected image can only be manually adjusted by adjusting the four vertices of the screen, which can be inaccurate. Summary of the Invention

[0003] Based on this, it is necessary to provide a projection method and projector that can accurately project onto the screen to address the above technical problems.

[0004] A method for projecting onto a screen, comprising the following steps:

[0005] Acquire a captured image, where the captured image includes the screen and an image projected from an effective display area based on the initial image;

[0006] Determine the homography transformation relationship based on the captured image and the initial image;

[0007] Obtain each closed contour in the captured image, and determine the border of the effective area of ​​the curtain in each closed contour;

[0008] According to the border of the effective area of ​​the screen and the homography transformation relationship, the corrected coordinates of the border of the effective area of ​​the screen in the initial image are obtained;

[0009] Based on the calibration coordinates, the effective display area in the initial screen is adjusted.

[0010] In one embodiment, the step of obtaining each closed contour in the captured image and determining the frame of the effective area of ​​the curtain in each closed contour includes the steps of:

[0011] Obtaining each closed contour in the captured image;

[0012] Fitting the closed contour based on the preset fitting accuracy, and screening out the fitting area that meets the preset fitting accuracy;

[0013] Determine the border of the effective area of ​​the curtain from the fitting area.

[0014] In one embodiment, the step of fitting a closed contour based on a preset fitting accuracy and selecting a fitting area that meets the preset fitting accuracy includes the steps of:

[0015] Confirm the starting point coordinates and the end point coordinates on the closed contour, and obtain the first straight line passing through the starting point coordinates and the end point coordinates;

[0016] Get the segmentation point on the closed contour with the largest distance to the first straight line, and get the distance from the segmentation point to the first straight line;

[0017] If the distance from the segmentation point to the first straight line is greater than or equal to the preset fitting accuracy, the segmentation point is retained;

[0018] Based on the retained segmentation points and the start point coordinates and the end point coordinates on the closed contour, the closed contour is segmented to form a closed contour with a smaller contour length;

[0019] Determining the segmentation points retained in each closed contour with a smaller contour length;

[0020] The fitting area is obtained based on the area enclosed by the starting point coordinates and the retained segmentation points.

[0021] In one embodiment, the step of fitting the closed contour based on a preset fitting accuracy and screening out a fitting area that meets the preset fitting accuracy includes the steps of:

[0022] Determine the starting point coordinates and the end point coordinates on the closed contour, and obtain a first straight line passing through the starting point coordinates and the end point coordinates;

[0023] Obtaining a segmentation point on the closed contour having the largest distance to the first straight line, and obtaining a distance from the segmentation point to the first straight line;

[0024] If the distance between the segmentation point and the first straight line is greater than or equal to the preset fitting accuracy, retaining the segmentation point;

[0025] Based on the segmentation point, the start point coordinates, and the end point coordinates retained on the closed contour, the closed contour is segmented to form two smaller next-level closed contours, and determining whether the segmentation point is retained in each of the next-level closed contours;

[0026] Repeating the steps of dividing the closed contour to form two smaller next-level closed contours based on the retained segmentation point, the start point coordinates, and the end point coordinates of the previous closed contour, and determining whether the retained segmentation point exists in each of the next-level closed contours, until no smaller next-level closed contour can be formed;

[0027] A fitting area is obtained based on the area enclosed by the starting point coordinates and the retained segmentation points.

[0028] In one embodiment,

[0029] In the steps of determining the border of the effective area of ​​the curtain from the fitting area:

[0030] If the fitting area includes a first fitting area and a second fitting area surrounded by the first fitting area, the second fitting area is used as the border of the effective area of ​​the screen;

[0031] or

[0032] In the steps of determining the border of the effective area of ​​the curtain from the fitting area:

[0033] If there are multiple fitting regions, and the first fitting regions include at least two groups of first fitting regions and second fitting regions that are in an enclosing relationship, then obtain the area ratio of each group of second fitting regions to the first fitting region, where the first fitting regions enclose the second fitting regions;

[0034] A second fitting area corresponding to an area ratio greater than a preset ratio is selected as the frame of the effective area of ​​the screen; the first fitting area surrounds the second fitting area; and the preset ratio is 80% to 90%.

[0035] or

[0036] In the step of determining the border of the effective area of ​​the curtain from the fitting area:

[0037] If the fitting area includes two or more sets of the first fitting areas and the second fitting areas that are in an enclosing relationship, and the first fitting areas enclose the second fitting areas, then the second fitting area with the largest area among the sets of the fitting areas is used as the border of the effective area of ​​the curtain;

[0038] or

[0039] In the steps of determining the border of the effective area of ​​the curtain from the fitting area:

[0040] If there are more than one fitting areas and there are no two fitting areas that are in a surrounding relationship, the fitting area with the largest area will be used as the border of the effective area of ​​the screen.

[0041] In one embodiment,

[0042] The step of determining the border of the effective area of ​​the curtain from the fitting area includes the steps of:

[0043] forming a preset reference area in the central area of ​​the captured image;

[0044] Determine whether the fitting area overlaps with the preset reference area;

[0045] If the fitted area surrounds the preset reference area, the fitted area will be used as the border of the effective area of ​​the screen.

[0046] In one embodiment, the preset fitting accuracy is 15% to 25% of the initial picture height.

[0047] In one embodiment, before the step of fitting the closed contour based on the preset fitting accuracy and screening out the fitting area that meets the preset fitting accuracy, the method further includes the following steps:

[0048] Performing filtering, edge detection, and binarization on the captured image in sequence to obtain a binary image of the captured image;

[0049] Retrieve the binary image and filter out closed contours from the binary image.

[0050] In one embodiment, the initial frame includes a reference pattern;

[0051] The step of determining the homography transformation relationship according to the captured image and the initial image includes the following steps:

[0052] Obtaining a first coordinate set of the reference pattern in a first coordinate system and a second coordinate set in a second coordinate system; the first coordinate system is a coordinate system corresponding to the initial screen, and the second coordinate system is a coordinate system corresponding to the captured image;

[0053] According to the first coordinate set and the second coordinate set, a perspective transformation matrix between the first coordinate system and the second coordinate system is obtained, and the perspective transformation matrix is ​​used as a homography conversion relationship.

[0054] In one embodiment, before the step of acquiring a first coordinate set of the reference pattern in the first coordinate system and a second coordinate set in the second coordinate system, the method comprises the following steps:

[0055] Compare the resolution of the initial image with the resolution of the captured image;

[0056] If the resolution of the initial screen is different from the resolution of the captured image, the resolution of the captured image is adjusted to the resolution of the initial screen.

[0057] In one embodiment, in the step of acquiring a captured image: acquiring an image captured by a mobile device.

[0058] A projector includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the projection-onto-screen method described in any one of the above embodiments when executing the computer program.

[0059] One of the above technical solutions has the following advantages and beneficial effects:

[0060] The projection-on-screen method provided in each embodiment of the present application, during implementation, obtains a captured image containing an initial screen; determines a homography transformation relationship based on the captured image and the initial screen; obtains each closed contour in the captured image, and determines the border of the effective area of ​​the screen in each closed contour; obtains the corrected coordinates of the border of the effective area of ​​the screen in the initial screen based on the border of the effective area of ​​the screen and the homography transformation relationship; and adjusts the effective display area in the initial screen based on the corrected coordinates. The present application captures an image of the initial screen projected by the projector on the screen, identifies closed contours from the captured image, and confirms the border of the effective area of ​​the screen from the closed contours. Using the border of the effective area of ​​the screen and the homography transformation relationship, the corrected coordinates are obtained, and the effective display area of ​​the initial screen is corrected based on the corrected coordinates so that the projected screen of the projector can accurately enter the screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 Schematic diagram of the process of projecting onto a screen in an embodiment of the present application.

[0062] Figure 2 Schematic diagram of the process of obtaining the conversion relationship in the projection-on-screen method in an embodiment of the present application.

[0063] Figure 3 Schematic diagram of the process of adjusting the resolution in the projection-on-screen method in an embodiment of the present application.

[0064] Figure 4 This is a schematic diagram of an image captured before removing non-closed contours in an embodiment of the present application.

[0065] Figure 5 This is a schematic diagram of an image captured in an embodiment of the present application after removing non-closed contours.

[0066] Figure 6 Schematic diagram of the process of obtaining a frame in the projection-on-screen method in an embodiment of the present application.

[0067] Figure 7 Schematic diagram of the process of obtaining the fitting area in the projection-on-screen method in an embodiment of the present application.

[0068] Figure 8 In the embodiment of the present application, a preset reference area is formed in the central area of ​​the captured image.

[0069] Figure 9 Schematic diagram of the flow of the pre-fitting step in the projection onto screen method in an embodiment of the present application.

[0070] Figure 10This is a structural block diagram of the projection screen device in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0072] In order to make the projection screen more intelligent and accurate, the present application provides a projection screen method and a projector. In one embodiment, Figure 1 As shown, a method for projecting onto a screen is provided, comprising the following steps:

[0073] Step S110: Acquire a captured image. The captured image includes the screen and the image projected from the effective display area of ​​the initial image. The captured image is an image captured by a camera module. In one example, the projector has a built-in camera module. After the projector projects the initial image onto the screen, the projector's control device controls the camera module to capture the screen and acquire a captured image. This captured image may be captured during the execution of the projection-on-screen method of the present application, or it may be a pre-captured image stored in the projector's memory and retrieved from the memory when the projection-on-screen method of the present application is executed. In another example, the step of acquiring a captured image includes acquiring an image captured by a mobile device. In this example, after the projector projects the initial image onto the screen, the mobile device captures the captured image and transmits the captured image to the projector. In the first method, when the mobile device transmits the captured image to the projector, it instructs the projector to execute the projection-on-screen method of the present application. In the second method, the mobile device transmits the captured image to the projector, which then stores the captured image in its memory and retrieves it from the memory when the projection-on-screen method of the present application is executed. It should be noted that the mobile device can be connected to the projector via a wired or wireless connection. Wireless connection can be achieved by having the projector and mobile device in the same network environment or through an application. The mobile device can be a smartphone, tablet computer, or video camera. The screen can be a hard screen, a soft screen, a rectangular frame on the wall, or other common projection targets.

[0074] The captured image needs to capture the screen and the image projected based on the effective display area of ​​the initial screen. The initial screen is the screen to be projected in the projector, and the resolution of the screen is generally but not limited to 1920*1080. In one example, the image projected based on the effective display area of ​​the initial screen must at least completely cover the screen, so as to ensure that there is room to "crop" the projected image to achieve the effect of entering the screen. The effective display area of ​​the initial screen can occupy 100% of the entire initial screen, that is, full-screen display, but it can also be only 90%, 80%, 70% or other sizes of the initial screen. The effective display area is the part of the initial screen with picture information. When it is not full-screen display, other areas of the initial screen have no picture information.

[0075] Step S120 : determining a homography transformation relationship according to the captured image and the initial image.

[0076] The projector projects the initial image based on its built-in coordinate system. If the image is captured by the projector's camera module, it is captured based on a coordinate system from a different perspective; if the image is captured by a mobile device, the mobile device captures it based on a coordinate system from a different perspective. Therefore, the coordinate systems of the captured image and the initial image are different. In one example, a homography transformation relationship is determined based on the coordinate systems of the captured image and the initial image.

[0077] The following provides a feasible implementation method for obtaining a homography transformation relationship. In this example, the effective display area of ​​the initial screen contains a reference pattern. The reference pattern is a pattern drawn within the initial screen by the light system for obtaining the homography transformation. In one example, the reference pattern is a QR code, a checkerboard, or a solid-color rectangular area. In addition to the reference pattern, a homography transformation relationship between the initial screen and the captured image can also be established by comparing the vertex coordinates of the effective display area in the initial screen with the vertex coordinates of the screen projected based on the effective display area captured in the captured image.

[0078] In this example, Figure 2 As shown, the step of determining the homography transformation relationship according to the captured image and the initial picture includes the following steps:

[0079] Step S210, obtain a first coordinate set of the reference pattern in the first coordinate system, and a second coordinate set in the second coordinate system. The first coordinate system is the coordinate system corresponding to the initial screen, and the second coordinate system is the coordinate system corresponding to the captured image. It should be noted that the first coordinate set can be the vertex coordinates of the reference pattern, the contour coordinates of the reference pattern, or all the coordinates in the reference pattern. Correspondingly, when the first coordinate set is the vertex coordinates of the reference pattern, the second coordinate set is also the vertex coordinates of the reference pattern; when the first coordinate set is the contour coordinates of the reference pattern, the second coordinate set is also the contour coordinates of the reference pattern; when the first coordinate set is all the coordinates in the reference pattern, the second coordinate set is also all the coordinates in the reference pattern.

[0080] In step S220, a perspective transformation matrix between the first coordinate system and the second coordinate system is obtained based on the first coordinate set and the second coordinate set, and the perspective transformation matrix is ​​used as a homography. It should be noted that the purpose of the perspective transformation matrix is ​​to establish a universal coordinate transformation relationship between the two images, so that the coordinates in one image can be mapped to the coordinates in the other image.

[0081] In order to improve the accuracy of data processing, in one example, Figure 3 As shown, before the step of obtaining a first coordinate set of the reference pattern in the first coordinate system and a second coordinate set in the second coordinate system, the steps include:

[0082] Step S310 compares the resolution of the initial screen with the resolution of the captured image. It should be noted that if the resolutions of the two screens are different, it is difficult to establish a simple coordinate transformation relationship between the two screens. To reduce interference caused by resolution differences, the resolution of the captured image needs to be adjusted to be consistent with the resolution of the initial screen.

[0083] In step S320 , if the resolution of the initial screen is different from the resolution of the captured image, the resolution of the captured image is adjusted to the resolution of the initial screen.

[0084] Step S130 , obtaining each closed contour in the captured image, and determining the frame of the effective area of ​​the curtain in each closed contour.

[0085] In the process of capturing an image, at least the screen must be captured, and other interfering objects may also be captured, such as brooms, tables, chairs, cardboard boxes erected around the screen, shadows cast by the screen, the screen's support structure, and other interfering objects. Figure 4 As shown in FIG, the captured image includes not only the effective area C1 of the curtain, but also the shadow C2 produced by the curtain, and other interference objects G1, G2, G3, G4 and G5. During the recognition process, non-closed contours are excluded, for example, Figure 3G3, G4 and G5 in the above example keep closed contours. Figure 5 As shown in the figure, the closed contours are C1, C2, G1 and G2, from which C1 is confirmed to be the border of the effective area of ​​the screen. It should be noted that the effective area of ​​the screen refers to the area on the screen that can be used to display the projection image projected by the projector. Some screens have white diffuse reflection areas (such as Figure 8 The effective projection area C1 of the screen and the outer black border area (such as Figure 8 The black frame outside the screen C2) is shown in the figure. The white diffuse reflection area is the effective area of ​​the screen, and the black frame area is the invalid projection area.

[0086] The following is a possible way, Figure 6 As shown, the steps of obtaining each closed contour in the captured image and determining the frame of the effective area of ​​the curtain in each closed contour include the following steps:

[0087] Step S610: Acquire each closed contour in the captured image. During the image capture process, at least the curtain must be captured, and other interfering objects may also be captured, such as brooms, tables, chairs, paper boxes erected around the curtain, shadows cast by the curtain, curtain support structures, and other interfering objects. Figure 3 As shown in FIG, the captured image includes not only the effective area C1 of the curtain, but also the shadow C2 produced by the curtain, and other interference objects G1, G2, G3, G4 and G5. During the recognition process, non-closed contours are excluded, for example, Figure 3 G3, G4 and G5 in the above example keep closed contours. Figure 4 As shown, the closed contours are C1, C2, G1 and G2, from which C1 is confirmed to be the border of the effective area of ​​the curtain.

[0088] Step S620, fitting the closed contour based on the preset fitting accuracy, and screening out the fitting area that meets the preset fitting accuracy. It should be noted that fitting is to fit the closed contour into a quadrilateral, and compare the fitted quadrilateral with the size of the screen. If the size of the screen is met, it means that the closed contour corresponding to the fitted quadrilateral is the border of the effective area of ​​the solid wood board. In one example, the preset fitting accuracy is 15% to 25% of the initial picture height, and the height of the picture can be represented by the pixel value of the picture in the height direction. For example, the preset fitting accuracy is 17%, 19%, 21% or 24% of the initial picture height.

[0089] In one example, a feasible fitting method is provided, such as Figure 7 As shown, the step of fitting a closed contour based on a preset fitting accuracy and screening out a fitting area that meets the preset fitting accuracy includes the following steps:

[0090] Step S710 , confirming the starting point coordinates and the end point coordinates on the closed contour, and obtaining a first straight line passing through the starting point coordinates and the end point coordinates.

[0091] For the first-level closed contour determined in step S710 , the starting point coordinates and the ending point coordinates are the same point, and the first straight line in this step is a tangent line passing through the point of the closed contour.

[0092] Step S720 , obtaining a segmentation point on the closed contour with the largest distance to the first straight line, and obtaining the distance from the segmentation point to the first straight line.

[0093] Step S730: If the distance from the segmentation point to the first straight line is greater than or equal to the preset fitting accuracy, the segmentation point is retained.

[0094] Step S740 : Based on the retained segmentation points and the start point coordinates and the end point coordinates on the closed contour, the closed contour is segmented to form closed contours with a smaller contour length.

[0095] Step S750: Determine the segmentation points retained in each closed contour with a smaller contour length.

[0096] Step S760: Obtain a fitting area based on the starting point coordinates and the area enclosed by the retained segmentation points.

[0097] It should be noted that the above steps are executed in a loop until no segmentation point is found whose distance from the starting point to the straight line from the end point to the end point is greater than the preset fitting accuracy. If the starting coordinates, end point coordinates, and retained segmentation points on the closed contour can be connected to form a closed quadrilateral, it means that the closed contour meets the requirements, and the closed contour is retained, and the corresponding fitting area is retained; if it cannot be connected to form a quadrilateral, it means that the closed contour does not meet the requirements, and the closed contour is discarded.

[0098] As described above, based on the recursive loop of step S740 and step S750, in some embodiments, these two steps include:

[0099] Step S760: based on the segmentation point, start point coordinates, and end point coordinates retained on the closed contour, the closed contour is segmented to form two smaller next-level closed contours, and it is determined whether there is a retained segmentation point in each next-level closed contour;

[0100] Step S770, repeat the steps of dividing the closed contour to form two smaller next-level closed contours based on the retained segmentation points, starting point coordinates and end point coordinates on the previous level closed contour, and judging whether there are retained segmentation points in each next-level closed contour (i.e. repeating step S760) until no smaller next-level closed contour can be formed.

[0101] The closed contour in step S710 is a first-level closed contour, that is, a closed contour that has not yet been segmented. The closed contour formed by the initial segmentation in step S740 is a second-level closed contour. The contours obtained based on the cycles of steps S740 and S750 correspond to third-level closed contours, fourth-level closed contours, and so on. Starting from the second-level closed contour, the number of closed contours at each level is twice the number of closed contours at the previous level. The starting point coordinates and end point coordinates of the previous level closed contour correspond to the starting point coordinates of the two next-level closed contours, and the segmentation points retained by the previous level closed contour correspond to the end point coordinates of the two next-level closed contours. Starting from the second-level closed contour, each level closed contour is formed by the line connecting its starting point coordinates and end point coordinates, and the contour line of the previous level closed contour between the starting point coordinates and end point coordinates.

[0102] Step S760 is to execute steps S710 to S730 again based on the next closed contour segmented from the previous closed contour to determine whether there are any retainable segmentation points in the next closed contour.

[0103] Step S630: Determine the border of the effective area of ​​the screen from the fitting area.

[0104] According to the different numbers of fitting areas, it is divided into the following cases:

[0105] In the first case, in the step of determining the border of the effective area of ​​the curtain from the fitting area:

[0106] If the fitting area includes a first fitting area and a second fitting area enclosed by the first fitting area, the second fitting area is used as the border of the screen's effective area. It should be noted that if a screen is hung on a wall or standing against a backdrop, a shadow will appear around the screen in the captured image. During the fitting process, a shadow will appear around the screen. After fitting is complete, a nested first and second fitting areas will appear, with the first fitting area enclosing the second. In this case, the second fitting area serves as the border of the screen's effective area.

[0107] In the second case, in the step of determining the border of the effective area of ​​the curtain from the fitting area:

[0108] If there are multiple fitting areas, and they include at least two groups of first fitting areas and second fitting areas that are in an enclosing relationship, then the area ratio value of each group of second fitting areas to the first fitting areas is obtained, where the first fitting area of ​​each group encloses the second fitting area.

[0109] A set of first and second fitting areas corresponding to area ratios greater than a preset ratio is selected, and the second fitting areas of this set are used as the border of the screen's effective area; the first fitting areas surround the second fitting areas. It should be noted that if an interfering object is present, after fitting, nested first and second fitting areas will also appear, interfering with the border of the selected effective area of ​​the screen. In this case, a selection is made based on the direct area ratio relationship between the screen and its shadow, a preset ratio value is set, the ratio values ​​of each set of first and second fitting areas are calculated, and the set of first and second fitting areas greater than the preset ratio value is selected. The preset ratio value is 80% to 90%, for example, the preset ratio value is 82%, 84%, 86%, or 88%.

[0110] In the third case, in the step of determining the border of the effective area of ​​the curtain from the fitting area:

[0111] If there are more than one fitting area and there are no two fitting areas that are in an enclosing relationship, the largest fitting area among the multiple fitting areas will be used as the border of the screen's effective area. It should be noted that the screen is directly hung without a wall behind it. In this case, the screen will not cast a shadow in the captured image, and no nested first and second fitting areas will be fitted. Since the shooting process is mainly focused on the screen, interfering objects may appear in the captured image, but it is impossible for interfering objects to be larger than the screen. Therefore, the largest fitting area among the multiple fitting areas is directly selected as the border of the screen's effective area.

[0112] In the fourth case, in the step of determining the border of the effective area of ​​the screen from the fitting area:

[0113] If the fitting area includes two or more first fitting areas and second fitting areas in an enclosing relationship, wherein the first fitting areas enclose the second fitting areas, the second fitting area with the largest area in each group of fitting areas is used as the border of the effective area of ​​the screen.

[0114] In addition to selecting the border of the effective area of ​​the curtain according to the number of different situations, in another example,

[0115] The step of determining the border of the effective area of ​​the curtain from the fitting area includes the steps of:

[0116] A preset reference area is formed in the central area of ​​the captured image (eg Figure 8(as shown); determine whether the fitting area overlaps with a preset reference area; if the fitting area surrounds the preset reference area, the fitting area is used as the border of the screen's effective area. The aforementioned embodiment of area A surrounding area B in this application includes areas B located within the area enclosed by area A, as well as within the area covered by area A. It should be noted that to photograph a screen, the screen is aligned with the camera, for example, focusing on the center of the screen. In this case, a preset reference area is set by the program within the central area of ​​the photo. This preset reference area must fall within the effective area of ​​the screen. The reference area is an area formed by the processing system in the coordinate system of the captured image. The reference area can be an area whose center point coincides with the center of the captured image and can be rectangular, circular, or other shapes. The height of the reference area should be less than 40% of the height of the captured image, and the width should be less than 40% of the width of the captured image, in pixels. In this case, the fitting area that overlaps with the preset reference area is selected as the border of the screen's effective area. In one example, if a first fitting area and a second fitting area are nested, and the first fitting area and the second fitting area overlap with the preset reference area, the smaller fitting area between the first and second fitting areas is selected as the border of the screen's effective area. In one example, if there are more than two sets of nested first fitting areas and second fitting areas, and the first fitting areas and the second fitting areas overlap with the preset reference area, then the set with a ratio value of the first fitting areas to the second fitting areas greater than the preset ratio value is selected, and the fitting area with a smaller area among the first fitting areas and the second fitting areas in this set is selected as the border of the effective area of ​​the screen.

[0117] Furthermore, in some embodiments, to improve the accuracy of detecting the effective area of ​​the curtain, the step of forming a preset reference area from the central area of ​​the captured image includes:

[0118] Acquire a shooting distance of a shooting device, wherein the shooting image is obtained by shooting the shooting device;

[0119] A preset reference area is formed in the central area of ​​the captured image, and the size of the preset reference area is inversely proportional to the shooting distance.

[0120] The capture device can be a mobile device or a projector with a camera. The capture distance is the distance between the capture device and the wall on which the screen is located. This can be measured by a rangefinder module on the capture device, such as a TOF (Time of Flight) module or a structured light module. In this embodiment, the longer the detected capture distance, the smaller the reference area in the captured image; the closer the detected capture distance, the larger the reference area in the captured image. As the capture distance changes, the size of the screen in the captured image also changes accordingly. Therefore, the size of the preset reference area needs to be adaptively adjusted based on the change in capture distance to avoid misjudgments caused by the preset reference area extending beyond the screen.

[0121] To improve the fitting accuracy, in an example, Figure 9 As shown, before the step of fitting the closed contour based on the preset fitting accuracy and screening out the fitting area that meets the preset fitting accuracy, the following steps are also included:

[0122] Step S910 , performing filtering processing, edge detection processing, and binarization processing on the captured image in sequence to obtain a binary image of the captured image.

[0123] Step S920: searching the binary image and filtering out closed contours from the binary image.

[0124] Step S140: Based on the bounding box of the screen's effective area and the homography, the calibrated coordinates of the bounding box of the screen's effective area in the initial image are obtained. To ensure that the effective display area of ​​the initial image is fully projected onto the screen's effective area, in one example, the calibrated coordinates may be the vertex coordinates of the bounding box of the screen's effective area.

[0125] Step S150 : adjusting the effective display area in the initial image based on the corrected coordinates.

[0126] After obtaining the calibration coordinates, the effective display area of ​​the initial screen is adjusted to the area surrounded by the calibration coordinates.

[0127] In one example, the correction coordinates are vertex coordinates of a border of the effective area of ​​the screen.

[0128] The step of adjusting the effective display area in the initial screen based on the correction coordinates includes the following steps:

[0129] During the projection process, the initial image gradually becomes larger or smaller until the vertices of the initial image coincide with or nearly coincide with the correction coordinates.

[0130] The projection-on-screen method provided in each embodiment of the present application, during implementation, obtains a captured image of a screen projected from an effective display area based on an initial screen; determines a homography transformation relationship based on the captured image and the initial screen; obtains each closed contour in the captured image, and determines the border of the effective screen area in each closed contour; obtains the corrected coordinates of the border of the effective screen area in the initial screen based on the border of the effective screen area and the homography transformation relationship; and adjusts the effective display area in the initial screen based on the corrected coordinates. The present application captures an image of the initial screen projected by a projector on the screen, identifies closed contours from the captured image, and confirms the border of the effective screen area from the closed contours. Using the border of the effective screen area and the homography transformation relationship, corrected coordinates are obtained, and the effective display area of ​​the initial screen is corrected based on the corrected coordinates, so that the projected screen of the projector can accurately enter the screen.

[0131] It should be understood that although Figure 1-3 The steps in the flowcharts of , 6, 7, and 9 are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-3 At least part of the steps in 6, 7, and 9 may include multiple sub-steps or multiple stages. These sub-steps or stages do not necessarily have to be performed at the same time, but can be performed at different times. The order of execution of these sub-steps or stages is not necessarily one by one, but can be performed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0132] In one embodiment, Figure 10 As shown, a projection screen device is provided, comprising:

[0133] The image acquisition module 101 is used to acquire a captured image, where the captured image includes a screen and an image projected from an effective display area based on an initial image.

[0134] The transformation relationship acquisition module 102 is used to determine the homography transformation relationship according to the captured image and the initial picture.

[0135] The frame acquisition module 103 is used to acquire each closed contour in the captured image and determine the frame of the effective area of ​​the curtain in each closed contour.

[0136] The correction coordinate acquisition module 104 is used to obtain the correction coordinates of the frame of the effective area of ​​the screen in the initial image according to the frame of the effective area of ​​the screen and the homography transformation relationship.

[0137] The adjustment module 105 is configured to adjust the effective display area in the initial screen based on the correction coordinates.

[0138] The specific definitions of the projection-on-screen device can be found in the definitions of the projection-on-screen method above and will not be repeated here. Each module in the aforementioned projection-on-screen device can be implemented in whole or in part via software, hardware, or a combination thereof. Each of these modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a computer device memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0139] In one embodiment, a projector is provided, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps described in any of the above embodiments are implemented, for example, the following steps:

[0140] Acquire a captured image, where the captured image includes the screen and an image projected from an effective display area based on the initial image;

[0141] Determine the homography transformation relationship based on the captured image and the initial image;

[0142] Obtain each closed contour in the captured image, and determine the border of the effective area of ​​the curtain in each closed contour;

[0143] According to the border of the effective area of ​​the screen and the homography transformation relationship, the corrected coordinates of the border of the effective area of ​​the screen in the initial image are obtained;

[0144] Based on the calibration coordinates, the effective display area in the initial screen is adjusted.

[0145] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0146] Obtaining each closed contour in the captured image;

[0147] Fitting the closed contour based on the preset fitting accuracy, and screening out the fitting area that meets the preset fitting accuracy;

[0148] Determine the border of the effective area of ​​the curtain from the fitting area.

[0149] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0150] Confirm the starting point coordinates and the end point coordinates on the closed contour, and obtain the first straight line passing through the starting point coordinates and the end point coordinates;

[0151] Get the segmentation point on the closed contour with the largest distance to the first straight line, and get the distance from the segmentation point to the first straight line;

[0152] If the distance from the segmentation point to the first straight line is greater than or equal to the preset fitting accuracy, the segmentation point is retained;

[0153] Based on the retained segmentation points and the start point coordinates and the end point coordinates on the closed contour, the closed contour is segmented to form a closed contour with a smaller contour length;

[0154] Determining the segmentation points retained in each closed contour with a smaller contour length;

[0155] The fitting area is obtained based on the area enclosed by the starting point coordinates and the retained segmentation points.

[0156] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0157] In the steps of determining the border of the effective area of ​​the curtain from the fitting area:

[0158] If there are multiple fitting areas, including a first fitting area and a second fitting area surrounded by the first fitting area, the second fitting area is used as the border of the effective area of ​​the screen;

[0159] or

[0160] In the steps of determining the border of the effective area of ​​the curtain from the fitting area:

[0161] If there are multiple fitting regions, and the fitting regions include at least two groups of first fitting regions and second fitting regions that are in an enclosing relationship, then the area ratio of each group of second fitting regions to the first fitting regions is obtained;

[0162] Select a set of first fitting areas and second fitting areas corresponding to area ratios greater than a preset ratio, and use the second fitting area as the border of the effective area of ​​the screen; the first fitting area surrounds the second fitting area; the preset ratio is 80% to 90%

[0163] or

[0164] In the steps of determining the border of the effective area of ​​the curtain from the fitting area:

[0165] If there are multiple fitting areas and there are no two fitting areas in a surrounding relationship, the fitting area with the largest area will be used as the border of the effective area of ​​the screen.

[0166] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0167] forming a preset reference area in the central area of ​​the captured image;

[0168] Determine whether the fitting area overlaps with the preset reference area;

[0169] If the fitted area surrounds the preset reference area, the fitted area will be used as the border of the effective area of ​​the screen.

[0170] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0171] Performing filtering, edge detection, and binarization on the captured image in sequence to obtain a binary image of the captured image;

[0172] Retrieve the binary image and filter out closed contours from the binary image.

[0173] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0174] Obtaining a first coordinate set of the reference pattern in a first coordinate system and a second coordinate set in a second coordinate system; the first coordinate system is a coordinate system corresponding to the initial screen, and the second coordinate system is a coordinate system corresponding to the captured image;

[0175] According to the first coordinate set and the second coordinate set, a perspective transformation matrix between the first coordinate system and the second coordinate system is obtained, and the perspective transformation matrix is ​​used as a homography conversion relationship.

[0176] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0177] Compare the resolution of the initial image with the resolution of the captured image;

[0178] If the resolution of the initial screen is different from the resolution of the captured image, the resolution of the captured image is adjusted to the resolution of the initial screen.

[0179] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0180] Get the captured image captured by the mobile device.

[0181] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0182] Acquire a captured image, where the captured image includes the screen and an image projected from an effective display area based on the initial image;

[0183] Determine the homography transformation relationship based on the captured image and the initial image;

[0184] Obtain each closed contour in the captured image, and determine the border of the effective area of ​​the curtain in each closed contour;

[0185] According to the border of the effective area of ​​the screen and the homography transformation relationship, the corrected coordinates of the border of the effective area of ​​the screen in the initial image are obtained;

[0186] Based on the calibration coordinates, the effective display area in the initial screen is adjusted.

[0187] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0188] Obtaining each closed contour in the captured image;

[0189] Fitting the closed contour based on the preset fitting accuracy, and screening out the fitting area that meets the preset fitting accuracy;

[0190] Determine the border of the effective area of ​​the curtain from the fitting area.

[0191] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0192] Confirm the starting point coordinates and the end point coordinates on the closed contour, and obtain the first straight line passing through the starting point coordinates and the end point coordinates;

[0193] Get the segmentation point on the closed contour with the largest distance to the first straight line, and get the distance from the segmentation point to the first straight line;

[0194] If the distance from the segmentation point to the first straight line is greater than or equal to the preset fitting accuracy, the segmentation point is retained;

[0195] Based on the retained segmentation points and the start point coordinates and the end point coordinates on the closed contour, the closed contour is segmented to form a closed contour with a smaller contour length;

[0196] Determining the segmentation points retained in each closed contour with a smaller contour length;

[0197] The fitting area is obtained based on the area enclosed by the starting point coordinates and the retained segmentation points.

[0198] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0199] In the steps of determining the border of the effective area of ​​the curtain from the fitting area:

[0200] If there are multiple fitting areas, including a first fitting area and a second fitting area surrounded by the first fitting area, the second fitting area is used as the border of the effective area of ​​the screen;

[0201] or

[0202] In the steps of determining the border of the effective area of ​​the curtain from the fitting area:

[0203] If there are multiple fitting regions, and the fitting regions include at least two groups of first fitting regions and second fitting regions that are in an enclosing relationship, then the area ratio of each group of second fitting regions to the first fitting regions is obtained;

[0204] Select a set of first fitting areas and second fitting areas corresponding to area ratios greater than a preset ratio, and use the second fitting area as the border of the effective area of ​​the screen; the first fitting area surrounds the second fitting area; the preset ratio is 80% to 90%

[0205] or

[0206] In the steps of determining the border of the effective area of ​​the curtain from the fitting area:

[0207] If there are multiple fitting areas and there are no two fitting areas in a surrounding relationship, the fitting area with the largest area will be used as the border of the effective area of ​​the screen.

[0208] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0209] forming a preset reference area in the central area of ​​the captured image;

[0210] Determine whether the fitting area overlaps with the preset reference area;

[0211] If the fitted area surrounds the preset reference area, the fitted area will be used as the border of the effective area of ​​the screen.

[0212] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0213] Performing filtering, edge detection, and binarization on the captured image in sequence to obtain a binary image of the captured image;

[0214] Retrieve the binary image and filter out closed contours from the binary image.

[0215] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0216] Obtaining a first coordinate set of the reference pattern in a first coordinate system and a second coordinate set in a second coordinate system; the first coordinate system is a coordinate system corresponding to the initial screen, and the second coordinate system is a coordinate system corresponding to the captured image;

[0217] According to the first coordinate set and the second coordinate set, a perspective transformation matrix between the first coordinate system and the second coordinate system is obtained, and the perspective transformation matrix is ​​used as a homography conversion relationship.

[0218] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0219] Compare the resolution of the initial image with the resolution of the captured image;

[0220] If the resolution of the initial screen is different from the resolution of the captured image, the resolution of the captured image is adjusted to the resolution of the initial screen.

[0221] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0222] Get the captured image captured by the mobile device.

[0223] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0224] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0225] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for projecting onto a screen, characterized in that: The following steps are involved: Acquire a captured image, wherein the captured image includes a screen and an image projected based on an effective display area of ​​an initial image; Determining a homography transformation relationship according to the captured image and the initial picture; Acquire each closed contour in the captured image, and determine the frame of the effective area of ​​the curtain in each closed contour; Obtaining, according to the frame of the effective area of ​​the screen and the homography transformation relationship, the corrected coordinates of the frame of the effective area of ​​the screen in the initial image; Adjusting the effective display area in the initial screen based on the corrected coordinates; The step of obtaining each closed contour in the captured image and determining the frame of the effective area of ​​the curtain in each closed contour includes the following steps: Acquire each closed contour in the captured image; Fitting the closed contour based on a preset fitting accuracy, and screening out a fitting area that meets the preset fitting accuracy; Determine the border of the effective area of ​​the curtain from the fitting area; The step of determining the border of the effective area of ​​the screen from the fitting area includes the steps of: forming a preset reference area in the central area of ​​the captured image; Determining whether the fitting area overlaps with the preset reference area; If the fitting area surrounds the preset reference area, the fitting area is used as the frame of the effective area of ​​the screen.

2. The projection onto screen method according to claim 1, characterized in that: The step of fitting the closed contour based on a preset fitting accuracy and screening out a fitting area that meets the preset fitting accuracy includes the steps of: Determine the starting point coordinates and the end point coordinates on the closed contour, and obtain a first straight line passing through the starting point coordinates and the end point coordinates; Obtaining a segmentation point on the closed contour having the largest distance to the first straight line, and obtaining a distance from the segmentation point to the first straight line; If the distance between the segmentation point and the first straight line is greater than or equal to the preset fitting accuracy, retaining the segmentation point; Based on the retained segmentation points and the start point coordinates and the end point coordinates on the closed contour, the closed contour is segmented to form closed contours with a smaller contour length; Determining the first bending point retained in each closed contour having a smaller contour length; A fitting area is obtained based on the area enclosed by the starting point coordinates and the retained segmentation points.

3. The projection onto screen method according to claim 1, characterized in that: In the step of determining the border of the effective area of ​​the curtain from the fitting area: If there are multiple fitting areas, including a first fitting area and a second fitting area surrounded by the first fitting area, the second fitting area is used as the border of the effective area of ​​the curtain; or In the step of determining the border of the effective area of ​​the curtain from the fitting area: If there are multiple fitting regions, and the fitting regions include at least two groups of the first fitting regions and the second fitting regions that are in an enclosing relationship, then obtaining an area ratio value of each group of the second fitting regions to the first fitting regions; A group of the first fitting area and the second fitting area corresponding to the area ratio value greater than the preset ratio value is selected, and the second fitting area is used as the frame of the effective area of ​​the screen; the first fitting area surrounds the second fitting area; The preset ratio is 80% to 90%. or In the step of determining the border of the effective area of ​​the curtain from the fitting area: If there are multiple fitting areas and there are no two fitting areas in a surrounding relationship, the one with the largest area among the fitting areas is used as the border of the effective area of ​​the screen.

4. The projection onto screen method according to claim 1, characterized in that: The preset fitting accuracy is 15% to 25% of the initial picture height.

5. The projection onto screen method according to claim 1, characterized in that: Before the step of fitting the closed contour based on a preset fitting accuracy and screening out a fitting area that meets the preset fitting accuracy, the method further includes the following steps: performing filtering processing, edge detection processing, and binarization processing on the captured image in sequence to obtain a binary image of the captured image; The binary image is retrieved, and the closed contour is screened out from the binary image.

6. The projection onto screen method according to any one of claims 1 to 5, characterized in that: The initial image includes a reference pattern; The step of determining the homography conversion relationship based on the captured image and the initial screen includes the following steps: Acquire a first coordinate set of the reference pattern in a first coordinate system and a second coordinate set in a second coordinate system; the first coordinate system is a coordinate system corresponding to the initial screen, and the second coordinate system is a coordinate system corresponding to the captured image; According to the first coordinate set and the second coordinate set, a perspective transformation matrix between the first coordinate system and the second coordinate system is obtained, and the perspective transformation matrix is ​​used as the homography conversion relationship.

7. The projection onto screen method according to claim 6, characterized in that: Before the step of acquiring the first coordinate set of the reference pattern in the first coordinate system and the second coordinate set in the second coordinate system, the method includes the following steps: comparing the resolution of the initial image with the resolution of the captured image; If the resolution of the initial screen is different from the resolution of the captured image, the resolution of the captured image is adjusted to the resolution of the initial screen.

8. The projection onto screen method according to claim 1, characterized in that: In the step of obtaining the captured image: the captured image captured by the mobile device is obtained.

Images