Projection screen correction method and apparatus

Abstract

The application discloses a projection picture correction method and device. The method comprises the following steps: collecting a projection picture of a first projection calibration picture projected by a projector, obtaining a second projection calibration picture, the first projection calibration picture having a corresponding first feature point set comprising position information of each feature point; determining position information of each feature point in the second projection calibration picture, obtaining a second feature point set; performing linear fitting on each feature point in the second feature point set, determining position information of intersection points of each straight line, obtaining a first intersection point set; mapping the second feature point set and the first intersection point set to the first projection calibration picture respectively, obtaining a third feature point set and a second intersection point set; determining position correction information based on the third feature point set and the second intersection point set, and correcting the projection picture based on the position correction information. The application solves the technical problem that the picture will be distorted when the projector projects on uneven media, and the user's viewing experience is affected.

Description

Technical Field

[0001] This application relates to the field of projection correction technology, and more specifically, to a projection image correction method and apparatus. Background Technology

[0002] With the development of projection technology, smart projectors are increasingly widely used in people's daily lives. To meet users' demands for a high-quality viewing experience, the image quality of smart projectors, including resolution, brightness, and color, has been rapidly improved. However, due to cost considerations, people generally do not use a dedicated projection screen as the projection surface, but instead project directly onto ordinary walls. However, ordinary walls often have uneven surfaces, which can lead to visible distortion in the projected image. This distortion is particularly noticeable when projecting high-definition or large images, severely impacting the user's viewing experience.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a projection image correction method and apparatus to at least solve the technical problem that image distortion occurs when a projector projects onto an uneven medium, affecting the user's viewing experience.

[0005] According to one aspect of the embodiments of this application, a projection image correction method is provided, comprising: acquiring a projection image of a first projection calibration map projected by a projector to obtain a second projection calibration map, wherein the first projection calibration map has a corresponding first feature point set, the first feature point set including position information of each feature point in the first projection calibration map; determining the position information of each feature point in the second projection calibration map to obtain a second feature point set; performing linear fitting on each feature point in the second feature point set to determine the position information of the intersection points of each line to obtain a first intersection point set; mapping the second feature point set and the first intersection point set onto the first projection calibration map respectively to obtain a third feature point set and a second intersection point set; determining position correction information based on the third feature point set and the second intersection point set, and correcting the projection image based on the position correction information.

[0006] Optionally, based on the least squares method, each feature point in the second feature point set is fitted with a straight line along the row direction to obtain a first number of row lines, and each feature point in the second feature point set is fitted with a straight line along the column direction to obtain a second number of column lines; the position information of the intersection points of each row line and each column line is determined to obtain a first set of intersection points, wherein each feature point in the first set of intersection points corresponds one-to-one with each feature point in the second set of feature points.

[0007] Optionally, a target mapping relationship is determined based on the first feature point set and the second feature point set, the target mapping relationship being used to map feature points in the second projection calibration map to the first projection calibration map; based on the target mapping relationship, each feature point in the second feature point set is mapped to the first projection calibration map to obtain the third feature point set, and each feature point in the first intersection point set is mapped to the first projection calibration map to obtain the second intersection point set, wherein each feature point in the third feature point set corresponds one-to-one with each intersection point in the second intersection point set.

[0008] Optionally, any set of corresponding feature points and intersections from the third feature point set and the second intersection point set are taken as a target point set. The positional difference information of the feature points and intersections in the target point set is determined. Based on the positional difference information, the positional correction information of the feature points in the first feature point set corresponding to the target point set is determined. Based on the positional correction information, the position of each feature point in the first feature point set is corrected to achieve the correction of the projected image.

[0009] Optionally, the first position information of each internal feature point in the second projection calibration map is determined to obtain the second feature point set; or, the first position information of each internal feature point in the second projection calibration map is determined, and the second position information of each boundary feature point in the second projection calibration map is determined to obtain the second feature point set.

[0010] Optionally, the second projection calibration image is binarized to obtain a binarized image; the binarized image is dilated to extract each contour from the dilated binarized image; the convex hull of each contour is calculated, and it is detected whether each convex hull satisfies the quadrilateral constraint condition to determine all quadrilaterals in the binarized image; for any two adjacent quadrilaterals at diagonal positions, the midpoint of the diagonal line between the two corner points at diagonal positions of the two adjacent quadrilaterals is determined as an internal feature point; the first position information of each internal feature point is determined.

[0011] Optionally, based on the corner points and internal feature points of each quadrilateral, the initial position information of each boundary feature point is initially determined; for any boundary feature point, a window of a preset size is selected around the initial position of the boundary feature point, each pixel in the window is traversed, and the second position information of the boundary feature point is determined based on the gradient of each pixel.

[0012] According to another aspect of the embodiments of this application, a projection image correction device is also provided, comprising: an acquisition module, configured to acquire the projection image of a first projection calibration map projected by a projector to obtain a second projection calibration map, wherein the first projection calibration map has a corresponding first feature point set, the first feature point set including the position information of each feature point in the first projection calibration map; a determination module, configured to determine the position information of each feature point in the second projection calibration map to obtain a second feature point set; a fitting module, configured to perform linear fitting on each feature point in the second feature point set to determine the position information of the intersection points of each line to obtain a first intersection point set; a mapping module, configured to map the second feature point set and the first intersection point set to the first projection calibration map respectively to obtain a third feature point set and a second intersection point set; and a correction module, configured to determine position correction information based on the third feature point set and the second intersection point set, and correct the projection image based on the position correction information.

[0013] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein, when the program is running, it controls the device where the non-volatile storage medium is located to perform the above-described projection image correction method.

[0014] According to another aspect of the embodiments of this application, an electronic device is also provided, the electronic device comprising: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described projection image correction method through the computer program.

[0015] In this embodiment, the projected image of a first projection calibration map projected by a projector is first acquired to obtain a second projection calibration map. The first projection calibration map has a corresponding first feature point set, which includes the position information of each feature point in the first projection calibration map. Simultaneously, the position information of each feature point in the second projection calibration map is determined to obtain a second feature point set. Then, straight line fitting is performed on each feature point in the second feature point set to determine the position information of the intersection points of each straight line, thus obtaining a first intersection point set. Next, the second feature point set and the first intersection point set are mapped onto the first projection calibration map respectively to obtain a third feature point set and a second intersection point set. Finally, position correction information is determined based on the third feature point set and the second intersection point set, and the projected image is corrected based on the position correction information. Specifically, by performing linear fitting on the feature points of the distorted second projection calibration map, the intersection point is taken as the standard position of the feature point. Then, the feature point and the intersection point are mapped to the same coordinate system, accurate position correction information can be calculated, thereby correcting the distorted projection image. This solves the technical problem of image distortion when the projector projects onto uneven media, affecting the user's viewing experience. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 This is a schematic flowchart of a projection image correction method according to an embodiment of this application;

[0018] Figure 2 This is a schematic diagram of a first projection calibration diagram according to an embodiment of this application;

[0019] Figure 3 This is a schematic diagram of feature points in a second projection calibration map according to an embodiment of this application;

[0020] Figure 4 This is a schematic diagram illustrating the determination of feature points in a second projection calibration map according to an embodiment of this application;

[0021] Figure 5 This is a schematic diagram of a projection screen correction device according to an embodiment of this application. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] Example 1

[0025] According to an embodiment of this application, an embodiment of a projection screen correction method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0026] Figure 1 This is a projection image correction method according to an embodiment of this application, such as... Figure 1 As shown, the method includes at least steps S102-S110, wherein:

[0027] Step S102: Acquire the projected image of the first projection calibration map projected by the projector to obtain the second projection calibration map. The first projection calibration map has a corresponding first feature point set, which includes the position information of each feature point in the first projection calibration map.

[0028] Among them, the first projection calibration diagram is a type of projection shown by the projector, such as... Figure 2 The chessboard grid diagram shown has corner points as feature points, each with fixed positional information. The feature points are sorted, and the coordinates of the i-th feature point are assumed to be (x...). i y i If the first feature point set corresponding to the first projection calibration map is f(x), then the first feature point set corresponding to the first projection calibration map is f(x).i y i ).

[0029] After the projector projects the first projection calibration map onto the wall, it can acquire an image of the corresponding projected image through an image sensor. This image is then used as the second projection calibration map. The image sensor can be an infrared camera or a full-color camera; it can be a mobile phone camera, a camera built into the projector, or an external camera. By performing linear fitting on the feature points of the distorted second projection calibration map, the intersection points are used as the standard positions of the feature points. The position information of the feature points and the intersection points is then compared to obtain the flatness information of the projection wall and the distortion information of the projected image. Finally, the projected image is adjusted based on the positional differences.

[0030] It should be noted that, in order to better collect the flatness information of the projection wall, when designing the first projection calibration map, the projection screen boundary must have feature point information, the feature area must have brightness information, and the feature point information should be as dense as possible, usually at least 9*9. In this embodiment, the number of feature points is set to M*N (M≥9, N≥9).

[0031] Step S104: Determine the position information of each feature point in the second projection calibration map to obtain the second feature point set.

[0032] Figure 3 This is a schematic diagram of feature points in an optional second projection calibration map, such as... Figure 3 As shown, the feature points in the second projection calibration map can be divided into internal feature points and boundary feature points. Internal feature points refer to the intersection points of the crosses of the black and white squares in the map, while boundary feature points refer to the outer corner points of the outermost ring of squares, that is, the corner points on the rectangle in the map, including T-shaped feature points on the sides and square-shaped feature points on the corners.

[0033] In some optional embodiments of this application, the first position information of each internal feature point in the second projection calibration map can be directly determined to obtain the second feature point set.

[0034] Specifically, the following steps can be taken to determine the initial location information of the internal feature points:

[0035] Step S1041: Binarize the second projection calibration map to obtain a binarized image. Considering that the local adaptive threshold algorithm is highly adaptable to uneven brightness, this method can be used to process the second projection calibration map to obtain a binarized image.

[0036] Step S1042: Dilate the binarized image and extract the contours from the dilated binarized image. Since the dilation affects white pixels, the black quadrilaterals can be shrunk, breaking their connections. The result is as follows: Figure 4 The image shown is only a portion of the processed image.

[0037] Step S1043: Calculate the convex hull of each contour and detect whether each convex hull satisfies the quadrilateral constraint condition to determine all quadrilaterals in the binarized image. At the same time, some interfering quadrilaterals can be removed by using constraints such as aspect ratio, perimeter, and area.

[0038] Step S1044: For any two adjacent quadrilaterals located at opposite corners, determine the midpoint of the diagonal line connecting the two corner points of the two adjacent quadrilaterals at opposite corners, and use it as an internal feature point, such as... Figure 4 As shown, for adjacent quadrilaterals a and b located at opposite corners, the midpoint of the diagonal AB connected to the two corner points A and B at opposite corners is an internal feature point.

[0039] By following the steps described above, the first position information of each internal feature point in the second projection calibration map can be determined. Optionally, other algorithms can also be used to determine the internal feature points, such as OpenCV's find-Chessboard-Corners algorithm.

[0040] To further optimize the final correction result, in some optional embodiments of this application, a more accurate feature point determination method is proposed. Specifically, the first position information of each internal feature point in the second projection calibration map and the second position information of each boundary feature point in the second projection calibration map can be determined simultaneously, and they are used together as the second feature point set.

[0041] The process of determining the first position information of the internal feature points is as described above. When determining the second position information of the boundary feature points, the initial position information of each boundary feature point can be initially determined based on the corner points and internal feature points of each quadrilateral. Then, for any boundary feature point, a window of a preset size is selected around the initial position of the boundary feature point, and each pixel in the window is traversed. The second position information of the boundary feature point is determined based on the gradient of each pixel.

[0042] like Figure 4 As shown, an internal feature point has been determined at corner C of quadrilateral c, while no internal feature points exist at the other three corner points. At this point, the positions of the other three corner points can be initially determined as the initial positions of the boundary feature points, and then the boundary feature points can be accurately located based on the gradient algorithm.

[0043] Specifically, for any boundary feature point, firstly, a window of a preset size (k1*k2, set by the user based on experience) is selected around its initial position. Then, each pixel in the window is traversed, and its gradient Gi (the gradient of the i-th pixel) is calculated. Finally, the second position information of the boundary feature point is calculated according to the following formula:

[0044]

[0045] In the formula, k1*k2 represents the number of pixels in the window, i represents the i-th pixel, and G... i Let [dx dy] be the gradient at the i-th pixel. For G i transpose matrix That is ω i p represents the weight of the i-th pixel. The weight decreases as the pixel is farther away from its initial position. i Let be the coordinates of the i-th pixel.

[0046] For k1*k2 pixels in the window, k1*k2 equations can be constructed to obtain a q value. The entire process is iterated and calculated multiple times until the calculation error between two adjacent calculations is less than det, where det is a value preset by the user based on experience. The final q is the second position information of the boundary feature point.

[0047] After determining all feature points in the second projection calibration map, and correlating them with the feature points in the first projection calibration map, the feature points in the second projection calibration map can be sorted. Let's assume the coordinates of the i-th feature point are (x...). i y i If the second feature point set corresponding to the second projection calibration map is Sp(x), then the second feature point set is Sp(x). i y i ).

[0048] Step S106: Perform line fitting on each feature point in the second feature point set to determine the position information of the intersection points of each line, and obtain the first intersection point set.

[0049] Each feature point in the second feature point set is a feature point in the second projection calibration map that has been distorted. To correct it, it is necessary to determine the standard position of the feature point. This application proposes to perform linear fitting on the feature points of the second projection calibration map, take the intersection of each line as the standard position of the feature point, compare the position information of the feature point and the intersection point, and then adjust the projected image based on the position difference information.

[0050] Specifically, based on the least squares method, straight lines can be fitted to each feature point in the second feature point set along the row direction to obtain a first number of row lines, and straight lines can be fitted to each feature point in the second feature point set along the column direction to obtain a second number of column lines; then the position information of the intersection points of each row line and each column line can be determined to obtain a first intersection point set, wherein each feature point in the first intersection point set corresponds one-to-one with each feature point in the second feature point set.

[0051] For example, for M*N feature points in the second feature point set, sorting all feature points from left to right, with N feature points in each row, yields M columns of feature points. Applying the least squares method to fit a straight line to each column of feature points yields M column lines. Similarly, sorting all feature points from top to bottom, with M feature points in each column, yields N rows of feature points. Applying the least squares method to fit a straight line to each row of feature points yields N row lines.

[0052] For any two straight lines a1*x + b1*y = c1 and a2*x + b2*y = c2, their intersection point is...

[0053]

[0054] Using the above formula, the positional information of the M*N intersection points of the M column lines and N row lines can be calculated, corresponding to the feature points in the first projection calibration map. The intersection points are then sorted, assuming the coordinates of the i-th intersection point are (x...). i y i If ), then the first set of intersection points is Dp(x). i y i ).

[0055] Step S108: Map the second set of feature points and the first set of intersection points onto the first projection calibration map to obtain the third set of feature points and the second set of intersection points.

[0056] Then, the second feature point set Sp(x) can be... i y i ) and the first intersection set Dp(x i y i Map them to the same coordinate system to determine the positional differences between each feature point and intersection point.

[0057] Specifically, a target mapping relationship can be determined based on a first set of feature points and a second set of feature points. The target mapping relationship is used to map feature points in the second projection calibration map to the first projection calibration map. Then, based on the target mapping relationship, each feature point in the second set of feature points is mapped to the first projection calibration map to obtain a third set of feature points. Each feature point in the first set of intersection points is mapped to the first projection calibration map to obtain a second set of intersection points. Each feature point in the third set of feature points corresponds one-to-one with each intersection point in the second set of intersection points.

[0058] Here, we can first establish the mapping equation src (original projected image) = H*dst (camera acquired image), and then set the first feature point set f(x) i y i ) and the second feature point set Sp(x) i y i Substituting into the equation, the target mapping relationship H is calculated using the least squares method. Then, the second feature point set Sp(x) is calculated. i y i Each feature point in the first projection calibration map is mapped to its coordinate information to obtain the third feature point set sp(x). i y i )=H*Sp(x i y i ); Calculate the first intersection set Dp(x) i y i Each intersection point in the first projection calibration map is mapped to the coordinate information of the second intersection point set dp(x) to obtain the second intersection point set dp(x) i y i )=H*Dp(x i y i ).

[0059] Step S110: Determine position correction information based on the third feature point set and the second intersection point set, and correct the projected image based on the position correction information.

[0060] Specifically, any set of corresponding feature points and intersections from the third feature point set and the second intersection point set are taken as the target point set. The positional difference information of the feature points and intersections in the target point set is determined, and the positional correction information of the feature points in the first feature point set corresponding to the target point set is determined based on the positional difference information. For example, for the i-th feature point in the first projection calibration map, its corresponding positional correction information, that is, the vector information that the feature point needs to be corrected, is dfi(xi, yi) = dp(xi, yi) - sp(xi, yi).

[0061] Subsequently, based on the position correction information, the positions of each feature point in the first feature point set are corrected to correct the projected image. Specifically, the position correction information can be transmitted to the projector's correction module to complete the correction of the distorted projected image.

[0062] Optionally, during the correction process, since the first feature point set has M*N feature points, which together form (M-2)*(N-2) square regions, for each square region, the position of all pixels in the square region can be corrected based on the position correction information corresponding to the four corner points of the square region, thereby realizing the distortion correction of the entire projected image.

[0063] In this embodiment, the projected image of a first projection calibration map projected by a projector is first acquired to obtain a second projection calibration map. The first projection calibration map has a corresponding first feature point set, which includes the position information of each feature point in the first projection calibration map. Simultaneously, the position information of each feature point in the second projection calibration map is determined to obtain a second feature point set. Then, straight line fitting is performed on each feature point in the second feature point set to determine the position information of the intersection points of each straight line, thus obtaining a first intersection point set. Next, the second feature point set and the first intersection point set are mapped onto the first projection calibration map respectively to obtain a third feature point set and a second intersection point set. Finally, position correction information is determined based on the third feature point set and the second intersection point set, and the projected image is corrected based on the position correction information. Specifically, by performing linear fitting on the feature points of the distorted second projection calibration map, the intersection point is taken as the standard position of the feature point. Then, the feature point and the intersection point are mapped to the same coordinate system, accurate position correction information can be calculated, thereby correcting the distorted projection image. This solves the technical problem of image distortion when the projector projects onto uneven media, affecting the user's viewing experience.

[0064] Example 2

[0065] According to an embodiment of this application, a projection image correction device for implementing the above-described projection image correction method is also provided, such as... Figure 5 As shown, the device includes an acquisition module 50, a determination module 52, a fitting module 54, a mapping module 56, and a correction module 58, wherein:

[0066] The acquisition module 50 is used to acquire the projected image of the first projection calibration map projected by the projector to obtain the second projection calibration map. The first projection calibration map has a corresponding first feature point set, which includes the position information of each feature point in the first projection calibration map.

[0067] Among them, the first projection calibration diagram is a type of projection shown by the projector, such as... Figure 2The chessboard grid diagram shown has corner points as feature points, each with fixed positional information. The feature points are sorted, and the coordinates of the i-th feature point are assumed to be (x...). i y i If the first feature point set corresponding to the first projection calibration map is f(x), then the first feature point set corresponding to the first projection calibration map is f(x). i y i ).

[0068] After the projector projects the first projection calibration map onto the wall, it can acquire an image of the corresponding projected image through an image sensor. This image is then used as the second projection calibration map. The image sensor can be an infrared camera or a full-color camera; it can be a mobile phone camera, a camera built into the projector, or an external camera. By performing linear fitting on the feature points of the distorted second projection calibration map, the intersection points are used as the standard positions of the feature points. The position information of the feature points and the intersection points is then compared to obtain the flatness information of the projection wall and the distortion information of the projected image. Finally, the projected image is adjusted based on the positional differences.

[0069] It should be noted that, in order to better collect the flatness information of the projection wall, when designing the first projection calibration map, the projection screen boundary must have feature point information, the feature area must have brightness information, and the feature point information should be as dense as possible, usually at least 9*9. In this embodiment, the number of feature points is set to M*N (M≥9, N≥9).

[0070] The determination module 52 is used to determine the position information of each feature point in the second projection calibration map, so as to obtain the second feature point set.

[0071] Figure 3 This is a schematic diagram of feature points in an optional second projection calibration map, such as... Figure 3 As shown, the feature points in the second projection calibration map can be divided into internal feature points and boundary feature points. Internal feature points refer to the intersection points of the crosses of the black and white squares in the map, while boundary feature points refer to the outer corner points of the outermost ring of squares, that is, the corner points on the rectangle in the map, including T-shaped feature points on the sides and square-shaped feature points on the corners.

[0072] In some optional embodiments of this application, the first position information of each internal feature point in the second projection calibration map can be directly determined to obtain the second feature point set.

[0073] Specifically, when determining the first location information of the internal feature points, the following steps can be followed: Binarize the second projection calibration map to obtain a binary image. Considering the strong adaptability of the local adaptive threshold algorithm to uneven brightness, this method can be used to process the second projection calibration map to obtain the binary image. Dilate the binary image and extract the contours from the dilated binary image. Since the dilated pixels are white pixels, the black quadrilaterals can be reduced in size, breaking their connections. The result is as follows: Figure 4 The image shown is only a portion of the processed image. The convex hull of each contour is calculated, and each convex hull is checked to determine if it satisfies quadrilateral constraints, thus identifying all quadrilaterals in the binarized image. Additionally, constraints such as aspect ratio, perimeter, and area can be used to remove some interfering quadrilaterals. For any two adjacent quadrilaterals at opposite corners, the midpoint of the diagonal between the two diagonal points of the two adjacent quadrilaterals is determined as an internal feature point, such as... Figure 4 As shown, for adjacent quadrilaterals a and b located at opposite corners, the midpoint of the diagonal AB connected to the two corner points A and B at opposite corners is an internal feature point.

[0074] By following the steps described above, the first position information of each internal feature point in the second projection calibration map can be determined. Optionally, other algorithms can also be used to determine the internal feature points, such as OpenCV's find-Chessboard-Corners algorithm.

[0075] To further optimize the final correction result, in some optional embodiments of this application, a more accurate feature point determination method is proposed. Specifically, the first position information of each internal feature point in the second projection calibration map and the second position information of each boundary feature point in the second projection calibration map can be determined simultaneously, and they are used together as the second feature point set.

[0076] The process of determining the first position information of the internal feature points is as described above. When determining the second position information of the boundary feature points, the initial position information of each boundary feature point can be initially determined based on the corner points and internal feature points of each quadrilateral. Then, for any boundary feature point, a window of a preset size is selected around the initial position of the boundary feature point, and each pixel in the window is traversed. The second position information of the boundary feature point is determined based on the gradient of each pixel.

[0077] like Figure 4 As shown, an internal feature point has been determined at corner C of quadrilateral c, while no internal feature points exist at the other three corner points. At this point, the positions of the other three corner points can be initially determined as the initial positions of the boundary feature points, and then the boundary feature points can be accurately located based on the gradient algorithm.

[0078] Specifically, for any boundary feature point, firstly, a window of a preset size (k1*k2, set by the user based on experience) is selected around its initial position. Then, each pixel in the window is traversed, and its gradient Gi (the gradient of the i-th pixel) is calculated. Finally, the second position information of the boundary feature point is calculated according to the following formula:

[0079]

[0080] In the formula, k1*k2 represents the number of pixels in the window, i represents the i-th pixel, and Gi is the gradient [dx dy] at the i-th pixel. For G i transpose matrix That is ω i p represents the weight of the i-th pixel. The weight decreases as the pixel is farther away from its initial position. i Let be the coordinates of the i-th pixel.

[0081] For k1*k2 pixels in the window, k1*k2 equations can be constructed to obtain a q value. The entire process is iterated and calculated multiple times until the calculation error between two adjacent calculations is less than det, where det is a value preset by the user based on experience. The final q is the second position information of the boundary feature point.

[0082] After determining all feature points in the second projection calibration map, and correlating them with the feature points in the first projection calibration map, the feature points in the second projection calibration map can be sorted. Let's assume the coordinates of the i-th feature point are (x...). i ,y i If the second feature point set corresponding to the second projection calibration map is Sp(x), then the second feature point set is Sp(x). i y i ).

[0083] The fitting module 54 is used to perform linear fitting on each feature point in the second feature point set, determine the position information of the intersection points of each line, and obtain the first intersection point set.

[0084] Each feature point in the second feature point set is a feature point in the second projection calibration map that has been distorted. To correct it, it is necessary to determine the standard position of the feature point. This application proposes to perform linear fitting on the feature points of the second projection calibration map, take the intersection of each line as the standard position of the feature point, compare the position information of the feature point and the intersection point, and then adjust the projected image based on the position difference information.

[0085] Specifically, based on the least squares method, straight lines can be fitted to each feature point in the second feature point set along the row direction to obtain a first number of row lines, and straight lines can be fitted to each feature point in the second feature point set along the column direction to obtain a second number of column lines; then the position information of the intersection points of each row line and each column line can be determined to obtain a first intersection point set, wherein each feature point in the first intersection point set corresponds one-to-one with each feature point in the second feature point set.

[0086] For example, for M*N feature points in the second feature point set, sorting all feature points from left to right, with N feature points in each row, yields M columns of feature points. Applying the least squares method to fit a straight line to each column of feature points yields M column lines. Similarly, sorting all feature points from top to bottom, with M feature points in each column, yields N rows of feature points. Applying the least squares method to fit a straight line to each row of feature points yields N row lines.

[0087] For any two straight lines a1*x + b1*y = c1 and a2*x + b2*y = c2, their intersection point is...

[0088]

[0089] Using the above formula, the positional information of the M*N intersection points of the M column lines and N row lines can be calculated, corresponding to the feature points in the first projection calibration map. The intersection points are then sorted, assuming the coordinates of the i-th intersection point are (x...). i y i If ), then the first set of intersection points is Dp(x). i y i ).

[0090] The mapping module 56 is used to map the second feature point set and the first intersection point set to the first projection calibration map respectively, so as to obtain the third feature point set and the second intersection point set.

[0091] Then, the second feature point set Sp(x) can be... i y i ) and the first intersection set Dp(x i y i Map them to the same coordinate system to determine the positional differences between each feature point and intersection point.

[0092] Specifically, a target mapping relationship can be determined based on a first set of feature points and a second set of feature points. The target mapping relationship is used to map feature points in the second projection calibration map to the first projection calibration map. Then, based on the target mapping relationship, each feature point in the second set of feature points is mapped to the first projection calibration map to obtain a third set of feature points. Each feature point in the first set of intersection points is mapped to the first projection calibration map to obtain a second set of intersection points. Each feature point in the third set of feature points corresponds one-to-one with each intersection point in the second set of intersection points.

[0093] Here, we can first establish the mapping equation src (original projected image) = H*dst (camera acquired image), and then set the first feature point set f(x) i y i ) and the second feature point set Sp(x) i y i Substituting into the equation, the target mapping relationship H is calculated using the least squares method. Then, the second feature point set Sp(x) is calculated. i y i Each feature point in the first projection calibration map is mapped to its coordinate information to obtain the third feature point set sp(x). i y i )=H*Sp(x i y i ); Calculate the first intersection set Dp(x) i y i Each intersection point in the first projection calibration map is mapped to the coordinate information of the second intersection point set dp(x) to obtain the second intersection point set dp(x) i y i )=H*Dp(x i y i ).

[0094] The correction module 58 is used to determine position correction information based on the third feature point set and the second intersection point set, and to correct the projected image based on the position correction information.

[0095] Specifically, any set of corresponding feature points and intersections from the third feature point set and the second intersection point set are taken as the target point set. The positional difference information of the feature points and intersections in the target point set is determined, and the positional correction information of the feature points in the first feature point set corresponding to the target point set is determined based on the positional difference information. For example, for the i-th feature point in the first projection calibration map, its corresponding positional correction information, that is, the vector information that the feature point needs to be corrected, is dfi(xi, yi) = dp(xi, yi) - sp(xi, yi).

[0096] Subsequently, based on the position correction information, the positions of each feature point in the first feature point set are corrected to correct the projected image. Specifically, the position correction information can be transmitted to the projector's correction module to complete the correction of the distorted projected image.

[0097] Optionally, during the correction process, since the first feature point set has M*N feature points, which together form (M-2)*(N-2) square regions, for each square region, the position of all pixels in the square region can be corrected based on the position correction information corresponding to the four corner points of the square region, thereby realizing the distortion correction of the entire projected image.

[0098] It should be noted that each module in the projection screen correction device in this application embodiment corresponds one-to-one with the implementation steps of the projection screen correction method in embodiment 1. Since embodiment 1 has been described in detail, some details not shown in this embodiment can be referred to embodiment 1, and will not be elaborated further here.

[0099] Example 3

[0100] According to an embodiment of this application, a non-volatile storage medium is also provided, which includes a stored program, wherein the program controls the device where the non-volatile storage medium is located to execute the projection screen correction method in Embodiment 1 during runtime.

[0101] According to an embodiment of this application, an electronic device is also provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the projection image correction method of Embodiment 1 through the computer program.

[0102] Optionally, the following steps are executed during program execution: acquiring the projected image of a first projection calibration map projected by the projector to obtain a second projection calibration map, wherein the first projection calibration map has a corresponding first feature point set, the first feature point set including the position information of each feature point in the first projection calibration map; determining the position information of each feature point in the second projection calibration map to obtain a second feature point set; performing line fitting on each feature point in the second feature point set to determine the position information of the intersection points of each line, obtaining a first intersection point set; mapping the second feature point set and the first intersection point set onto the first projection calibration map respectively to obtain a third feature point set and a second intersection point set; determining position correction information based on the third feature point set and the second intersection point set, and correcting the projected image based on the position correction information.

[0103] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0104] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0106] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0107] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0108] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0109] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for correcting a projected image, characterized in that, include: The projected image of the first projection calibration map projected by the projector is collected to obtain a second projection calibration map with image distortion. The first projection calibration map has a corresponding first feature point set, which includes the position information of each feature point in the first projection calibration map. Determine the position information of each feature point in the second projection calibration map to obtain the second feature point set; Line fitting is performed on each feature point in the second feature point set to determine the position information of the intersection points of each line, thus obtaining the first intersection point set; The second set of feature points and the first set of intersection points are mapped onto the first projection calibration map to obtain a third set of feature points and a second set of intersection points, wherein each feature point in the third set of feature points corresponds one-to-one with each intersection point in the second set of intersection points; The process of determining position correction information based on the third set of feature points and the second set of intersection points, and correcting the projected image based on the position correction information, includes: taking each set of corresponding feature points and intersection points in the third set of feature points and the second set of intersection points as a target point group; for each target point group, determining the position difference information of the feature points and intersection points in the target point group; determining the position correction information of the feature points in the first set of feature points corresponding to the target point group based on the position difference information; and correcting the position of each feature point in the first set of feature points based on the position correction information to achieve the correction of the projected image.

2. The method according to claim 1, characterized in that, Line fitting is performed on each feature point in the second feature point set to determine the position information of the intersection points of each line, thus obtaining the first intersection point set, including: Based on the least squares method, straight lines are fitted to each feature point in the second feature point set along the row direction to obtain a first number of row lines, and straight lines are fitted to each feature point in the second feature point set along the column direction to obtain a second number of column lines. The position information of the intersection points of each row line and each column line is determined to obtain the first intersection point set, wherein each feature point in the first intersection point set corresponds one-to-one with each feature point in the second feature point set.

3. The method according to claim 1, characterized in that, Mapping the second set of feature points and the first set of intersection points onto the first projection calibration map respectively yields a third set of feature points and a second set of intersection points, including: A target mapping relationship is determined based on the first set of feature points and the second set of feature points. The target mapping relationship is used to map feature points in the second projection calibration map to the first projection calibration map. Based on the target mapping relationship, each feature point in the second feature point set is mapped to the first projection calibration map to obtain the third feature point set, and each feature point in the first intersection point set is mapped to the first projection calibration map to obtain the second intersection point set.

4. The method according to claim 1, characterized in that, The positional information of each feature point in the second projection calibration map is determined to obtain the second feature point set, including: Determine the first position information of each internal feature point in the second projection calibration map to obtain the second feature point set; or, The first position information of each internal feature point in the second projection calibration map is determined, and the second position information of each boundary feature point in the second projection calibration map is determined, thereby obtaining the second feature point set.

5. The method according to claim 4, characterized in that, Determining the first position information of each internal feature point in the second projection calibration map includes: The second projection calibration map is binarized to obtain a binarized image; The binarized image is dilated, and the contours in the dilated binarized image are extracted. Calculate the convex hull of each contour and detect whether each convex hull satisfies the quadrilateral constraint to determine all quadrilaterals in the binarized image; For any two adjacent quadrilaterals located at opposite corners, the midpoint of the diagonal line between the two corner points located at opposite corners of the two adjacent quadrilaterals is determined as an internal feature point; Determine the first position information of each of the internal feature points.

6. The method according to claim 5, characterized in that, Determining the second position information of each boundary feature point in the second projection calibration map includes: Based on the corner points and internal feature points of each quadrilateral, the initial position information of each boundary feature point is initially determined; For any of the boundary feature points, a window of a preset size is selected around the initial position of the boundary feature point, and each pixel in the window is traversed. The second position information of the boundary feature point is determined based on the gradient of each pixel.

7. A projection image correction device, characterized in that, include: The acquisition module is used to acquire the projected image of the first projection calibration map projected by the projector and obtain a second projection calibration map with image distortion. The first projection calibration map has a corresponding first feature point set, which includes the position information of each feature point in the first projection calibration map. The determination module is used to determine the position information of each feature point in the second projection calibration map, and obtain the second feature point set; The fitting module is used to perform line fitting on each feature point in the second feature point set, determine the position information of the intersection points of each line, and obtain the first intersection point set. The mapping module is used to map the second feature point set and the first intersection point set to the first projection calibration map respectively to obtain the third feature point set and the second intersection point set, wherein each feature point in the third feature point set corresponds one-to-one with each intersection point in the second intersection point set; The correction module is used to determine position correction information based on the third feature point set and the second intersection point set, and to correct the projected image based on the position correction information. This includes: treating each set of corresponding feature points and intersection points in the third feature point set and the second intersection point set as a target point group; for each target point group, determining the position difference information between the feature points and intersection points in the target point group; determining the position correction information of the feature points in the first feature point set corresponding to the target point group based on the position difference information; and correcting the position of each feature point in the first feature point set based on the position correction information to achieve the correction of the projected image.

8. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the non-volatile storage medium to perform the projection image correction method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the projection image correction method according to any one of claims 1 to 6 through the computer program.

Images