Image correction method and ultra-short focus projection device

Abstract

The embodiment of the application discloses an image correction method and an ultra-short-focus projection device, and belongs to the technical field of projection display. Two cameras are arranged in the ultra-short-focus projection device. Since the two cameras are located on the two sides of a light outlet, the two cameras can form a field of view difference. The two cameras are used to respectively shoot a projection screen on which a correction graph is projected. Compared with a scheme in which one camera is arranged, the field of view difference of the two cameras can obtain information with higher resolution, and is beneficial to image correction, so as to better improve perspective distortion and lens distortion. In addition, information that cannot be collected by one camera, for example, a part of the projection screen that is shielded by ambient light, has a certain probability of being collected by the other camera, so that the influence of ambient light and the difficulty of image correction are reduced by using the two cameras.

Description

Technical Field

[0001] This application relates to the field of projection display technology, and in particular to an image correction method and an ultra-short-throw projection device. Background Technology

[0002] With technological advancements, ultra-short-throw lenses are now used to manufacture ultra-short-throw projection devices such as laser TVs and projectors. Ultra-short-throw projectors use a camera on the projector itself to capture images projected onto the screen for image correction. This ensures the projected image is not only projected onto the screen but also exhibits minimal distortion, guaranteeing alignment between the projected image and the screen. In addition to using an ultra-short-throw lens for the projection lens, the camera also needs to employ an ultra-short-throw lens to capture the most complete projected image. However, due to the large field of view of ultra-short-throw lenses and the non-parallelism between the optical axes of the projection and camera lenses, the images captured by the camera suffer from significant perspective and lens distortion, making image correction quite challenging. Furthermore, ambient light also affects the effectiveness of the images captured by the camera, further complicating image correction. Summary of the Invention

[0003] This application provides an image correction method and an ultra-short-throw projection device, which can better alleviate perspective distortion and lens distortion, reduce the influence of ambient light, and simplify image correction. The technical solution is as follows:

[0004] On the one hand, an image correction method is provided for use in an ultra-short-throw projection device, the ultra-short-throw projection device including a projection screen and a projection host, the projection host including a first camera, a second camera and a light outlet, the first camera and the second camera being located on both sides of the light outlet respectively;

[0005] The method includes:

[0006] The calibration chart is projected onto the projection screen through the light outlet;

[0007] The first camera and the second camera are controlled to capture images of the projection screen on which the calibration chart is projected, so as to obtain a first captured image and a second captured image.

[0008] The image to be projected by the projector is corrected based on the first captured image and the second captured image.

[0009] Optionally, the correction chart has feature points;

[0010] The step of correcting the image to be projected by the projector based on the first captured image and the second captured image includes:

[0011] Based on the first captured image and the second captured image, determine the positional relationship between the feature points on the correction chart and the edge of the projection screen;

[0012] Based on the positional relationship, the image to be projected is corrected.

[0013] Optionally, the calibration chart includes multiple charts, and different charts have different feature points. The first captured image includes multiple first images that correspond one-to-one with the multiple charts, and the second captured image includes multiple second images that correspond one-to-one with the multiple charts.

[0014] Determining the positional relationship between feature points on the calibration chart and the edge of the projection screen based on the first captured image and the second captured image includes:

[0015] Feature points in the plurality of first images and the plurality of second images, as well as the edges of the projection screen, are detected to determine at least one pair of images. Each pair of images includes a first image and a second image corresponding to the same image card, and each pair of images contains all feature points on the same image card and the edges of the projection screen.

[0016] The positional relationship is determined based on the at least one pair of images.

[0017] Optionally, if both the first image and the second image corresponding to the same image card contain the first feature point, then the positional relationship between the first feature point and the edge of the projection screen is determined based on the first feature point with higher resolution among the two first feature points, wherein the first feature point is a feature point in the same image card; and / or,

[0018] If the first image corresponding to the same image card contains the first feature point, but the second image corresponding to the same image card does not contain the first feature point, then the positional relationship between the first feature point and the edge of the projection screen is determined based on the first feature point contained in the first image corresponding to the same image card, wherein the first feature point is a feature point in the same image card.

[0019] Optionally, the multiple image cards are projected sequentially, and the non-first image card is projected when neither the first camera nor the second camera has captured all the feature points on the previous image card.

[0020] Optionally, the method further includes:

[0021] If, after projecting the multiple image cards, not all feature points on the same image card and / or the edges of the projection screen are captured, a first prompt message is generated to prompt the user to block ambient light.

[0022] Optionally, the image correction includes image compression;

[0023] The correction is performed when the projected correction chart is detected to cover the projection screen.

[0024] Optionally, the method further includes:

[0025] The detection results are obtained by detecting whether the projected image of the calibration chart covers the projection screen.

[0026] If the detection result indicates that the projected image does not cover the projection screen, a second prompt message is generated, which instructs the user to adjust the position of the projection host.

[0027] Optionally, the projection screen includes a first part and a second part, wherein the first part is located within the shooting range of the first camera and the second part is located within the shooting range of the second camera.

[0028] On the other hand, an image correction device is provided for use in an ultra-short-throw projection device, the ultra-short-throw projection device including a projection screen and a projection host, the projection host including a first camera, a second camera and a light outlet, the first camera and the second camera being located on both sides of the light outlet respectively;

[0029] The device includes:

[0030] The projection module is used to project the calibration chart onto the projection screen through the light outlet;

[0031] The control module is used to control the first camera and the second camera to respectively capture images of the projection screen on which the correction chart is projected, so as to obtain a first captured image and a second captured image;

[0032] The correction module is used to correct the image to be projected by the projector based on the first captured image and the second captured image.

[0033] Optionally, the correction chart has feature points;

[0034] The correction module includes:

[0035] The determination submodule is used to determine the positional relationship between feature points on the calibration chart and the edge of the projection screen based on the first captured image and the second captured image;

[0036] The correction submodule is used to correct the image to be projected based on the positional relationship.

[0037] Optionally, the calibration chart includes multiple charts, and different charts have different feature points. The first captured image includes multiple first images that correspond one-to-one with the multiple charts, and the second captured image includes multiple second images that correspond one-to-one with the multiple charts.

[0038] The determination submodule is specifically used for:

[0039] Feature points in the plurality of first images and the plurality of second images, as well as the edges of the projection screen, are detected to determine at least one pair of images. Each pair of images includes a first image and a second image corresponding to the same image card, and each pair of images contains all feature points on the same image card and the edges of the projection screen.

[0040] The positional relationship is determined based on the at least one pair of images.

[0041] Optionally, if both the first image and the second image corresponding to the same image card contain the first feature point, then the positional relationship between the first feature point and the edge of the projection screen is determined based on the first feature point with higher resolution among the two first feature points, wherein the first feature point is a feature point in the same image card; and / or,

[0042] If the first image corresponding to the same image card contains the first feature point, but the second image corresponding to the same image card does not contain the first feature point, then the positional relationship between the first feature point and the edge of the projection screen is determined based on the first feature point contained in the first image corresponding to the same image card, wherein the first feature point is a feature point in the same image card.

[0043] Optionally, the multiple image cards are projected sequentially, and the non-first image card is projected when neither the first camera nor the second camera has captured all the feature points on the previous image card.

[0044] Optionally, the device further includes:

[0045] The first generation module is used to generate a first prompt message if, after projecting the multiple image cards, not all feature points on the same image card and / or the edge of the projection screen are captured, the first prompt message is used to prompt the user to block the ambient light.

[0046] Optionally, the image correction includes image compression;

[0047] The correction is performed when the projected correction chart is detected to cover the projection screen.

[0048] Optionally, the device further includes:

[0049] The detection module is used to detect whether the projected image of the calibration chart covers the projection screen, so as to obtain the detection result;

[0050] The second generation module is used to generate a second prompt message if the detection result indicates that the projected image does not cover the projection screen, and the second prompt message instructs the user to adjust the position of the projection host.

[0051] Optionally, the projection screen includes a first part and a second part, wherein the first part is located within the shooting range of the first camera and the second part is located within the shooting range of the second camera.

[0052] On the other hand, an ultra-short-throw projection device is provided, which includes a projection screen and a projection host. The projection host includes a first camera, a second camera, and a light outlet, with the first camera and the second camera located on opposite sides of the light outlet. The ultra-short-throw projection device is used to implement the steps of the image correction method described above.

[0053] Optionally, the projection host further includes a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the communication bus. The memory is used to store computer programs, and the processor is used to execute the programs stored in the memory to implement the steps of the image correction method described above.

[0054] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of the image correction method described above.

[0055] On the other hand, a computer program product containing instructions is provided that, when run on a processor, causes the processor to perform the steps of the image correction method described above.

[0056] The technical solutions provided in this application can bring at least the following beneficial effects:

[0057] When the optical axis of an ultra-short-throw projector lens is not parallel to the optical axis of the camera, the image captured by the camera is prone to significant distortion. To address this issue, two cameras are placed within the ultra-short-throw projector. Since these two cameras are located on opposite sides of the light outlet, they create a difference in field of view. By using these two cameras to capture images of the projection screen displaying the calibration chart, compared to a single-camera setup, the difference in field of view between the two cameras allows for the acquisition of higher-resolution information, which is beneficial for image correction and better reduces perspective and lens distortion. Furthermore, information that one camera cannot capture, such as parts of the projection screen obscured by ambient light, has a certain probability of being captured by the other camera. Thus, using two cameras reduces the impact of ambient light and the difficulty of image correction. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a schematic diagram of the principle of an ultra-short throw projection device in related technologies;

[0060] Figure 2 This is a top view schematic diagram of an ultra-short-throw projection device provided in an embodiment of this application;

[0061] Figure 3 This is a side view of a projection host provided in an embodiment of this application;

[0062] Figure 4 This is a schematic diagram of an ultra-short-throw projection device provided in an embodiment of this application;

[0063] Figure 5 This is a schematic diagram of another ultra-short-throw projection device provided in the embodiments of this application;

[0064] Figure 6 This is a schematic diagram of a dual-camera partitioned shooting method provided in an embodiment of this application;

[0065] Figure 7 This is a schematic diagram of another ultra-short-throw projection device provided in the embodiments of this application;

[0066] Figure 8 This is a flowchart of an image correction method provided in an embodiment of this application;

[0067] Figure 9This is a schematic diagram illustrating the difference in field of view between two cameras according to an embodiment of this application;

[0068] Figure 10 This is a flowchart of another image correction method provided in the embodiments of this application;

[0069] Figure 11 This is a schematic diagram of the structure of an image correction device provided in an embodiment of this application;

[0070] Figure 12 This is a schematic diagram of the structure of an ultra-short-throw projection device provided in an embodiment of this application. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0072] To facilitate understanding, the background technology involved in the embodiments of this application will be introduced first.

[0073] See Figure 1 Ultra-short-throw projection devices, such as laser TVs and ultra-short-throw projectors, include a projection screen and a projector unit. The projector unit includes a camera, a projection lens (not shown), and a light output port. The projection lens is an ultra-short-throw lens. The camera, in order to capture as much of the projected image and the edges of the projection screen as possible, also needs to use an ultra-short-throw lens. Because the optical axis of the projection lens is not parallel to the optical axis of the camera, in addition to the significant lens distortion of the ultra-short-throw lens, the image captured by the camera also suffers from significant perspective distortion, i.e., distortion where objects appear smaller than they are larger. For example... Figure 1 As shown, the images captured by the camera exhibit severe distortion. Furthermore, when the camera shoots upwards or downwards at the projection screen, it is significantly affected by ambient light, potentially capturing abnormal light spots that obscure the projected feature points, hindering image correction.

[0074] This application provides an image correction method that can be applied to various ultra-short-throw projection devices in various scenarios, effectively improving perspective distortion and lens distortion, reducing the influence of ambient light, and simplifying image correction.

[0075] Figure 2 This is a top view schematic diagram of an ultra-short-throw projection device provided in an embodiment of this application. Figure 3 This is a side view of a projection host provided in an embodiment of this application. See also... Figure 2 and Figure 3The ultra-short-throw projection device includes a projection screen and a projection unit. The projection unit includes a first camera, a second camera, and a light output port, with the first and second cameras located on opposite sides of the light output port. It should be understood that the projection unit also includes a projection lens (not shown). The projection screen can be a projection screen, a television screen, or a wall surface, etc.

[0076] Figure 2 Taking a rectangular light-emitting port as an example, other shapes are possible in other embodiments. The distance between the first camera and the light-emitting port is X1, and the distance between the second camera and the light-emitting port is X2. X1 and X2 have no size restrictions; X1 can be equal to X2 or not. When the light-emitting port is located in the middle of the projector and X1 equals X2, that is, in the case of a symmetrical dual-camera layout, the projector is relatively more aesthetically pleasing. The following description will use X1 equal to X2 as an example, meaning that the distance between the first camera and the light-emitting port is equal to the distance between the second camera and the light-emitting port.

[0077] In this embodiment, the first camera and the second camera can be the same camera or different cameras. This example uses the first camera and the second camera being the same camera.

[0078] like Figure 4 and Figure 5 As shown, the larger the values ​​of X1 and X2, the closer the two cameras are to the edge of the projector, i.e., the farther away from the light outlet. Due to perspective distortion, the greater the difference between the images captured by the two cameras, for example... Figure 5 As shown, the left side of the image captured by the first camera is magnified, appearing much larger than the left side of the image captured by the second camera. However, this magnification results in higher resolution, more pixels, and richer detail in the images captured by each camera, which can improve the accuracy of image correction. The smaller the values ​​of X1 and X2, the closer the two cameras are to the projector, resulting in smaller differences between the images captured by these two cameras, and a more symmetrical left and right side of the image.

[0079] In this embodiment, the distance between the first camera and the projection screen is equal to the distance between the second camera and the projection screen, or the distance between the first camera and the projection screen is not equal to the distance between the second camera and the projection screen. Furthermore, the distances between the first camera and the projection screen, and the distances between the second camera and the projection screen, are both not less than the distance between the light output port and the projection screen. This ensures that the focal lengths of the first and second cameras are less than or equal to the focal length of the projection lens, and that as many images as possible are captured. In other embodiments, the distances between the first camera and the projection screen, and the distances between the second camera and the projection screen, may also be less than the distance between the light output port and the projection screen.

[0080] like Figure 2 As shown, the distance between the light outlet and the projection screen is L1, and the distances between the first camera and the projection screen, as well as the distances between the second camera and the projection screen, are both L2. In this embodiment, L2 is greater than or equal to L1.

[0081] The projection screen comprises a first part and a second part, wherein the first part is within the field of view of a first camera, and the second part is within the field of view of a second camera. It should be understood that all or part of the first part may also be within the field of view of the second camera, and all or part of the second part may also be within the field of view of the first camera. That is, the combined field of view of the first and second cameras is sufficient to cover the entire projection screen; either the field of view of the first camera or the field of view of the second camera can encompass the entire projection screen. Alternatively, to reduce the focal length of the cameras, such as... Figure 6 As shown, the left side of the projection screen is the first part, and the right side is the second part. The first camera mainly captures the left side of the projection screen, and the second camera mainly captures the right side of the projection screen. By integrating the images captured by the first and second cameras, the entire projection screen image can be obtained. This can further improve the clarity or resolution of the image required for image correction and reduce the impact of lens distortion.

[0082] Figure 7 This is a schematic diagram illustrating the principle of another ultra-short-throw projection device provided in an embodiment of this application. See also... Figure 7 An ultra-short-throw projector consists of a projector unit and a projection screen. The projector unit includes a power supply, an optical engine, a display system, a main control system, and two cameras, which can be referred to as the first camera and the second camera.

[0083] The power source can be any type of power source.

[0084] An optical engine is used for projection display. The optical engine includes a light source module, an illumination module, a projection chip, a projection lens (using an ultra-short throw lens), and a light output port. The light source module receives power and outputs a light signal. The illumination module receives this light signal and illuminates the screen. The projection chip modulates the light under the illumination and the drive signal of the display, thereby projecting the image to be projected from the light output port through the projection lens. For example, the optical engine is used to project a calibration chart onto the projection screen through the light output port.

[0085] The display system receives image or video signals from the main control system to obtain the image to be projected and sends display drive signals to the projection chip. For example, the display system receives calibration charts from the main control system. The display system also sends laser drive signals to the power supply to instruct the power supply to power the laser in the light source module.

[0086] The main control system is used for image correction and optionally also has functions such as video decoding, image decoding, and human-computer interaction. In this embodiment, the main control system is used to send a correction pattern card to the display system and control the first camera and the second camera to respectively capture images of the projection screen on which the correction pattern card is projected, so as to obtain a first captured image and a second captured image. The main control system is also used to determine image correction parameters based on the first captured image and the second captured image. After determining the image correction parameters, the main control system uses the image correction parameters to correct the image to be projected on the projector and sends the corrected image to the display system for projection. Alternatively, after determining the image correction parameters, the main control system sends the image correction parameters to the display system, and the display system uses the image correction parameters to correct the image to be projected.

[0087] The first and second cameras are used to capture images of the projection screen onto which the calibration chart is projected. Optionally, the images captured by these two cameras are used by the main control system to determine the positional relationship between feature points in the calibration chart and the projection screen, and based on this positional relationship, to determine image correction parameters.

[0088] The projection screen is used to display images. In this embodiment, automatic image projection is achieved through image correction, that is, the image is projected onto the inside of the projection screen with minimal distortion.

[0089] It should be understood that the system architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0090] The image correction method provided in the embodiments of this application will now be explained in detail.

[0091] Figure 8 This is a flowchart illustrating an image correction method provided in an embodiment of this application. The image correction method is applied to an ultra-short-throw projection device, which includes a projection screen and a projection host. The projection screen includes a first camera, a second camera, and a light outlet, with the first and second cameras located on opposite sides of the light outlet. This ultra-short-throw projection device can... Figures 2 to 7 The ultra-short throw projection device shown is for your reference. Figure 8 The method includes the following steps.

[0092] Step 801: Project the calibration chart onto the projection screen through the light outlet.

[0093] In this embodiment, the ultra-short-throw projection device stores a calibration chart, which is used for calibration. During image calibration, the ultra-short-throw projection device projects the calibration chart onto the projection screen through its light output port.

[0094] The calibration chart includes at least one chart, each chart having feature points, which can be circles, squares, or other types of dots. For example, the chart is as follows: Figure 9 The black dots shown, or the pattern like a black and white checkerboard.

[0095] In some embodiments, the calibration chart includes multiple charts, each with different feature points. Optionally, these multiple charts are projected sequentially, with non-first charts projected only after neither the first nor the second camera has captured all the feature points of the previous chart. It should be understood that for each chart projected and captured by the two cameras, two images are obtained, and the combined images are checked to see if they contain all the feature points of the captured chart. In short, multiple charts are projected until complete feature point information for the same chart is acquired.

[0096] For example, the ultra-short-throw projection device stores 10 image cards, each with multiple black dots. The positions of the black dots differ between different image cards. In this instance, three image cards are projected to capture all feature points on the same image card using both the first and second cameras. Specifically, after projecting the first image card, neither the first nor second camera captures all feature points on it. The second image card is then projected, and so on, until the last image card (the third image card) is projected. Only then do the first and second cameras capture all feature points on the last image card. It should be understood that storing multiple image cards reduces the influence of ambient light, thus enabling image correction. For instance, some feature points on the first and second projected image cards may be obscured by ambient light, preventing the first and second cameras from capturing them. However, the feature points on the third image card are not obscured by ambient light and can be captured by the first and / or second cameras.

[0097] See Figure 9 The ultra-short throw projection device projects a picture card onto the projection screen. The picture card includes four feature points, namely feature points 1-4. The first camera and the second camera are located on both sides of the light outlet. Due to the influence of ambient light, the first camera on the left did not capture feature point 1, while the second camera on the right was able to capture feature point 1.

[0098] Optionally, the calibration chart includes multiple charts, which are projected sequentially. However, after each chart is projected and photographed to obtain two images, it is not necessary to immediately detect whether the two images combined contain all the feature points on the chart.

[0099] For example, each image correction is set to project and photograph 5 image cards, and these 5 image cards are projected and photographed in sequence. Regardless of whether all feature points on the same image card were captured in the previous photograph, the next image card is projected.

[0100] Optionally, the multiple image cards may be the same size or they may differ. This application embodiment uses the example of multiple image cards being the same size for illustration.

[0101] Before image correction, the ultra-short-throw projector initiates the image correction process. That is, before projecting the correction chart onto the projection screen through the light output port, the ultra-short-throw projector initiates the image correction process to facilitate image correction as needed.

[0102] There are various ways to initiate image correction, and this application embodiment does not limit this method. For example, the image correction process can be triggered by a first camera, a second camera, or other sensors (such as a gyroscope or a level) when the projected image is detected to be misaligned with the projection screen or when the host device moves. Another example is that the image correction process can be initiated manually by the user.

[0103] For example, combined Figure 7 If the projector of an ultra-short throw projector moves, causing a positional difference between the projected image and the projection screen, the first camera, which works in real time, will capture an image of the projection screen and send the captured image to the main control system. Based on this image, the main control system will determine that the projected image and the projection screen are not aligned and will initiate the image correction process.

[0104] Step 802: Control the first camera and the second camera to capture images of the projection screen on which the calibration chart is projected, so as to obtain the first captured image and the second captured image.

[0105] When the calibration chart includes a single chart, after projecting the chart, the ultra-short-throw projection device controls the first and second cameras to respectively capture images of the projection screen displaying the chart, thereby obtaining a first image and a second image. Each of the first and second images consists of a single image.

[0106] When the calibration chart includes multiple charts, the first captured image includes multiple first images that correspond one-to-one with the multiple charts, and the second captured image includes multiple second images that correspond one-to-one with the multiple charts.

[0107] In one embodiment, after projecting the first image card, the ultra-short-throw projection device controls a first camera and a second camera to respectively capture images of the projection screen displaying the first image card, thus obtaining a first image and a first second image. The ultra-short-throw projection device detects feature points in the obtained first and second images. If the detection result indicates that not all feature points on the first image card have been captured, the device continues to project the second image card, controlling the first and second cameras to respectively capture images of the projection screen displaying that image card, thus obtaining the next first image and the next second image. The ultra-short-throw projection device detects feature points in the obtained first and second images. If the detection result indicates that not all feature points on the first image card have been captured, the device continues to project the second image card, until the last image card in the plurality of image cards is projected and all feature points on the last image card are detected. At this point, the ultra-short-throw projection device controls the first and second cameras to complete the capturing process, obtaining multiple first images and multiple second images.

[0108] In another embodiment, the ultra-short-throw projection device projects multiple image cards sequentially, eliminating the need for feature point detection after each acquisition of a first image and a second image.

[0109] Taking the images of image card 1 captured by the first camera and the second camera as an example, the differences between the images captured by the first camera and the second camera are as follows: Figure 5 As shown.

[0110] Optionally, since the ultra-short-throw projection device subsequently performs image correction by determining the positional relationship between feature points and the projection screen, when acquiring the first first image and the first second image, or when acquiring multiple first images and multiple second images, the ultra-short-throw projection device detects whether the integrated first and second images contain complete edge information of the projection screen. If edge information is detected, the ultra-short-throw projection device continues to acquire subsequent images. If complete edge information is not detected, the ultra-short-throw projection device generates a prompt message indicating that image correction cannot be completed and the position of the projector needs to be adjusted.

[0111] In some embodiments, image correction includes image compression, and image correction is performed upon detection that the projected correction pattern card covers the projection screen. Based on this, upon acquiring a first first image and a first second image, or upon acquiring multiple first images and multiple second images, the ultra-short-throw projection device further detects whether the projected image of the correction pattern card covers the entire projection screen based on the acquired images, to obtain a detection result. If the detection result indicates that the projected image does not cover the projection screen, the ultra-short-throw projection device generates a second prompt message, which instructs the user to adjust the position of the projector.

[0112] In other embodiments, image correction includes image expansion. Based on this, when a first first image and a first second image are acquired, or when multiple first images and multiple second images are acquired, the ultra-short-throw projection device does not need to detect whether the projected image covers the entire projection screen based on the acquired images.

[0113] Furthermore, if the ultra-short-throw projector fails to capture all feature points on the same image card and / or the edges of the projection screen after projecting multiple image cards, the ultra-short-throw projector generates a first prompt message to remind the user to shield the ambient light.

[0114] Optionally, the distances between the first camera and the projection screen, and between the second camera and the projection screen, are both no less than the distance between the light output port and the projection screen. This ensures that the focal lengths of the first and second cameras are less than or equal to the focal length of the projection lens, capturing as many images as possible. In other words, the smaller the focal length of the camera, the less distortion, which is more conducive to image correction and also facilitates the design, manufacturing, and assembly of the projector.

[0115] Optionally, the distance between the first camera and the light outlet is equal to the distance between the second camera and the light outlet, that is, a symmetrical dual-camera layout.

[0116] Furthermore, the projection screen comprises a first part and a second part. The first part is located within the shooting range of the first camera, and the second part is located within the shooting range of the second camera. In one implementation, the first camera primarily captures the left side of the projection screen, and the second camera primarily captures the right side of the projection screen. By integrating the images captured by the first and second cameras, a complete image of the projection screen can be obtained. In this way, the two cameras selectively capture specific areas of the projection screen, thereby further improving the image clarity required for image correction, i.e., improving the clarity of the acquired feature points and reducing the impact of lens distortion.

[0117] Step 803: Based on the first captured image and the second captured image, correct the image to be projected on the projector.

[0118] There are several ways to correct the image to be projected on the projector host based on the first and second captured images in an ultra-short throw projection device. One of these methods will be introduced below.

[0119] In one implementation, the calibration chart has feature points. The process of the ultra-short throw projection device calibrating the image to be projected by the projector based on the first and second captured images includes: determining the positional relationship between the feature points on the calibration chart and the edge of the projection screen based on the first and second captured images, and calibrating the image to be projected based on the positional relationship.

[0120] When the calibration chart includes one chart, the first captured image includes a first image, and the second captured image includes a second image. The ultra-short-throw projection device detects feature points in the first and second images, as well as the edges of the projection screen, to obtain the positional relationship between the feature points on the chart and the edges of the projection screen. Based on this positional relationship, the ultra-short-throw projection device determines image calibration parameters and, based on these parameters, calibrates the image to be projected.

[0121] It should be understood that if the calibration chart includes a single chart, the ultra-short-throw projector cannot perform image calibration if it fails to capture all feature points on that chart and / or the edges of the projection screen, and may prompt the user to move the projector. Similarly, if the calibration chart includes multiple charts, the ultra-short-throw projector may also prompt the user to move the projector, i.e., adjust the position of the projector, if it fails to capture all feature points on the same chart and / or the edges of the projection screen.

[0122] When the calibration chart includes multiple charts, as mentioned above, different charts have different feature points. The first captured image includes multiple first images corresponding one-to-one with the multiple charts, and the second captured image includes multiple second images corresponding one-to-one with the multiple charts. Based on this, the process by which the ultra-short-throw projection device determines the positional relationship between feature points on the calibration chart and the edge of the projection screen based on the first and second captured images includes: detecting feature points in the multiple first and second images and the edge of the projection screen to determine at least one pair of images. Each pair of images includes a first image and a second image corresponding to the same chart, and each pair of images contains all feature points on the same chart and the edge of the projection screen. Based on the at least one pair of images, the ultra-short-throw projection device determines at least one set of positional relationships corresponding one-to-one with the at least one pair of images. Each set of positional relationships includes the positional relationship between all feature points on the corresponding chart and the edge of the projection screen.

[0123] It should be understood that if the multiple image cards are projected sequentially, and the ultra-short-throw projection device detects the feature points in the first and second images captured for each image card projected, and if the non-first image card is projected only after neither the first nor the second camera has captured all the feature points on the previous image card, then the ultra-short-throw projection device will ultimately determine a pair of images, and this pair of images refers to the last captured first and second images.

[0124] If the multiple image cards are projected sequentially, and the ultra-short-throw projector detects feature points in the first and second images captured after each image card is projected, but does not consider whether the first and second cameras captured all feature points on the previous image card before the first image card, the ultra-short-throw projector may ultimately determine more than one pair of images, or it may determine one pair of images.

[0125] If the multiple images are projected sequentially, but the ultra-short throw projector performs feature point detection on the multiple first images and multiple second images after projecting the multiple images, the ultra-short throw projector may ultimately determine more than one pair of images, or it may determine only one pair of images.

[0126] When an ultra-short-throw projection device determines a pair of images, it determines a set of positional relationships based on these images. This set of positional relationships includes the positional relationships between all feature points on the corresponding charts of the pair of images and the edges of the projection screen.

[0127] When an ultra-short-throw projection device determines multiple pairs of images, it determines multiple sets of positional relationships that correspond one-to-one with each pair of images. Each set of positional relationships includes the positional relationship between all feature points on the corresponding image card and the edge of the projection screen.

[0128] The process by which an ultra-short-throw projection device determines multiple sets of positional relationships corresponding one-to-one with these multiple pairs of images includes: determining the effective feature information in each pair of images, wherein the effective feature information includes information on all feature points on the corresponding image card and information on the edge of the projection screen. The ultra-short-throw projection device then determines a set of positional relationships corresponding to each pair of images based on the effective feature information in each pair of images.

[0129] In this embodiment, the process of determining effective feature information includes selecting the feature point with higher resolution from the same feature points included in each pair of images. Specifically, as mentioned above, there is a difference in the field of view between the first camera and the second camera. Based on this, the first feature point in the image captured by the camera with higher resolution is selected, and then the selected first feature point is used to determine the positional relationship between the first feature point and the edge of the projection screen. In other words, if both the first image and the second image corresponding to the same image card contain the first feature point, then the positional relationship between the first feature point and the edge of the projection screen is determined based on the first feature point with higher resolution among the two.

[0130] If the first camera captures the first feature point, but the second camera does not, the positional relationship between the first feature point and the edge of the projection screen is determined based on the first feature point captured by the first camera. In other words, if the first image corresponding to the same image card contains the first feature point, but the second image corresponding to the same image card does not, the positional relationship between the first feature point and the edge of the projection screen is determined based on the first feature point contained in the first image corresponding to the same image card.

[0131] The phrase "not including the first feature point" in this article can mean either that the first feature point was not captured, or that although the first feature point was captured, its resolution was less than a resolution threshold. The resolution threshold refers to the minimum resolution of the feature point used by the ultra-short throw projection device to determine the aforementioned positional relationship; feature points below the resolution threshold may be unusable.

[0132] After determining at least one set of positional relationships, the ultra-short-throw projection device determines at least one set of reference correction parameters that correspond one-to-one with the at least one set of positional relationships, determines image correction parameters based on the at least one set of reference correction parameters, and corrects the image to be projected on the projection host based on the image correction parameters.

[0133] The ultra-short throw projection device can average the at least one set of reference correction parameters to determine the image correction parameters. For example, each set of reference correction parameters includes four vertex compression ratios, and the average value of at least one top-left vertex compression ratio in the at least one set of reference correction parameters is determined as the top-left vertex compression ratio in the image correction parameters.

[0134] When image correction includes image compression, the aforementioned image correction parameters include image compression parameters, such as the compression ratio of vertices and edges of the image to be projected. When image correction includes image expansion, the aforementioned image correction parameters include image expansion parameters, such as the expansion ratio of vertices and edges of the image to be projected.

[0135] It should be understood that the positional relationship between a feature point and the edge of the projection screen represents the distance between the feature point and the edge of the projection screen. If a chart contains four feature points, each located near one of the four vertices of the chart, then it is desired that after projecting the chart, these four feature points will be located near the four vertices of the projection screen. If the positional relationship between these four feature points and the edge of the projection screen indicates that the top-left feature point is located outside the projection screen and far from the edge, then the top-left corner of the image to be projected can be compressed during image correction. Through similar processing, image correction, i.e., automatic image placement on the screen, can be achieved.

[0136] Figure 10 This is a flowchart of another image correction method provided in an embodiment of this application. Please refer to the following... Figure 10 The embodiments of this application will be described again by way of example.

[0137] When the projector moves, the projected image (i.e., the displayed image) becomes misaligned with the projection screen, resulting in a positional difference between the two. Ultra-short-throw projectors detect this misalignment using a camera or other sensor on the projector unit and automatically initiate an image correction process, or prompt the user to manually activate the correction.

[0138] The ultra-short-throw projector projects image card 1 onto the projection screen through its light output port. Taking image correction, including image compression, as an example, image card 1 is used to determine the boundaries of the projected image, i.e., whether the projected image covers the entire projection screen. Image card 1 is also used for feature point information acquisition. The ultra-short-throw projector controls cameras 1 and 2 to capture images of the projection screen projected with image card 1, obtaining two images. The ultra-short-throw projector detects whether the projected image in these two images covers the entire projection screen, i.e., determines the image range. If the detection result indicates that the projected image does not cover the entire projection screen, the ultra-short-throw projector prompts the user to adjust the position of the projector and restarts the image correction process. If the detection result indicates that the projected image covers the entire projection screen, the ultra-short-throw projector detects whether these two images contain all the feature points in image card 1, i.e., determines the validity of the feature information.

[0139] If both images contain all the feature points in Chart 1, the ultra-short-throw projection device determines the information of all feature points in Chart 1 based on these two images. Specifically, it identifies the feature points with higher resolution in the two images. In simpler terms, the parallax of the dual cameras is used to compensate for the resolution of the feature information, thereby improving the accuracy of the feature information. Specifically, if both cameras capture a feature point on Chart 1, the parallax of the dual cameras is used to compensate for the resolution of that feature point, i.e., the higher-resolution feature point is used to determine the positional relationship between that feature point and the projection screen. If only one camera captures a feature point on Chart 1, i.e., only one camera acquires valid feature information, the feature point acquired by that camera is used to determine the positional relationship between that feature point and the projection screen.

[0140] If neither of the two images contains all the feature points in image 1, the ultra-short throw projection device projects image 2, and controls camera 1 and camera 2 to respectively capture images of the projection screen on which image 2 is projected, and performs a validity determination of the feature information, until all feature points on the same image are collected, and then uses the parallax of the dual cameras to compensate for the resolution of the feature information.

[0141] Ultra-short-throw projection devices determine the positional relationship between each feature point on the same chart and the projection screen based on the collected feature points and the edges of the projection screen. Based on this positional relationship, image correction parameters are determined. The ultra-short-throw projection device then uses these image correction parameters to correct the projected image, i.e., adjust the displayed image, to complete the image correction.

[0142] If, after projecting N image cards, not all feature points on the same image card are captured, meaning the feature information is invalid or some feature point information is missing, this may be due to the influence of ambient light. In this case, the ultra-short throw projector will prompt the user to shield the ambient light and restart the image correction process.

[0143] In summary, in this embodiment, the ultra-short-throw projection device incorporates two cameras. Since these two cameras are located on opposite sides of the light outlet, they create a field-of-view difference. By using these two cameras to capture images of the projection screen displaying the calibration chart, the camera closer to a certain position on the projection screen captures a higher resolution image than the other. Compared to a single-camera setup, utilizing the field-of-view difference between the two cameras allows for the acquisition of higher-resolution information, which is beneficial for image correction and better mitigation of perspective and lens distortion. Furthermore, information that one camera cannot capture, such as portions of the projection screen obscured by ambient light, has a certain probability of being captured by the other camera. Therefore, using two cameras reduces the impact of ambient light and the difficulty of image correction.

[0144] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this application, and the embodiments of this application will not be described in detail one by one.

[0145] Figure 11 This is a schematic diagram of an image correction device 1100 provided in an embodiment of this application. The image correction device 1100 can be implemented as part or all of an ultra-short-throw projection device by software, hardware, or a combination of both. In this embodiment, the image correction device 1100 is applied to an ultra-short-throw projection device, which includes a projection screen and a projection host. Optionally, the image correction device 1100 is applied to the projection host. The projection host includes a first camera, a second camera, and a light output port, with the first and second cameras located on opposite sides of the light output port. Please refer to... Figure 11 The image correction device 1100 includes: a projection module 1101, a control module 1102, and a correction module 1103.

[0146] Projection module 1101 is used to project the calibration chart onto the projection screen through the light output port;

[0147] The control module 1102 is used to control the first camera and the second camera to capture images of the projection screen on which the calibration chart is projected, so as to obtain the first captured image and the second captured image.

[0148] The correction module 1103 is used to correct the image to be projected on the projector based on the first captured image and the second captured image.

[0149] Optionally, the calibration chart has feature points;

[0150] The calibration module 1103 includes:

[0151] The determination submodule is used to determine the positional relationship between feature points on the calibration chart and the edge of the projection screen based on the first and second captured images;

[0152] The correction submodule is used to correct the projected image based on positional relationships.

[0153] Optionally, the calibration chart includes multiple charts, and different charts have different feature points. The first captured image includes multiple first images that correspond one-to-one with the multiple charts, and the second captured image includes multiple second images that correspond one-to-one with the multiple charts.

[0154] The determination submodule is specifically used for:

[0155] Feature points in multiple first images and multiple second images, as well as the edges of the projection screen, are detected to determine at least one pair of images. Each pair of images includes a first image and a second image corresponding to the same image card, and each pair of images contains all feature points on the same image card and the edges of the projection screen.

[0156] The positional relationship is determined based on the at least one pair of images.

[0157] Optionally, if both the first image and the second image corresponding to the same image card contain the first feature point, then the positional relationship between the first feature point and the edge of the projection screen is determined based on the first feature point with higher resolution among the two first feature points, wherein the first feature point is a feature point in the same image card; and / or,

[0158] If a first image corresponding to the same image card contains a first feature point, but a second image corresponding to the same image card does not contain a first feature point, then the positional relationship between the first feature point and the edge of the projection screen is determined based on the first feature point contained in the first image corresponding to the same image card, where the first feature point is a feature point in the same image card.

[0159] Optionally, the multiple image cards are projected sequentially, and the non-first image card is projected when neither the first nor the second camera has captured all the feature points on the previous image card.

[0160] Optionally, the device 1100 further includes:

[0161] The first generation module is used to generate a first prompt message if, after projecting multiple image cards, not all feature points on the same image card and / or the edge of the projection screen are captured, the first prompt message is used to prompt the user to block the ambient light.

[0162] Optionally, image correction includes image compression;

[0163] The calibration is performed when the projected calibration chart is detected to cover the projection screen.

[0164] Optionally, the device 1100 further includes:

[0165] The detection module is used to detect whether the projected image of the calibration chart covers the projection screen in order to obtain the detection result;

[0166] The second generation module is used to generate a second prompt message if the detection result indicates that the projected image does not cover the projection screen, and the second prompt message instructs the user to adjust the position of the projector host.

[0167] Optionally, the distance between the first camera and the projection screen, and the distance between the second camera and the projection screen, shall not be less than the distance between the light outlet and the projection screen.

[0168] Optionally, the projection screen includes a first part and a second part, wherein the first part is located within the shooting range of the first camera and the second part is located within the shooting range of the second camera.

[0169] In summary, in this embodiment, the ultra-short-throw projection device incorporates two cameras. Since these two cameras are located on opposite sides of the light outlet, they create a field-of-view difference. By using these two cameras to capture images of the projection screen displaying the calibration chart, the camera closer to a certain position on the projection screen captures a higher resolution image than the other. Compared to a single-camera setup, utilizing the field-of-view difference between the two cameras allows for the acquisition of higher-resolution information, which is beneficial for image correction and better mitigation of perspective and lens distortion. Furthermore, information that one camera cannot capture, such as portions of the projection screen obscured by ambient light, has a certain probability of being captured by the other camera. Therefore, using two cameras reduces the impact of ambient light and the difficulty of image correction.

[0170] It should be noted that the image correction device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image correction device and the image correction method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0171] Figure 12 This is a structural block diagram of an ultra-short-throw projection device 1200 provided in an embodiment of this application. The ultra-short-throw projection device 1200 includes a projection host 1201 and a projection screen 1202. The projection host 1201 may include a processor 1203 and a memory 1204. The projection screen 1202 is used to display information such as graphics, text, icons, and videos.

[0172] Processor 1203 includes one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 1203 can be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1203 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1203 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1203 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0173] The memory 1204 may include one or more computer-readable storage media, which may be non-transitory. The memory 1204 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1204 are used to store at least one instruction, which is executed by the processor 1203 to implement the projection image correction method provided in the method embodiments of this application.

[0174] Those skilled in the art will understand that Figure 12 The structure shown does not constitute a limitation on the ultra-short throw projection device 1200, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0175] In some embodiments, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of the image correction method described above. For example, the computer-readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0176] It is worth noting that the computer-readable storage medium mentioned in the embodiments of this application can be a non-volatile storage medium, in other words, it can be a non-transient storage medium.

[0177] It should be understood that all or part of the steps of the above embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially in the form of a computer program product. The computer program product includes one or more computer instructions. The computer instructions can be stored in the above-described computer-readable storage medium.

[0178] That is, in some embodiments, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the steps of the image correction method described above.

[0179] It should be understood that "at least one" as mentioned herein refers to one or more, and "multiple" refers to two or more. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, in order to clearly describe the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and the terms "first," "second," etc., are not necessarily different.

[0180] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the projected images and videos involved in the embodiments of this application were obtained under full authorization.

[0181] The above descriptions are embodiments provided in this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image correction method, characterized in that, The invention is applied to an ultra-short-throw projection device, which includes a projection screen and a projection host. The projection host includes a first camera, a second camera, and a light outlet, with the first camera and the second camera located on opposite sides of the light outlet. The method includes: The calibration chart is projected onto the projection screen through the light outlet. The calibration chart has feature points, and the calibration chart includes multiple charts, with different charts having different feature points. The first camera and the second camera are controlled to capture images of the projection screen on which the calibration chart is projected, so as to obtain a first captured image and a second captured image. The first captured image includes multiple first images that correspond one-to-one with the multiple charts, and the second captured image includes multiple second images that correspond one-to-one with the multiple charts. Feature points in the plurality of first images and the plurality of second images, as well as the edges of the projection screen, are detected to determine at least one pair of images. Each pair of images includes a first image and a second image corresponding to the same image card, and each pair of images contains all feature points on the same image card and the edges of the projection screen. Based on the at least one pair of images, determine the positional relationship between feature points on the calibration chart and the edge of the projection screen; Based on the aforementioned positional relationship, the image to be projected by the projection host is corrected.

2. The method according to claim 1, characterized in that, If both the first image and the second image corresponding to the same image card contain the first feature point, then the positional relationship between the first feature point and the edge of the projection screen is determined based on the first feature point with higher resolution among the two first feature points. The first feature point is the feature point in the same image card. And / or, If the first image corresponding to the same image card contains the first feature point, but the second image corresponding to the same image card does not contain the first feature point, then the positional relationship between the first feature point and the edge of the projection screen is determined based on the first feature point contained in the first image corresponding to the same image card, wherein the first feature point is a feature point in the same image card.

3. The method according to claim 1, characterized in that, The multiple image cards are projected sequentially, and the non-first image card is projected when neither the first camera nor the second camera has captured all the feature points on the previous image card.

4. The method according to claim 1, characterized in that, The method further includes: If, after projecting the multiple image cards, not all feature points on the same image card and / or the edges of the projection screen are captured, a first prompt message is generated to prompt the user to block ambient light.

5. The method according to any one of claims 1-4, characterized in that, The image correction includes image compression; The correction is performed when the projected correction chart is detected to cover the projection screen.

6. The method according to claim 5, characterized in that, The method further includes: The detection results are obtained by detecting whether the projected image of the calibration chart covers the projection screen. If the detection result indicates that the projected image does not cover the projection screen, a second prompt message is generated, which instructs the user to adjust the position of the projection host.

7. The method according to any one of claims 1-4, characterized in that, The projection screen includes a first part and a second part, wherein the first part is located within the shooting range of the first camera and the second part is located within the shooting range of the second camera.

8. An ultra-short-throw projection device, characterized in that, The ultra-short-throw projection device includes a projection screen and a projection host. The projection host includes a first camera, a second camera, and a light outlet. The first camera and the second camera are located on opposite sides of the light outlet. The ultra-short-throw projection device is used to implement the steps of the method according to any one of claims 1-7.

Images