Projection device and projection image correction method

Abstract

Some embodiments of the present application provide a projection device and a projection picture correction method. After receiving a projection picture correction instruction, the method can control the light emitting assembly to project a correction image of a pure color chart and a feature chart. A first sampling image of the pure color chart and a second sampling image of the feature chart are obtained by a camera. A feature contour region is determined according to the first sampling image, and a feature point is extracted in the second sampling image according to the feature contour region. An included angle between the projection surface and the light emitting assembly is calculated based on the feature point extracted from the second sampling image, and the light emitting assembly is controlled to project the projection content to the projection surface according to the included angle, so as to ensure that when the projection surface is blocked by an obstacle, the projection device can extract the feature point according to the feature contour region after obstacle avoidance, and project a corrected projection image on the projection surface, thereby improving the user experience when using the projection device.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to a projection device and a projection image correction method. Background Technology

[0002] A projection device is a display device that projects images or videos onto a screen. It uses laser light of a specific color, refracted by optical lens components, to project a concrete image onto the screen. During projection, a certain distance must be maintained between the projection device and the screen so that the image formed on the screen conforms to the focal length range of the optical lens components, resulting in a clear image.

[0003] Due to the complexity of the environment, it is inevitable that the light-emitting components of the projection device will not be perpendicular to the projection wall or screen, resulting in a trapezoidal projected image. Additionally, obstacles in the projection area may obstruct the projected image, significantly impacting the user experience. Therefore, projection devices need to automatically detect obstacles during projection to avoid them, and use automatic correction functions to recalibrate the projected image, thereby improving the user experience.

[0004] The automatic correction function utilizes a projection feature map card without determining the spatial location of the feature points. It directly identifies the feature points by taking a picture with a camera and then uses 3D reconstruction to fit the projection surface for automatic correction. However, if there is more than one projection wall, or if some feature points are projected onto other objects instead of a plane, the projection will be over multiple areas. If feature points are identified in multiple areas, errors will occur in the fitted plane, causing the automatically corrected shape to no longer be a rectangle, resulting in an incorrect correction. Summary of the Invention

[0005] Some embodiments of this application provide a projection image correction method to solve the problem that when there are obstacles on the projection surface, the projection device will generate errors in the fitting plane when it identifies feature points in multiple areas, resulting in deviations in the shape after automatic correction.

[0006] On one hand, some embodiments of this application provide a projection device, including: a light-emitting component, a camera, and a controller. The light-emitting component is configured to project projection content onto a projection surface; the camera is configured to capture sampled images; and the controller is configured to:

[0007] In response to a projection image correction command, the light-emitting component is controlled to project a correction image, which includes a solid color image card and a feature image card;

[0008] Acquire a first sampled image obtained by the camera from the solid color image card, and a second sampled image obtained by the camera from the feature image card;

[0009] A feature contour region is determined based on the first sampled image, wherein the feature contour region is the contour region with the largest area in the first sampled image or a contour region specified by the user.

[0010] Feature points are extracted from the second sampled image according to the feature contour region;

[0011] The angle between the projection surface and the light-emitting component is calculated based on the feature points, and the light-emitting component is controlled to project the content onto the projection surface according to the angle.

[0012] On the other hand, some embodiments of this application also provide a projection image correction method applied to a projection device, the projection device including a light-emitting component, a camera, and a controller; the projection image correction method includes:

[0013] In response to a projection image correction command, the light-emitting component is controlled to project a correction image, which includes a solid color image card and a feature image card;

[0014] Acquire a first sampled image obtained by the camera from the solid color image card, and a second sampled image obtained by the camera from the feature image card;

[0015] A feature contour region is determined based on the first sampled image, wherein the feature contour region is the contour region with the largest area in the first sampled image or a contour region specified by the user.

[0016] Feature points are extracted from the second sampled image according to the feature contour region;

[0017] The angle between the projection surface and the light-emitting component is calculated based on the feature points, and the light-emitting component is controlled to project the content onto the projection surface according to the angle.

[0018] As can be seen from the above solutions, some embodiments of this application provide a projection device and projection image correction method that, upon receiving a projection image correction command, controls the light-emitting component to project corrected images of the initial color map card and the feature map card. It also acquires a first sampled image of the solid color map card and a second sampled image of the feature map card taken by the camera. A feature contour region is determined based on the first sampled image, and then feature points are extracted from the second sampled image according to the feature contour region. The angle between the projection surface and the light-emitting component is calculated based on the feature points extracted from the second sampled image. The light-emitting component projects the projection content onto the projection surface according to the angle, ensuring that when the projection surface is obstructed by an obstacle, the projection device can extract feature points from the feature contour region after obstacle avoidance and project a corrected projection image onto the projection surface. Attached Figure Description

[0019] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the projection state of the projection device in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of the projection device structure in an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of the optical engine architecture of the projection device in the embodiments of this application;

[0023] Figure 4 This is a schematic diagram of the optical path of the projection device in the embodiments of this application;

[0024] Figure 5 This is a schematic diagram of the system framework of the projection device in the embodiments of this application;

[0025] Figure 6 This is a schematic diagram of the imaging of the projection device when the projection surface is tilted in an embodiment of this application;

[0026] Figure 7 This is a schematic diagram of the imaging process when there is an obstacle between the projection device and the projection surface in an embodiment of this application;

[0027] Figure 8 This is a schematic diagram of the first sampled image captured by the camera in an embodiment of this application;

[0028] Figure 9 This is a schematic diagram of the second sampled image captured by the camera in an embodiment of this application;

[0029] Figure 10 This is a schematic diagram of the extraction of candidate contour regions in the second sampled image in an embodiment of this application;

[0030] Figure 11 This is a projection diagram of a light-emitting component with an uneven projection surface in an embodiment of this application;

[0031] Figure 12 This is a schematic diagram of coordinate system transformation between the camera and the light-emitting component in an embodiment of this application;

[0032] Figure 13 This is a schematic diagram illustrating the process of defining the target projection area based on the projected image in an embodiment of this application;

[0033] Figure 14 This is a schematic diagram of the process of delineating the target projection area according to the obstacle avoidance command in an embodiment of this application;

[0034] Figure 15 This is a flowchart illustrating a projection image correction method provided in an embodiment of this application. Detailed Implementation

[0035] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0036] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0037] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0038] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0039] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0040] The embodiments of this application can be applied to various types of projection devices. The following description will use a projector as an example to illustrate the projection device and the automatic focusing method.

[0041] A projector is a device that projects images or videos onto a screen. Projectors can connect to computers, cable TV networks, the internet, VCD (Video Compact Disc), DVD (Digital Versatile Disc Recordable), game consoles, DV camcorders, and other devices via various interfaces to play corresponding video signals. Projectors are widely used in homes, offices, schools, and entertainment venues.

[0042] Figure 1 A schematic diagram of the placement of a projection device according to an embodiment of this application is shown. Figure 2 A schematic diagram of the optical path of a projection device according to an embodiment of this application is shown.

[0043] In some embodiments, reference Figure 1-2 This application provides a projection device including a projection screen 1 and a projection device 2. The projection screen 1 is fixed in a first position, and the projection device 2 is placed in a second position so that the image projected by the device matches the projection screen 1. The projection device includes a laser light source 100, an optical engine 200, a lens 300, and a projection surface 400. The laser light source 100 provides illumination for the optical engine 200, which modulates the light beam and outputs it to the lens 300 for imaging, projecting it onto the projection surface 400 to form a projected image. Since the laser light source 100, the optical engine 200, and the lens 300 are used together to emit projection light to project the image, in some embodiments of this application, the laser light source 100, the optical engine 200, and the lens 300 are collectively referred to as the light-emitting assembly.

[0044] In some embodiments, the laser source 100 of the projection device includes a laser assembly 110 and an optical lens assembly 120. The light beam emitted by the laser assembly 110 can pass through the optical lens assembly 120 to provide illumination for the optical engine 200. For example, the optical lens assembly 120 requires a high level of environmental cleanliness and airtightness; while the chamber in which the laser assembly is installed can be sealed with a lower level of dustproof sealing to reduce sealing costs.

[0045] In some embodiments, the optical engine 200 of the projection device may include a blue optical engine, a green optical engine, and a red optical engine, and may also include a heat dissipation system, a circuit control system, etc. It should be noted that in some embodiments, the light-emitting component of the projector may also be implemented using an LED light source.

[0046] Figure 3 A schematic diagram of the circuit architecture of a projection device according to an embodiment of this application is shown. In some embodiments, the projection device may include a display control circuit 10, a laser light source 20, at least one laser driving component 30, and at least one brightness sensor 40. The laser light source 20 may include at least one laser corresponding to at least one laser driving component 30. Here, "at least one" refers to one or more, and "more than one" refers to two or more.

[0047] Based on this circuit architecture, the projection device can achieve adaptive adjustment. For example, by setting a brightness sensor 40 in the light output path of the laser light source 20, the brightness sensor 40 can detect the first brightness value of the laser light source and send the first brightness value to the display control circuit 10.

[0048] The display control circuit 10 can acquire the second brightness value corresponding to the driving current of each laser, and determine that the laser has a COD fault when the difference between the second brightness value and the first brightness value of the laser is greater than the difference threshold. Then the display control circuit can adjust the current control signal of the corresponding laser driving component until the difference is less than or equal to the difference threshold, thereby eliminating the COD fault of the blue laser. The projection device can eliminate the COD fault of the laser in a timely manner, reduce the damage rate of the laser, and improve the image display effect of the projection device.

[0049] Figure 4 A schematic diagram of the structure of a projection device according to an embodiment of this application is shown.

[0050] In some embodiments, the laser light source 20 in the projection device may include independently configured blue laser 201, red laser 202 and green laser 203. The projection device may also be called a three-color projection device. The blue laser 201, red laser 202 and green laser 203 are all modular lightweight (Mirai Console Loader, MCL) packaged lasers, which are small in size and facilitate compact arrangement of the optical path.

[0051] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM (random access memory), ROM (read-only memory), a first to an nth interface for input / output, a communication bus, etc.

[0052] In some embodiments, the projection device may be configured with a camera for working in conjunction with the projection device to achieve adjustment and control of the projection process. For example, the camera configured with the projection device may be specifically implemented as a 3D camera or a binocular camera; when the camera is implemented as a binocular camera, it specifically includes a left camera and a right camera; the binocular camera can acquire the image and playback content presented on the screen corresponding to the projection device, i.e., the projection surface, which is projected by the optical engine 200 built into the projection device.

[0053] When the projection device moves, its projection angle and distance to the projection surface change, which will cause the projected image to be distorted. The projected image will be displayed as a trapezoidal image or other distorted image. The projection device controller 500 can automatically correct the trapezoidal shape based on the image captured by the camera 600 by coupling the angle between the optical engine projection surfaces and the correct display of the projected image.

[0054] Figure 5 A schematic diagram of the system framework for display control of a projection device according to an embodiment of this application is shown.

[0055] In some embodiments, the projection device has the characteristics of a long-throw micro-projector, and its controller can control the display of the projected light image through a preset algorithm to achieve functions such as automatic keystone correction, automatic screen entry, automatic obstacle avoidance, automatic focus adjustment, and eye protection.

[0056] In some embodiments, the projection device is equipped with a gyroscope sensor; during the movement of the device, the gyroscope sensor can sense the position movement and actively collect movement data; then the collected data is sent to the application service layer through the system framework layer to support the application data required during user interface interaction and application interaction. The collected data can also be used by the controller for data calls in the algorithm service implementation.

[0057] In some embodiments, the projection device is equipped with a time-of-flight sensor. After the time-of-flight sensor collects the corresponding data, the data will be sent to the time-of-flight service corresponding to the service layer. After the time-of-flight service obtains the data, it will send the collected data to the application service layer through a process communication framework. The data will be used for data calls, user interfaces, program applications, and other interactive applications of the controller.

[0058] In some embodiments, the projection device is configured with a camera for acquiring images, which may be a binocular camera, a depth camera, or a 3D camera, etc. The camera acquisition data is sent to a camera service, and then the camera service sends the acquired image data to a process communication framework and / or a projection device correction service. The projection device correction service can receive the camera acquisition data sent by the camera service, and the controller can call the corresponding control algorithm in the algorithm library for different functions to be implemented.

[0059] In some embodiments, data interaction is performed with the application service through a process communication framework, and the calculation results are then fed back to the correction service through the process communication framework. The correction service sends the obtained calculation results to the projection device operating system to generate control signaling, and sends the control signaling to the light output component control driver to control the operating conditions of the light output component and realize automatic correction of the displayed image.

[0060] In some embodiments, the projection device uses an autofocus algorithm and its configured laser rangefinder to obtain the current object distance, calculate the initial focal length and search range, and then drives the camera to take pictures and uses the corresponding algorithm to evaluate the sharpness.

[0061] Within the aforementioned search range, the projection device uses a search algorithm to find the optimal focal length, then repeats the steps of taking photos and evaluating sharpness. Finally, it finds the optimal focal length through sharpness comparison and completes automatic focusing.

[0062] For example, after the projection device is turned on, the user moves the device; the projection device automatically completes the correction and refocuses, and the controller will detect whether the autofocus function is enabled; when the autofocus function is not enabled, the controller will end the autofocus operation; when the autofocus function is enabled, the projection device will obtain the detection distance of the time-of-flight sensor through the middleware for calculation.

[0063] The controller queries a preset mapping table based on the acquired distance to obtain the focal length of the projection device; then the middleware sets the acquired focal length to the light-emitting component of the projection device; after the light-emitting component emits laser light at the aforementioned focal length, the camera executes the image capture command; the controller determines whether the projection device has completed focusing based on the acquired image and evaluation function.

[0064] If the judgment result meets the preset completion conditions, the automatic focus adjustment process ends; if the judgment result does not meet the preset completion conditions, the middleware will fine-tune the focal length parameters of the light output component of the projection device. For example, the focal length can be finely adjusted gradually by preset step size, and the adjusted focal length parameters are set back to the light output component. This achieves repeated photo taking and sharpness evaluation steps, and finally finds the optimal focal length through sharpness comparison to complete the automatic focus adjustment.

[0065] like Figure 6 and Figure 7 As shown, during the process of users projecting content using projection devices, due to the complexity of the environment, it is inevitable that the light-emitting components of the projection device will not be perpendicular to the projection wall or screen, causing the projected image to be distorted, and there will be obstacles in the projection area that will block the projected image, which will greatly affect the user experience.

[0066] To address this, projection devices are typically designed with automatic obstacle avoidance and automatic correction functions. These devices need to automatically detect obstacles during projection, avoid projecting onto them, and then use automatic correction to recalibrate the projected image. The automatic correction function works by using a camera to capture and identify feature points directly from a 600-degree image of the feature map card, without determining the spatial location of the feature points, and then reconstructing and fitting the 3D model to the projection surface to achieve automatic correction.

[0067] However, if there is more than one projection wall, or if some feature points are projected onto other objects instead of a plane, the projection will be projected onto multiple areas. If feature points are identified in multiple areas, a fitting plane error will occur, causing the automatically corrected shape to no longer be a rectangle, resulting in an incorrect correction.

[0068] To ensure that a corrected projected image can still be projected even when the projection surface 400 is obstructed by an obstacle during the projection process, some embodiments of this application also provide a projection device, which may include a light-emitting component, a camera 600, and a controller 500. The light-emitting component is used to project the projection content onto the projection surface 400. The camera 600 is used to capture sampled images. The controller 500 is configured to:

[0069] S100: In response to the projection image correction command, control the light-emitting component to project the corrected image.

[0070] The projection device can receive projection image correction commands input by the user and control the light-emitting component to project the corrected image according to the projection image correction commands. The projection image correction commands can be issued by the user through buttons on the control device of the projection device (remote control, etc.) or through a mobile terminal (smartphone, laptop, etc.) that has established a communication connection with the projection device.

[0071] The corrected image includes a solid color image card and a feature image card. When the corrected image is a solid color image card, the projection device projects a solid color image onto the projection surface 400 through the light-emitting component. When the solid color image card is projected onto the projection surface 400 by the light-emitting component, some of the light from the solid color image card will illuminate obstacles located between the light-emitting component and the projection surface 400. A shadow area will appear on the corresponding position of the projection surface 400 due to the obstruction by the obstacle. The controller 500 can identify the location of the obstacle by recognizing the shadow area in the solid color image card and perform subsequent obstacle avoidance functions based on the obstacle's location.

[0072] It should be noted that in some embodiments of this application, in order to more clearly identify the shadow areas, the solid color chart should be a light color, such as light yellow, light blue, white, gray, etc. In this embodiment, a light color is defined as a color with a depth less than or equal to 1 / 12 of the standard dye color depth. This application does not impose any other restrictions on the color of the solid color chart.

[0073] When the corrected image is a feature map, the projection device projects a projected image with several feature points onto the projection surface 400 through the light-emitting component. These feature points represent characteristics of a preset area of ​​the projected image and are used for subsequent automatic correction of the projected image.

[0074] S200: Acquire a first sampled image obtained by the camera from the solid color image card, and a second sampled image obtained by the camera from the feature image card.

[0075] The controller 500 can also control the camera 600 to capture images of the solid color image card and the feature image card to obtain a first sampled image captured by the solid color image card and a second sampled image captured by the feature image card. In some embodiments, the camera 600 of the projection device can be built-in or externally mounted. The camera 600 can capture images of the projected image projected by the projection device through the light-emitting component to obtain sampled images. After the projection device captures an image with the camera 600, it can also perform a sharpness detection on the sampled image captured by the camera 600 to determine whether the focal length of the projection device is appropriate. If the sharpness of the detected sampled image is low, the focal length of the projection device is adjusted, and the sampled image is recaptured by the camera 600. The sharpness of the sampled images is compared according to the capture time sequence to determine the focal length parameter of the projection device when the sharpness of the sampled image is the highest.

[0076] In some embodiments of this application, during the process of determining the focal length parameters of the projection device, the camera can be switched to continuous shooting mode while adjusting the focal length parameters, and a removable watermark can be added to the captured sampled images. The watermark content is the focal length parameter. By comparing the sharpness of all sampled images, the focal length parameters of the projection device that achieve the highest sharpness of the sampled images are determined.

[0077] For example, the first sampled image acquired by the controller 500 is as follows: Figure 8 As shown, the first sampled image contains a rectangular obstacle 1 and a circular obstacle 2.

[0078] S300: Determine the feature contour region based on the first sampled image.

[0079] The feature contour region is the largest contour region in the first sampled image or a contour region specified by the user. The controller 500 can determine the feature contour region based on the positions of obstacle 1 and obstacle 2 in the first sampled image to ensure that the projected image projected by the projection device avoids the obstacles 1 and 2, so that the user can see a projected image that is not obstructed by the obstacles.

[0080] In some embodiments of this application, the feature contour region can be the largest contour region in the first sampled image or a contour region specified by the user. The controller 500 can determine which of the above contour regions is selected as the feature contour region based on the setting state of the obstacle avoidance switch of the projection device. Because obstacles existing between the light-emitting component and the projection surface 400 will form shadow areas on the projected image, and the shadow areas will split the original projected image into multiple contour regions without shadow areas. At this time, the controller 500 can extract all contour regions in the first sampled area and detect the setting state of the obstacle avoidance switch of the projection device. The setting state of the obstacle avoidance switch includes on and off. The setting state can be manually set by the user using the projection device to be on or off, or it can be automatically switched to the on state when the projection device detects the presence of obstacles in the projected image.

[0081] When the projection device is in obstacle avoidance mode, the projected image needs to avoid obstacles during projection. If the setting is enabled, the controller 500 will iterate through the area of ​​each contour region in the first sampled image and filter out candidate contour regions. These candidate contour regions are then specified by the user as feature contour regions. During the filtering process, in addition to the contour region, other parameters can be set for filtering. For example, to ensure a clearer view of the projected image, the controller 500 can set the distance between the contour region and the user as a filtering parameter. Or, if the contour region is an irregular shape, and the projection needs to be projected onto a rectangular area, the controller 500 will also need to set the maximum rectangular area within each contour region and use the maximum rectangular area as the filtering parameter.

[0082] Images are composed of pixels. In some embodiments of this application, the controller 500 can also delineate contour regions based on the pixel color values ​​in the first sampled image. Since the solid color image is light-colored, while the shadow area after being occluded by an obstacle is usually black, there is a significant color difference between the two. Therefore, the controller 500 can also set a color difference threshold, iterate through the pixel color values ​​in the first sampled image, obtain pixels whose adjacent pixel color difference values ​​are greater than or equal to the color difference threshold, and identify boundary patterns based on these adjacent pixel color difference values. The boundary pattern is the shape of the obstacle between the light-emitting component and the projection surface 400.

[0083] After recognizing the boundary pattern, the controller 500 delineates the contour region based on the edges of the first sampled image. Thus, within the shadow region formed by obstacle occlusion, all pixels are black, and the color difference between adjacent pixels is small and will not reach the color difference threshold. However, the pixels in the contour part of the shadow region are black, while the pixels outside the contour are the solid color of the image card, which has a large color difference from black. Therefore, the shadow region formed by obstacle occlusion can be accurately identified, and the contour region can be precisely delineated.

[0084] In some embodiments of this application, the controller 500 may also input the first sampled image captured by the camera 600 into the recognition model. The recognition model is a neural network model trained based on the sample images. The recognition model can be trained to convergence using a large number of sampled images with obstacles and sampled images without obstacles. After the controller 500 inputs the first sampled image into the recognition model, it will obtain the recognition result output by the recognition model. The recognition result is the classification probability that the first sampled image contains an obstacle target. The classification probability can be used to determine whether the first sampled image contains an obstacle target. If the recognition result indicates that an obstacle target is contained, it means that there is an obstacle between the light-emitting component and the projection surface 400. The controller 500 will remove the contour region corresponding to the obstacle target from the first sampled image and determine the feature contour region in the first sampled image after removing the obstacle target. If the recognition result indicates that no obstacle target is contained, it means that there is no obstacle between the light-emitting component and the projection surface 400. The controller 500 will determine the feature contour region based on the first sampled image.

[0085] In some embodiments of this application, if the obstacle avoidance switch is set to off, in order to enable the user to see the projected content in the projected image while avoiding obstacles, the controller 500 will filter out the contour region with the largest area as the feature contour region after traversing the area of ​​each contour region in the first sampled image.

[0086] In some embodiments of this application, the parameters for filtering contour regions can also be prioritized. For example, the area of ​​each contour region can be selected as the first filtering parameter, and if the contour regions are the same, the distance between the contour region and the user can be selected as the second filtering parameter. The above is only an illustrative description of this embodiment. When setting priorities, the filtering parameters can be interchanged, and some embodiments of this application do not impose specific limitations on this.

[0087] In some embodiments of this application, the controller 500 may set a parameter threshold based on the filtering parameters before filtering the contour region. For example, when the area of ​​the contour region is used as the filtering parameter, an area parameter threshold can be set. If the area of ​​the contour region is greater than the parameter threshold, then the contour region meets the filtering conditions and can be used as a candidate contour region; if the area of ​​the contour region is less than the parameter threshold, it means that the area of ​​the contour region is too small, and the user cannot see the projected content in the projected image clearly at a specified distance, thus it does not meet the filtering conditions and cannot be used as a candidate contour region.

[0088] Before filtering candidate contour regions, the controller 500 can generate a list of contour regions based on the area of ​​each contour region in the first sampled image. If a contour region in the list does not meet the filtering criteria, the controller 500 will remove the corresponding contour region from the list. After filtering all contour regions, the list of contour regions will contain only candidate contour regions that meet the filtering criteria. This list of candidate contour regions is then available for the user to specify as feature contour regions.

[0089] Before filtering the contour regions, the controller 500 can also select three candidate contour regions based on the number of regions in the candidate contour region list, for example, setting the number of regions in the candidate contour region list to three. In this case, the controller 500 will traverse the area of ​​each contour region in the first sampled image, filter out three candidate contour regions according to the size of the contour region area, and arrange them in the candidate contour region list in a default or user-specified order.

[0090] S400: Extract feature points in the second sampled image according to the feature contour region.

[0091] In some embodiments of this application, after traversing the area of ​​each contour in the first sampled image and filtering out candidate wheel library regions, the controller 500 further controls the light-emitting component to project a feature map onto the projection surface 400, and after the light-emitting component projects the feature map, controls the camera 600 to capture the feature map to obtain a second sampled image. Figure 9 A second sampled image is shown in an embodiment of this application. The controller 500 can then identify feature points within the same contour region in the second sampled image based on candidate contour regions selected from the first sampled image, and calculate the average depth of the feature points located within the same candidate contour region in the second sampled image. Feature points refer to points where the image grayscale value changes drastically or points with large curvature at the image edge, i.e., the intersection of two edges. Figure 9 The ring-shaped rectangle in the image represents the feature points on the feature map.

[0092] Methods for calculating feature points can include geometric triangulation, inverse depth, particle filtering, etc. During the calculation of the average depth of feature points located within the same candidate contour region in the second sampled image, the controller 500 can also iterate through the color values ​​of feature points within the candidate contour region. The color of the feature point contour should have a significant color difference from the solid color chart to clearly show the feature point contour and its position. In some embodiments of this application, to distinguish it from the shadows cast by obstacles on the projection surface, the color of the feature points can be any dark color other than black, and the color of the feature points does not need to be uniform. In this embodiment, a dark color is defined as a color with a depth greater than 1 / 12 of the standard dye color depth. This application does not impose other restrictions on the color of the feature points.

[0093] Figure 10 The outlines of two candidate regions in the second sampled image are shown. Figure 10 In the second sampled image, the areas indicated by the two dashed lines are the candidate region contours. The candidate contour region to the left of obstacle 2 is defined as the first candidate contour region, and the candidate contour region to the right of obstacle 2 is defined as the second candidate contour region. When calculating the average depth of feature points, the controller 500 calculates based on feature points within the same candidate contour region. For the first candidate contour region, the controller 500 calculates the average depth based on 9 feature points within the first candidate contour region; for the second candidate contour region, the controller 500 calculates the average depth based on 3 feature points within the second candidate contour region.

[0094] In some embodiments of this application, when the controller 500 calculates the average depth of feature points located within the same candidate contour region in the second sampled image, it can also iterate through the color values ​​of the feature points within the candidate contour region. When the controller 500 controls the light-emitting component to project the feature map, the position of the feature points can be displayed because the color values ​​of the feature points are significantly different from the color values ​​in the solid color map.

[0095] Due to the complexity of the environment, various situations may arise when a projection device projects an image onto the projection surface 400 via its light-emitting components. If the projection surface 400 is a wall, the same image may be projected onto two walls at different distances. In this case, the image projected onto the wall closer to the projection device will appear larger than the image projected onto the wall farther away, resulting in inconsistent image sizes and distorted projection shapes. If the projection surface 400 is a projection screen, issues such as image distortion may occur due to the placement of the projection device. For example, inconsistent vertical positioning of the projection device or placement that is too far to the left or right may cause the projected image to appear trapezoidal.

[0096] To this end, the controller 500 can also extract the projected shape based on the color values ​​of multiple feature points. In this process, to extract the projected shape more quickly, the controller 500 can acquire boundary feature points in the contour of the candidate region. These boundary feature points are feature points that can represent the shape of the candidate contour region. For example, when the candidate contour region is rectangular, four feature points corresponding to the four corners of the rectangle can be extracted, and the projected shape can be extracted based on these four feature points. Alternatively, two feature points corresponding to the two opposite corners of the rectangle can be extracted, and the projected shape can be extracted based on these two feature points.

[0097] Because the projection area of ​​most projection devices is rectangular, when there are obstacles between the light-emitting component and the projection surface 400, the shadow area formed by the obstacles in the projected image will divide the projected image into irregular shapes. To address this, the controller 500 can further define the largest rectangular area within the candidate contour areas after filtering out candidate contour areas, and use the largest rectangular area of ​​the candidate contour areas as the effective projection area within the candidate contour areas.

[0098] After the controller 500 extracts the projected shape, it can also acquire the hardware parameters of the camera 600 and the light-emitting component. The camera 600's hardware parameters include ISO sensitivity, white balance, metering, focus, exposure compensation, and focal length. Before setting the ISO sensitivity of the camera 600, the controller 500 can also acquire the ambient light level from the camera and adjust the ISO sensitivity based on the light intensity. For better viewing, the ambient light around the projection device is usually low during projection. Therefore, the camera 600's ISO sensitivity needs to be adjusted to a higher value.

[0099] The focus modes include single-point autofocus (AF-S), servo autofocus (AF-C), intelligent autofocus (AF-A), and manual focus.

[0100] When setting exposure compensation, the Camera 600 can also detect the brightness of the photo. When the photo brightness is moderate, simply keep it at "0"; when the photo is too dark, increase the exposure compensation; when the photo is too bright, decrease the exposure compensation.

[0101] When setting the focal length on a 600mm camera, to capture a wider field of view, simply adjust the lens zoom ring to its minimum; to capture distant scenes, simply zoom in. Additionally, wider focal lengths produce photos with greater depth of field, while longer focal lengths produce photos with shallower depth of field.

[0102] The hardware parameters of the light-emitting component include resolution, projection brightness, contrast ratio, focus method, and display ratio.

[0103] In some embodiments of this application, while acquiring the hardware parameters of the camera 600 and the light-emitting component, the controller 500 can also acquire the standard shape on the feature map in the second sampled image. When the light-emitting component projects the feature map onto the neat projection surface 400 vertically, the projected shape is the same as the standard shape. At this time, the controller 500 can calculate the distance from the light-emitting component to each feature point based on the feature points in the candidate contour region, which is the depth of the feature point.

[0104] However, for the uneven projection surface 400, the feature map card will be projected onto planes at different distances, and correspondingly, the feature points on the feature map card will be located on different planes. For feature points within the same candidate contour region, since the distances between each plane in the projection surface 400 and the optical engine 200 are different, the depth of the feature points will also be different. The controller 500 will calculate the distances between the multiple feature points and the optical engine 200 based on the projection shape extracted from the color values ​​of multiple feature points, the standard shape on the feature map card in the second sampled image, and the hardware parameters of the camera 600 and the light-emitting assembly. It will then calculate the average of these distances to obtain the average depth.

[0105] During the calculation, the controller 500 can calculate the average depth of feature points on the same plane within the projection surface 400. Before the calculation, the controller 500 can also obtain the number of planes contained in the projection surface 400 by recognizing the feature contours of the projection surface 400. For example... Figure 11 As shown, the projection device projects the feature map card onto the projection surface 400 through the optical engine 200. There are two protruding walls in the projection surface 400, which divide the projection surface 400 into 5 planes. Three of the planes are at the same distance from the light-emitting component, and the other two planes are closer to the light-emitting component.

[0106] For the five planes mentioned above, the controller 500 can determine which planes each candidate contour region contains, then calculate the depth of feature points within those planes, and calculate the average depth of the feature points within that plane based on the depth of the feature points. After calculating the average depth of all planes contained within the candidate contour region, the controller then calculates the average depth of the feature points within the candidate contour region based on the average depth of the planes.

[0107] In some embodiments of this application, if one of the planes in the projection surface 400 has a small area and no feature points in the plane, the controller 500 can divide the area of ​​that plane into adjacent planes to ensure that each area in the projection surface 400 is calculated.

[0108] After calculating the average depth of the feature points, for each candidate region contour, the controller 500 can also calculate the area of ​​each candidate region contour and the area of ​​the second sampled image, and calculate the area ratio of the candidate contour region relative to the second sampled image based on the area of ​​the candidate region contour and the area of ​​the second sampled image. The area ratios can be sorted from largest to smallest. After calculating the average depth of the feature points within the candidate contour region and the area ratio of the candidate contour region relative to the second sampled image, the controller 500 can also generate a first prompt message based on the above two data points, and control the light-emitting component to project the first prompt message when the obstacle avoidance switch is enabled. The first prompt message includes each candidate contour region, and the average depth and area ratio of the feature points corresponding to each candidate contour region.

[0109] The user can view the selectable candidate contour areas through the first prompt information projected onto the projection surface 400. The user can then select one of these candidate contour areas as the feature contour area. Based on the first prompt information, the user can generate a selection command using the buttons on the control device of the projection equipment. Upon receiving the selection command, the controller 500, in response to the selection command, marks the candidate contour area specified in the selection command as the feature contour area.

[0110] In some embodiments of this application, the first prompt information can be displayed in the form of a list, which includes selectable candidate contour regions and the average depth and area ratio of the feature points of the corresponding candidate contour regions. When the user selects a candidate contour region through the control device, the controller 500 can also control the light-emitting component to mark the contour portion of the selected candidate contour region so that the user can more intuitively see the area of ​​the candidate contour region. The controller 500 can choose a color different from the color of the feature map card or feature points as the mark color; this application embodiment does not impose specific limitations on the mark color.

[0111] S500: Calculate the angle between the projection surface and the light-emitting component based on the feature points, and control the light-emitting component to project the projection content onto the projection surface according to the angle.

[0112] After determining the feature contour region, the controller 500 extracts feature points from the second sampled image based on the feature contour region, calculates the angle between the projection surface 400 and the light-emitting component based on the feature points, and controls the optical engine 200 to project the corrected projection content onto the projection surface 400 based on the angle.

[0113] In some embodiments of this application, during the process of calculating the angle between the projection surface 400 and the light-emitting component based on feature points, the controller 500 can also control the camera 600 to retrieve the camera coordinate system and obtain the feature point coordinates of the feature points within the feature contour area in the camera coordinate system. The feature point coordinates are usually the center of the graphic where the feature point is located. For example, when the feature point is a square or rectangle, the feature point coordinates are the coordinates of the center of the square or rectangle; when the feature point is a circle, the feature point coordinates are the coordinates of the center of the circle.

[0114] To facilitate the capture of sampled images by the camera 600, the camera 600 is typically positioned directly in front of the projection device along with the optical engine 200. After acquiring the feature point coordinates in the camera coordinate system, the controller 500 also needs to control the light-emitting component to switch to the light-emitting component coordinate system. Based on the hardware parameters of the camera 600 and the light-emitting component, the feature point coordinates are converted into light-emitting point coordinates in the light-emitting component coordinate system. These hardware parameters can be the vector displacement values ​​of the lens center of the camera 600 and the center of the optical engine 200 of the light-emitting component.

[0115] In some embodiments of this application, the controller 500 may first obtain the coordinates of the center of the camera lens of the camera 600, such as... Figure 12 As shown, the X1Y1 coordinate system is the light-emitting component coordinate system, and the X2Y2 coordinate system is the camera coordinate system. When the center of the camera lens of camera 600 is at coordinates (0, 0) in the camera coordinate system, these coordinates are then transformed into the light-emitting component coordinate system, and the coordinates of the camera lens center are repositioned. For example, the repositioned coordinates are (30, -40). The controller 500 can then calculate the vector displacement value of 50 between the center of the camera lens and the center of the optical engine 200 of the light-emitting component, based on the coordinates before and after repositioning. After calculating the vector displacement value, the controller 500 can convert all feature point coordinates in the camera coordinate system into light-emitting point coordinates in the light-emitting component coordinate system.

[0116] After converting the feature point coordinates into light-emitting point coordinates in the light-emitting component coordinate system, the controller 500 can fit a new projection surface 400 in the light-emitting component coordinate system based on multiple light-emitting point coordinates. After fitting the new projection surface 400, the controller 500 calculates the angle between the projection surface 400 and the corresponding light-emitting surface of the light-emitting component, and controls the light-emitting component to project the corrected projection content onto the projection surface according to the angle, so that the user can see the corrected projection image.

[0117] In some embodiments of this application, during the process of controlling the light-emitting component to project content onto the projection surface 400 according to the included angle, the controller 500 can also acquire the operating parameters of the light-emitting component and calculate the projectable area based on the operating parameters and the included angle between the projection surface 400 and the light-emitting component. The operating parameters may include the projection distance, the focal length or resolution of the optical engine 200, etc. For example, the controller 500 can calculate the area of ​​the projected image when projecting vertically (90°) based on the projection distance. Then, based on the included angle between the projection surface 400 and the light-emitting component, the controller determines the trigonometric function value corresponding to that angle. Finally, the projectable area of ​​the projected image is calculated based on the area of ​​the projected image and the trigonometric function value.

[0118] To suit playback devices such as mobile terminals or smart TVs, projection devices typically project rectangular images. However, when the projectable area is an irregular or non-rectangular shape, the controller 500 can define a target projection area within the projectable area. The target projection area is the largest inscribed rectangular area within the projectable area, and this rectangular area has a preset aspect ratio. The target projection area can adapt to projection images displayed in either landscape or portrait orientations. Figure 13 As shown, before defining the target projection area, the controller 500 can also identify the video or image playback source to be projected and extract the aspect ratio of the projected content. Typical aspect ratios include 4:3, 16:9, 2.39:1, or 1.85:1. Based on the comparison between the extracted aspect ratio and the aspect ratio of the rectangular area, it is determined whether the target projection area is the largest inscribed rectangle in the horizontal direction or the largest inscribed rectangle in the vertical direction.

[0119] Figure 14 A flowchart illustrating the process of dividing a target projection area based on an obstacle avoidance command is provided. When defining the target projection area, the controller 500 can also detect obstacle avoidance commands input by the user to activate the obstacle avoidance function. If an obstacle avoidance command is detected, the controller 500 switches the obstacle avoidance state of the projection device to active and defines the target projection area based on feature points extracted from the second sampled image. If no obstacle avoidance command is detected, the obstacle avoidance state of the projection device remains active, and the controller 500 defines the target projection area based on the vertex coordinates of the light-emitting surface of the light-emitting component.

[0120] After defining the target projection area, the controller 500 can also inversely transform the coordinates of the target projection area to the light-emitting surface of the light-emitting component, and control the light-emitting component to project the projection content according to the inversely transformed target projection area coordinates.

[0121] In some embodiments of this application, a projection image correction method is also provided, applied to the projection device, which includes a light-emitting component, a camera 600, and a controller 500. Figure 15The diagram shows a flowchart of a projection image correction method according to some embodiments of this application. The image correction method includes:

[0122] S100: In response to the projection image correction command, control the light-emitting component to project a correction image, the correction image including a solid color image card and a feature image card;

[0123] S200: Acquire a first sampled image obtained by the camera 600 from the solid color image card, and a second sampled image obtained by the camera 600 from the feature image card;

[0124] S300: Determine a feature contour region based on the first sampled image, wherein the feature contour region is the contour region with the largest area in the first sampled image or a contour region specified by the user.

[0125] S400: Extract feature points from the second sampled image according to the feature contour region;

[0126] S500: Calculate the angle between the projection surface 400 and the light-emitting component based on the feature points, and control the light-emitting component to project the content onto the projection surface 400 according to the angle.

[0127] As can be seen from the above solutions, some embodiments of this application provide a projection device and projection image correction method that, upon receiving a projection image correction command, controls the light-emitting component to project corrected images of the initial color map and the feature map. It also acquires a first sampled image of the solid color map and a second sampled image of the feature map. Based on the first sampled image, a feature contour region is determined, and feature points are extracted from the second sampled image according to the feature contour region. Based on the feature points extracted from the second sampled image, the angle between the projection surface and the light-emitting component is calculated. The light-emitting component projects the projection content onto the projection surface according to the angle, ensuring that when the projection surface is obstructed by an obstacle, the projection device can extract feature points from the feature contour region after obstacle avoidance and project a corrected projection image onto the projection surface, thereby improving the user experience when using the projection device.

[0128] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0130] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the foregoing exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be made based on the foregoing teachings. The selection and description of the above embodiments are for the purpose of better explaining the contents of this disclosure, thereby enabling those skilled in the art to better utilize the described embodiments.

Claims

1. A projection device, characterized by The application comprises: a light-emitting component configured to project projection content to a projection surface; a camera configured to capture sample images; a controller configured to: in response to a projection picture correction instruction, control the light-emitting component to project a correction image, the correction image comprising a pure color chart and a feature chart; obtain a first sample image captured by the camera from the pure color chart and a second sample image captured by the camera from the feature chart; extract contour regions in the first sample image; detect a setting state of an obstacle avoidance switch; if the setting state is on, traverse areas of the contour regions in the first sample image to filter out a preset number of candidate contour regions, the candidate contour regions being used for a user to designate as a feature contour region; calculate an average depth of feature points in the second sample image within a same candidate contour region; calculate an area ratio of the candidate contour region relative to the second sample image; generate first prompt information according to the average depth and the area ratio, and control the light-emitting component to project the first prompt information; obtain a selection instruction input by a user based on the first prompt information; in response to the selection instruction, mark the candidate contour region designated in the selection instruction as the feature contour region; extract feature points in the second sample image according to the feature contour region; calculate an included angle between the projection surface and the light-emitting component based on the feature points, and control the light-emitting component to project projection content to the projection surface according to the included angle.

2. The projection device according to claim 1, characterized in that, The controller configured to determine a feature contour region according to the first sample image is further configured to: if the setting state is off, traverse areas of the contour regions in the first sample image to filter out a contour region with a largest area as the feature contour region.

3. The projection device according to claim 2, characterized in that, The controller configured to calculate an average depth of feature points in the second sample image within a same candidate contour region is further configured to: traverse color values of the feature points in the candidate contour region; extract a projection shape according to the color values of the feature points; obtain hardware parameters of the camera and the light-emitting component, and obtain a standard shape on the feature chart in the second sample image; calculate distances between the feature points and the light-emitting component according to the projection shape, the standard shape, and the hardware parameters of the camera; calculate an average value of the distances between the feature points and the light-emitting component to obtain the average depth.

4. The projection apparatus according to claim 1, wherein, The controller configured to extract contour regions in the first sample image is further configured to: traverse color values of pixel points in the first sample image; recognize a boundary figure according to the color values of the pixel points, the boundary figure being a figure composed of pixel points with a color difference value greater than or equal to a color difference threshold value; delimit the contour regions according to the boundary figure and an edge of the first sample image.

5. The projection apparatus according to claim 1, wherein, The controller configured to calculate an included angle between the projection surface and the light-emitting component based on the feature points is further configured to: obtain feature point coordinates of the feature points in the feature contour region in a camera coordinate system; According to hardware parameters of the camera and the light-emitting assembly, the feature point coordinates are converted into light-emitting point coordinates in a light-emitting assembly coordinate system; A projection surface in the light-emitting assembly coordinate system is fitted according to the light-emitting point coordinates; An included angle between the projection surface and a corresponding light-emitting surface of the light-emitting assembly is calculated.

6. The projection apparatus according to claim 1, wherein, The controller performs control on the light-emitting assembly to project projection content to the projection surface according to the included angle, and is further configured to: Obtain operation parameters of the light-emitting assembly; According to the operation parameters and the included angle, a projectable area is calculated; A target projection area is demarcated in the projectable area, the target projection area being a maximum inscribed rectangular area in the projectable area, the rectangular area having a preset aspect ratio; According to the operation parameters and the included angle, the target projection area coordinates are inversely transformed to the light-emitting surface of the light-emitting assembly; The light-emitting assembly is controlled to project projection content according to the inversely transformed target projection area coordinates.

7. The projection apparatus according to claim 6, wherein, The controller performs demarcation of the target projection area in the projectable area, and is further configured to: Detect an obstacle avoidance instruction input by a user for starting an obstacle avoidance function; If the obstacle avoidance instruction is detected, the target projection area is demarcated according to the feature points; If the obstacle avoidance instruction is not detected, the target projection area is demarcated according to vertex coordinates of the light-emitting surface of the light-emitting assembly.

8. The projection apparatus according to claim 1, wherein, The controller performs determination of a feature contour area according to the first sample image, and is further configured to: The first sample image is input into a recognition model, the recognition model being a neural network model trained according to sample images; An identification result output by the recognition model is obtained, the identification result being a classification probability of the first sample image containing an obstacle target; If the identification result is that the first sample image contains an obstacle target, a contour area corresponding to the obstacle target is removed from the first sample image, and a feature contour area is determined in the first sample image after the obstacle target is removed; If the identification result is that the first sample image does not contain an obstacle target, a feature contour area is determined according to the first sample image.

9. A method of correcting a projected image, characterized by, The projection device comprises a light-emitting assembly, a camera, and a controller, and the projection picture correction method comprises: In response to a projection picture correction instruction, the light-emitting assembly is controlled to project a correction image, the correction image comprising a pure color chart and a feature chart; A first sample image obtained by the camera shooting the pure color chart and a second sample image obtained by the camera shooting the feature chart are obtained; Contour areas are extracted in the first sample image; A setting state of an obstacle avoidance switch is detected; If the setting state is on, areas of the contour areas in the first sample image are traversed to screen a preset number of candidate contour areas, the candidate contour areas being used for a user to specify as the feature contour area; An average depth of feature points in the second sample image located in a same candidate contour area is calculated; An area ratio of the candidate contour area relative to the second sample image is calculated; First prompt information is generated according to the average depth and the area ratio, and the light-emitting assembly is controlled to project the first prompt information; acquire a selection instruction input by a user based on the first prompt information; in response to the selection instruction, mark the alternative contour region specified in the selection instruction as the feature contour region; extract a feature point in the second sampling image according to the feature contour region; calculate an included angle between a projection plane and the light emitting component based on the feature point, and control the light emitting component to project projection content to the projection plane according to the included angle.

Images