Image projection equipment and its control method

Abstract

An image projection device and its control method may be provided. Specifically, an image projection device may include a first camera, a second camera, a light source, an image processor, a memory storing one or more computer programs, and one or more processors. The processors may be configured to obtain attribute information about the properties of the surface on which the image is projected, obtain viewpoint information about the user's viewpoint of viewing the image, generate a virtual image based on the attribute information and viewpoint information, and project the image onto the surface based on the virtual image and the user's position.

Description

Technical Field

[0001] This invention relates to an image projection device and its control method. Background Technology

[0002] Image projection devices can be electronic devices that project images onto a surface located away from the image projection device. For example, an image projection device can be a beam projector, which displays images or videos by projecting them onto a screen or wall located away from the image projection device. Image projection devices can project large images onto a surface other than a display, allowing a large number of viewers to view the image with less space constraints.

[0003] Image projection devices can project images assuming the surface onto which the image is projected is flat and uniformly white. For example, an image projection device can project an image assuming the screen or wall is flat and uniformly white. The image projected by the image projection device may be distorted when the surface onto which the image is projected is folded, forms at least one curve on the surface, or includes at least one color. Therefore, distortion may occur in the projected image when the image projection device projects an image onto a wall with curved portions, a curved surface (such as a curtain), or a colored surface. Summary of the Invention

[0004] [Solution to the problem]

[0005] An image projection device according to an embodiment of the present disclosure may include a first camera, a second camera, a light source, an image processor, a memory storing one or more computer programs, and one or more processors communicatively coupled to the first camera, the second camera, the light source, the image processor, and the memory, wherein the one or more computer programs include computer-executable instructions that, when executed individually or jointly by the one or more processors, cause the image projection device to: obtain attribute information about the properties of a surface on which an image is projected; obtain viewpoint information about the viewpoint from which a user views the image; generate a virtual image based on the attribute information and the viewpoint information; and project the image onto the surface based on the virtual image and the user's position.

[0006] The control method of the image projection device according to embodiments of the present disclosure may include: obtaining attribute information about the properties of the surface on which the image is projected by the image projection device; obtaining viewpoint information about the viewpoint from which the user views the image by the image projection device; generating a virtual image by the image projection device based on the attribute information and the viewpoint information; and projecting the image onto the surface by the image projection device based on the virtual image and the user's position. Attached Figure Description

[0007] Figure 1 An image projection device is shown that projects an image according to an embodiment of the present disclosure.

[0008] Figure 2 An image projection device is shown that projects an image according to an embodiment of the present disclosure.

[0009] Figure 3 A distorted image projected by an image projection device is shown.

[0010] Figure 4 The causes of image distortion projected by an image projection device are shown.

[0011] Figure 5 This is a block diagram of an image projection apparatus according to an embodiment of the present disclosure.

[0012] Figure 6 This is a flowchart illustrating a control method for an image projection apparatus according to an embodiment of the present disclosure.

[0013] Figure 7 This is a flowchart illustrating a method for obtaining attribute information from an image projection device according to an embodiment of the present disclosure.

[0014] Figure 8 This is a flowchart illustrating a method for obtaining viewpoint information from an image projection device according to an embodiment of the present disclosure.

[0015] Figure 9 This is a flowchart illustrating a method for projecting an image by an image projection device according to an embodiment of the present disclosure.

[0016] Figure 10 The image projection device is shown calculating distance according to an embodiment of the present disclosure.

[0017] Figure 11 A virtual image generated by an image projection device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0018] The terminology used in this disclosure is briefly described, and embodiments of this disclosure are described in detail.

[0019] In consideration of the functions described herein, the terms used herein have been selected from currently widely used general terminology. However, these terms may vary depending on the intent of a person skilled in the art, precedent, and the emergence of new technologies. Furthermore, for specific cases, the meanings of the terms chosen by the applicant are described in detail in the specification section. Therefore, the terms used herein are defined based on their meaning in relation to the content discussed throughout the specification, rather than by their simplistic meaning.

[0020] When a part may "comprise" or "contain" a constituent element, it may not be construed as excluding another constituent element, but rather as including other constituent elements, unless otherwise stated. Furthermore, terms such as "...part", "...unit", "...module" and "...block" as used in this disclosure may refer to a unit that performs at least one function or operation, and such unit may be embodied in hardware, software, or a combination of hardware and software.

[0021] In the following description, embodiments of the present disclosure will be explained in detail with reference to the accompanying drawings. However, the present disclosure is not limited thereto, and it will be understood that various changes in form and detail may be made therein without departing from the spirit and scope of the appended claims. In the drawings, portions irrelevant to the description have been omitted for clarity, and similar portions are indicated by similar reference numerals throughout the specification.

[0022] Any function or operation described herein can be processed by a processor or a combination of processors. A processor or a combination of processors is circuitry that performs processing and includes, for example, an application processor (AP, such as a central processing unit (CPU)), a communication processor (CP, such as a modem), a graphics processing unit (GPU), a neural processing unit (NPU) (e.g., an artificial intelligence (AI) chip), a Wi-Fi chip, Bluetooth, etc. ® Circuits for chips, including GPS chips, NFC chips, connectivity chips, sensor controllers, touch controllers, fingerprint sensor controllers, display driver integrated circuits (ICs), audio CODEC chips, Universal Serial Bus (USB) controllers, camera controllers, image processing ICs, microprocessor units (MPUs), system-on-a-chip (SoCs), and other ICs.

[0023] Figure 1 An image projection device 110 according to an embodiment of the present disclosure is shown projecting an image.

[0024] Image projection device 110 can be an electronic device that projects images onto a surface located remotely from the image projection device 110. For example, image projection device 110 can be a beam projector that projects and displays images or videos on a screen or wall remotely from the image projection device 110. Image projection device 110 can project large images onto a surface other than a display, thereby enabling a large number of viewers to view images with less space constraints. Specifically, image projection device 110 according to embodiments of this disclosure can improve the quality of images projected onto a surface in various environments.

[0025] The image projection device 110 according to embodiments of the present disclosure may include a plurality of built-in projectors 111. The image projection device 110 may include at least one built-in camera. The image projection device 110 may have a projection area of ​​180° to 360°. The image projection device 110 may be mounted on a ceiling or wall in an interior space. For example, the image projection device 110 may be mounted on a ceiling in an interior space. For example, the image projection device 110 may be mounted on a side wall in an interior space.

[0026] The image projection device 110 according to embodiments of the present disclosure can project images onto a surface having curved portions. For example, the image projection device 110 according to embodiments of the present disclosure can project images onto a wall having curved portions via an upper surface, a lower surface, a left surface, and a right surface, thereby projecting an image including a main portion 120, a first edge portion 121, a second edge portion 122, a third edge portion 123, and a fourth edge portion 124. The image projection device 110 can reduce distortion caused by curved portions in the projected image.

[0027] The image projection device 110 according to embodiments of the present disclosure can seamlessly stitch together images projected from multiple built-in projectors 111. The image projection device 110 according to embodiments of the present disclosure can correct the projected image by reflecting the user's position in real time.

[0028] Figure 2 An image projection device 110 according to an embodiment of the present disclosure is shown projecting an image.

[0029] The image projection device 110 according to an embodiment of the present disclosure can be installed on the floor indoors. For example, the image projection device 110 can be installed on the floor indoors. For example, the image projection device 110 can be installed on a desk, table, or dressing table indoors.

[0030] The image projection device 110 according to embodiments of the present disclosure can project images onto at least one surface. For example, the image projection device 110 can project a first image 210 onto the front sidewall and a second image 220 onto the left sidewall. The image projection device 110 can project different images onto multiple surfaces. The image projection device 110 can also consider the position and angle of each of the multiple surfaces when projecting images.

[0031] The image projection device 110 according to embodiments of the present disclosure can project images onto surfaces located remotely from the image projection device 110 in various external environments. For example, the image projection device 110 can project images from an external environment onto the surface, the external environment including at least one of various illuminance and various colors. The image projection device 110 can compensate for image distortion caused by the external environment. The image projection device 110 can project images in various external environments, thereby reducing limitations caused by the external environment during the use of the image projection device 110.

[0032] Specifically, the image projection device 110 according to embodiments of this disclosure can project images onto various surfaces. For example, the image projection device 110 can project images onto walls with curved portions, curved surfaces (such as curtains), or colored surfaces. The image projection device 110 can compensate for image distortion caused by the surface onto which the image is projected. The image projection device 110 can project images onto surfaces other than flat and uniformly white surfaces, thereby increasing the applicability of the image projection device 110.

[0033] Figure 3 The image distortion projected by the image projection device is shown.

[0034] Image projection devices can be non-linear devices that project images in a form optimally viewed when perceived by a user. Considering that the human eye perceives color and brightness through non-linear types and methods, image projection devices can project images onto a surface. An image projection device may include multiple image processing modules to process the image to be projected onto the surface. The image projection device can process the image assuming it is projected onto a flat and uniformly white surface.

[0035] When an image projection device projects an image onto an irregular surface, distortion may occur in the projected image. For example, when the image projection device projects an image onto a surface with curves and translucency (such as a curtain), the original main image 310 to be projected and the distorted image 320 projected from that surface may appear to overlap each other. Additionally, when the image projection device projects an image onto a wall with curved sections, an irregular surface, or a colored surface, the original image to be projected may be distorted and displayed on the surface.

[0036] The image projection device 110 according to embodiments of the present disclosure can employ solutions Figure 3 The method for solving the problem shown.

[0037] Figure 4 The causes of image distortion projected by an image projection device are shown.

[0038] When an image projection device projects an image onto an irregular surface 410, the projected image may be distorted for each region due to the irregular surface 410. The distance from the image projection device to the irregular surface 410 may differ for each region of the irregular surface 410. For example, the distance from the image projection device to a concave portion of the irregular surface 410 may be greater than the distance from the image projection device to a convex portion of the irregular surface 410. Therefore, the image projected onto the irregular surface 410 may be distorted, decreasing in size in concave portions and increasing in size in convex portions.

[0039] The projected image can be perceived differently depending on the user's perceived position. When the image projection device projects an image onto the irregular surface 410, the distortion occurring in the projected image may be perceived as more severe depending on the user's position. For example, when the user's position is close to a curve on the irregular surface 410, the distortion occurring in the projected image may be perceived as more severe.

[0040] The image projection device 110 according to embodiments of the present disclosure can employ solutions Figure 4 The method for solving the problem shown.

[0041] Figure 5 This is a block diagram of an image projection device 110 according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the image projection device 110 may include a first camera 510, a second camera 520, a light source 530, an image processing module 540, and a processor 550.

[0042] The first camera 510 can capture images of the surface projected by the image projection device 110. The first camera 510 can obtain state information about the state of the surface. For example, the first camera 510 can obtain information about whether the surface is flat or curved, information about the curvature of the surface, and information about the color of the surface. The first camera 510 can be a red-green-blue (RGB) camera or an infrared camera.

[0043] The second camera 520 can capture images of the surface projected by the image projection device 110. The second camera 520 can obtain depth information about the surface's depth. For example, the second camera 520 can obtain information about the distance from the image projection device 110 to the surface. The second camera 520 can capture images of the user. The second camera 520 can obtain information about the user's position. The second camera 520 can obtain information about the user's viewing angle of the surface. The second camera 520 can be a time-of-flight (ToF) camera.

[0044] Light source 530 can project an image projected by image projection device 110 onto a surface. Light source 530 can project optical information onto the surface to display an image on the surface. Light source 530 can project a patterned image onto the surface to measure the shape of the surface and the distance to the surface. Light source 530 may include at least one projector. For example, light source 530 may include multiple projectors that project light including optical information in different directions. Light source 530 may include a sensor module for measuring the distance to the surface. For example, light source 530 may include a light detection and ranging (LiDAR) sensor.

[0045] Image processing module 540 can generate an image to be projected onto a surface. Image processing module 540 can generate the image based on state information about the surface's state and depth information about the surface's depth. Image processing module 540 can consider the user's position when generating the image. Image processing module 540 can consider the surface's curvature, curvature shape, and color when generating the image. Image processing module 540 can compensate for distortions caused by the surface's curvature, curvature shape, and color. Image processing module 540 can consider perceiving the image at the user's position to determine the image's viewpoint.

[0046] In this embodiment, the image processing module 540 can calculate information about the geometry of the surface on which the image is projected. This geometric information may include information about the three-dimensional shape of the surface, information about the curves of the surface, and information about the color of the surface. The geometric information may also include information about the shape of the surface that cannot be obtained by the first camera 510 or the second camera 520. When calculating information about the surface geometry, the image processing module 540 may even consider distortions that are not perceived by the first camera 510 or the second camera 520 but are observed in the projected image from the user's perspective.

[0047] Processor 550 can control first camera 510, second camera 520, light source 530, and image processing module 540. Processor 550 can control first camera 510 to capture images of the surface to be projected. Processor 550 can receive state information about the surface obtained by first camera 510. Processor 550 can control second camera 520 to capture images of the user or the surface to which the image is to be projected. Processor 550 can receive depth information about the surface depth and information about the user's position obtained by second camera 520. Processor 550 can control light source 530 to project optical information onto the surface. Processor 550 can receive information about the surface shape and distance to the surface obtained by light source 530. Processor 550 can send the obtained state information about the surface state, depth information about the surface depth, and information about the user's position to image processing module 540. Processor 550 can receive images generated by image processing module 540. Processor 550 can control light source 530 to project the generated images onto the surface.

[0048] Figure 6 This is a flowchart of a control method for an image projection device 110 according to an embodiment of the present disclosure.

[0049] In operation 610, the image projection device 110 according to an embodiment of the present disclosure can obtain attribute information about the properties of the surface on which the image is projected. The attribute information about the surface properties may include information about the surface's curvature, curves, or color. The surface of the projected image may have curvature such as a corner between walls, curves such as curtains, or color. The image projection device 110 can identify curvature, curves, and colors in the surface on which the image is projected. The image projection device 110 can obtain information about the curvature, curves, and colors in the surface.

[0050] In operation 620, the image projection device 110 according to embodiments of the present disclosure can obtain viewpoint information about the viewpoint from which a user views an image. The viewpoint information may include the user's viewpoint from their position. The image projection device 110 can detect the user's position. For example, the image projection device 110 can capture the user's position. The image projection device 110 can obtain information about the user's position. For example, the image projection device 110 can obtain the user's position as coordinate information. The image projection device 110 can calculate the user's viewpoint from their position when viewing the image.

[0051] In operation 630, the image projection device 110 according to embodiments of the present disclosure can generate a virtual image based on attribute information and viewpoint information. When an image is projected onto a surface based on attribute information, the image projection device 110 can generate a virtual image to compensate for distortions occurring in the image. For example, to compensate for the distortion that an image projected onto a surface with curved portions appears curved, the image projection device 110 can generate a virtual image in which the curved portions are processed as seamless. For example, to compensate for the distortion that an image projected onto a surface with curved portions appears curved, the image projection device 110 can generate a virtual image in which the curved portions are processed as smooth. For example, to compensate for the distortion that an image projected onto a colored surface appears curved, the image projection device 110 can generate a virtual image with an additional color effect opposite to the color of the surface.

[0052] In operation 640, the image projection device 110 according to embodiments of the present disclosure can project an image onto a surface based on a virtual image and the user's position. The image projection device 110 can generate an image to be projected onto the surface based on the generated virtual image. The image projection device 110 can consider the viewpoint when viewing the image projected onto the surface from the user's position. For example, the image projection device 110 can calculate the distance from the user's position to the surface and the angle at which the user views the surface. The image projection device 110 can correct the image to be projected onto the surface by reflecting the distance from the user's position to the surface and the angle at which the user views the surface. The image projection device 110 can then project the corrected image onto the surface.

[0053] Figure 7 This is a flowchart illustrating a method for obtaining attribute information from an image projection device according to an embodiment of the present disclosure.

[0054] In operation 710, the image projection device 110 according to embodiments of the present disclosure can project a patterned image onto a surface. The patterned image can be a specific image used to obtain state information about the state of the surface. For example, the patterned image can be an image including dot patterns and stripe patterns to identify surface curvature. For example, the patterned image can be an image including wavy patterns to identify surface curves. The image projection device 110 can capture a patterned image projected onto the surface. The image projection device 110 can compare the initially intended patterned image with the actual patterned image projected onto the surface. The image projection device 110 can obtain state information about the state of the surface based on the comparison result.

[0055] In operation 720, the image projection device 110 according to embodiments of the present disclosure can be corrected based on a patterned image. The image projection device 110 can obtain state information about the state of a surface based on the patterned image. The image projection device 110 can predict the degree of distortion when an image is projected onto the surface based on the state information about the surface state. The image projection device 110 can also be corrected based on the prediction result. For example, to compensate for the predicted degree of image distortion, the image projection device 110 can adjust the intensity, angle, and brightness of the light projected from the image projection device 110. For example, the image projection device 110 can adjust the light projected onto a portion of the image distorted by the surface by controlling at least some of the projectors of the image projection device 110.

[0056] In operation 730, the image projection device 110 according to embodiments of the present disclosure can obtain depth information about the surface depth based on a pattern image and a correction result. The image projection device 110 can capture a pattern image and a pattern image projected onto the surface after correcting the light projected from the image projection device 110. The image projection device 110 can measure the distance from the image projection device 110 to the surface based on the pattern image correction result.

[0057] In operation 740, the image projection device 110 according to embodiments of the present disclosure can obtain surface color information and surface curve information based on a pattern image and depth information. The surface color information may include information about the color of the surface. The surface curve information may include information about the shape of a curve formed on the surface. The image projection device 110 can obtain the shape of a pattern image projected onto the surface. The image projection device 110 can consider the depth information in the pattern image projected onto the surface. The image projection device 110 can analyze how the pattern image is distorted by the surface based on the pattern image projected onto the surface and the depth information. The image projection device 110 can identify the color of the surface and the shape of the curves on the surface based on the analysis results. The image projection device 110 can obtain surface color information and surface curve information based on the identification results.

[0058] Figure 8 This is a flowchart of a method for obtaining viewpoint information from an image projection device 110 according to an embodiment of the present disclosure.

[0059] In operation 810, the image projection device 110 according to embodiments of the present disclosure can locate the user's position. The image projection device 110 can obtain location information about the user's position. For example, the image projection device 110 can obtain information about the user's position by photographing the place the user has left. For example, the image projection device 110 can obtain information about the user's position by using a sensor module. For example, the image projection device 110 can receive information about the user's position obtained from an external device. The image projection device 110 can store information about the user's position. The image projection device 110 can locate the user's position based on the stored information about the user's position. For example, the image projection device 110 can convert the user's position into coordinate values ​​relative to the image projection device 110. The image projection device 110 can locate the user's position by using the current position of the image projection device 110 as a reference position and converting the user's position into coordinate values.

[0060] In operation 820, the image projection device 110 according to embodiments of the present disclosure can obtain the viewpoint of a user viewing an image based on the position of a located user. The image projection device 110 can calculate the distance from the located user's position to the surface on which the image is projected. The image projection device 110 can calculate the distance from the located user's position to the surface on which the image is projected. For example, the image projection device 110 can calculate the distance from the user's position to the surface using a lidar sensor. The image projection device 110 can calculate the direction of the viewpoint from the located user's position to the surface on which the image is projected. For example, the image projection device 110 can capture and obtain the direction of the user's viewing angle from the user's position to the surface. For example, the image projection device 110 can receive the direction from the user's position toward the surface obtained by an external device. The image projection device 110 can obtain the viewpoint of the user viewing the image based on the distance from the user's position to the surface and the direction from the user's position to the surface. The image projection device 110 can predict what the image projected onto the surface will look like from the user's viewpoint.

[0061] In operation 830, the image projection device 110 according to an embodiment of the present disclosure can generate a virtual field of view corresponding to an unphotographable area based on depth information and the user's position. The unphotographable area can be a portion of an image projected onto a surface that cannot be photographed by the image projection device 110. For example, the unphotographable area can be a portion not projected due to the state of the surface in the image to be projected onto the surface. For example, the unphotographable area can be a portion of an image projected onto the surface located at an angle that cannot be photographed by a camera included in the image projection device 110. The image projection device 110 can assume the image as viewed from the user's position based on information about the distance from the user's position to the surface obtained from the depth information and the user's position. The image projection device 110 can predict what the image will look like in the unphotographable area by assuming the image as viewed from the user's position. The image projection device 110 can also generate a virtual field of view corresponding to the unphotographable area based on the prediction result.

[0062] Figure 9 This is a flowchart of a method for projecting an image using an image projection device 110 according to an embodiment of the present disclosure.

[0063] In operation 910, the image projection device 110 according to an embodiment of the present disclosure can project an image onto a surface such that the image appears to have an optimal shape when viewed from the user's position. The image projection device 110 can generate a virtual field of view that shows what the image looks like when viewed from the user's position. The image projection device 110 can project the image onto the surface based on the virtual field of view.

[0064] In operation 920, the image projection device 110 according to an embodiment of the present disclosure can identify whether the user's position has changed. The image projection device 110 can periodically acquire location information about the user's position. For example, the image projection device 110 can detect the user's location by periodically using a lidar sensor. For example, the image projection device 110 can periodically receive location information from an external device. When the user's position is maintained (No in operation 920), the image projection device 110 can proceed to operation 930. When the user's position has changed (Yes in operation 920), the image projection device 110 can proceed to operation 940.

[0065] In operation 930, the image projection device 110 according to an embodiment of the present disclosure can maintain the shape of the projected image. When the user's position is maintained, the image projection device 110 can maintain the shape of the image that is projected and currently optimized to the user's position.

[0066] In operation 940, the image projection device 110 according to an embodiment of the present disclosure can correct the shape of the projected image to correspond to the changed position of the user. When the user's position changes, the image projection device 110 can generate a virtual field of view corresponding to the changed position of the user. The image projection device 110 can also correct the shape of the projected image based on the newly generated virtual field of view.

[0067] Figure 10 The image projection device 110 calculates the distance according to an embodiment of the present disclosure.

[0068] Image projection device 110 can calculate the distance to surface 1030 on which the image is projected. Image projection device 110 can perform system correction to calculate the distance to surface 1030. System correction may include internal correction using each of the first module 1010 and the second module 1020 included in image projection device 110 and external correction using the spatial relationship between the first module 1010 and the second module 1020. The first module 1010 and the second module 1020 may be two viewpoints for measuring the distance from image projection device 110 to surface 1030. For example, the first module 1010 and the second module 1020 may be a first camera 510 and a second camera 520, respectively. For example, the first module 1010 and the second module 1020 may be a first camera 510 and a light source 530, respectively. For example, the first module 1010 and the second module 1020 may be a second camera 520 and a light source 530, respectively.

[0069] Image projection device 110 can project a pattern image onto a surface 1030 on which the image is projected using a first module 1010 and a second module 1020. The pattern image can undergo pixel positioning. Pixel positioning can be a process of separating the pattern image reflected from the surface 1030 into pixel units on a first virtual surface 1011 and a second virtual surface 1021 and analyzing them. Image projection device 110 can obtain state information and depth information of surface 1030 based on the pixel positioning results of the pattern image. Image projection device 110 can calculate the distance to surface 1030 based on the state information and depth information of surface 1030.

[0070] The image projection device 110 can calculate the user's viewpoint information relative to the image projection device 110 on the coordinate axes. For example, the image projection device 110 can photograph the user and use the first module 1010 and the second module 1020 as the origin to calculate the user's position as coordinate values. For example, the image projection device 110 can photograph the user, identify the direction the user is facing, and calculate the identified direction as a unit vector value. For example, the image projection device 110 can assume the direction the user is facing as the direction facing the surface 1030.

[0071] Figure 11 A virtual image generated by an image projection device 110 according to an embodiment of the present disclosure is shown.

[0072] Image projection device 110 can project images from viewpoint 1110 toward a surface. Viewpoint 1110 may include a camera (e.g., a first camera 510 and a second camera 520) and a light source 530. Image projection device 110 can project images from viewpoint 1110 toward the surface at a minimum distance d. min 1120 Projected image. The minimum distance surface 1120 can be the surface on which an image is projected onto an image projection device 110. A virtual image 1240 can be generated based on the minimum distance on the surface 1120, projected towards the minimum distance surface 1120. The image projection device 110 can project from a viewpoint 1110 towards the maximum distance surface d. max 1130 Projected image. The maximum distance surface 1130 can be the maximum distance at which a user can perceive an image projected onto the surface. The image projection device 110 can generate a virtual three-dimensional space (volume) 1150 based on the image projected toward the maximum distance surface 1130.

[0073] The image projection device 110 can obtain depth information of at least one of the surfaces 1120 and 1130 when projecting an image. For example, the image projection device 110 can obtain depth information of at least one of the surfaces 1120 and 1130 by analyzing the pixel distribution of the projected image after projecting the pattern image.

[0074] Image projection device 110 can generate a virtual image corresponding to a user's viewpoint and project the virtual image onto at least one of surfaces 1120 and 1130. Image projection device 110 can generate the virtual image corresponding to the user's viewpoint by using acquired depth information. Image projection device 110 can additionally acquire information about the user's position. Image projection device 110 can generate a virtual image to be projected onto at least one of surfaces 1120 and 1130 with a shape optimized for the user's viewpoint.

[0075] The purpose of this disclosure is to provide an image projection device and its control method, which can reduce the distortion of the projected image even when the surface is uneven, and project an image suitable for the user's viewpoint.

[0076] An image projection device 110 according to an embodiment of the present disclosure may include a first camera 510, a second camera 520, a light source 530, an image processing module 540, a memory storing one or more computer programs, and one or more processors 550 communicatively coupled to the first camera 510, the second camera 520, the light source 530, the image processing module 540, and the memory. According to an embodiment of the present disclosure, the one or more computer programs include computer-executable instructions that, when executed individually or jointly by the one or more processors 550, cause the image projection device to: obtain attribute information about the properties of a surface on which an image is projected; obtain viewpoint information about the viewpoint from which a user views the image; generate a virtual image based on the attribute information and the viewpoint information; and project an image onto the surface based on the virtual image and the user's position.

[0077] According to embodiments of the present disclosure, when executed individually or jointly by one or more processors, the instructions can cause the image projection device to project a patterned image onto a surface and correct the image projection device 110 based on the patterned image.

[0078] According to embodiments of this disclosure, when instructions are executed individually or jointly by one or more processors, the instructions can enable an image projection device to obtain depth information about the depth of a surface based on a pattern image and the result of correction, and to obtain surface color information and surface curve information based on the pattern image and the depth information.

[0079] According to embodiments of this disclosure, instructions can enable an image projection device to locate a user's position when executed individually or jointly by one or more processors.

[0080] According to embodiments of this disclosure, when instructions are executed individually or jointly by one or more processors, the instructions can cause the image projection device to generate a virtual field of view corresponding to an unphotographable area based on depth information and the user's position.

[0081] When executed individually or jointly by one or more processors, the instructions can cause the image projection device to project an image onto a surface according to embodiments of the present disclosure, such that the image appears to have an optimal shape when viewed from the user's position, detect whether the user's position has changed, and correct the shape of the projected image to correspond to the changed user position.

[0082] According to embodiments of this disclosure, when instructions are executed individually or jointly by one or more processors, the instructions can cause the image projection device to perform system correction calculations to determine the distance to the surface on which the image is projected.

[0083] According to embodiments of this disclosure, when instructions are executed individually or jointly by one or more processors, the instructions enable the image projection device to obtain state information about the state of the surface and depth information about the depth of the surface based on the pixel positioning results of the pattern image, and to calculate the distance to the surface based on the state information and the depth information.

[0084] According to embodiments of this disclosure, when executed individually or jointly by one or more processors, instructions can cause an image projection device to calculate the viewpoint information of a user relative to the image projection device 110 on a coordinate axis.

[0085] According to embodiments of this disclosure, when executed individually or jointly by one or more processors, the instructions can cause an image projection device to generate a virtual image based on an image projected toward a surface at minimum distance, and to generate a virtual three-dimensional space based on an image projected toward a surface at maximum distance.

[0086] The control method of the image projection device according to embodiments of the present disclosure may include: obtaining attribute information about the properties of the surface on which the image is projected by the image projection device; obtaining viewpoint information about the viewpoint from which the user views the image by the image projection device; generating a virtual image by the image projection device based on the attribute information and the viewpoint information; and projecting the image onto the surface by the image projection device based on the virtual image and the user's position.

[0087] The control method of the image projection device according to the embodiments of the present disclosure may further include: projecting a pattern image onto a surface by the image projection device; and correcting the image projection device based on the pattern image by the image projection device.

[0088] The control method of the image projection device according to the embodiments of the present disclosure may further include: obtaining depth information about the depth of the surface by the image projection device based on the pattern image and the result of correction; and obtaining surface color information and surface curve information by the image projection device based on the pattern image and the depth information.

[0089] The control method of the image projection device according to embodiments of the present disclosure may further include locating the user's position by the image projection device.

[0090] The control method of the image projection device according to the embodiments of the present disclosure may further include: generating a virtual field of view corresponding to an unphotographable area based on depth information and the user's position by the image projection device.

[0091] The control method of the image projection device according to the embodiments of the present disclosure may further include: projecting an image onto a surface using the image projection device such that the image appears to be in the optimal shape when viewed from the user's position; detecting whether the user's position has changed using the image projection device; and correcting the shape of the projected image using the image projection device to correspond to the changed user's position.

[0092] The control method of the image projection device according to the embodiments of the present disclosure may further include: the image projection device calculating the distance to the surface on which the image is projected by performing system correction.

[0093] The control method of the image projection device according to the embodiments of the present disclosure may further include: obtaining state information about the state of the surface and depth information about the depth of the surface from the result of pixel positioning of the pattern image by the image projection device; and calculating the distance to the surface by the image projection device based on the state information and the depth information.

[0094] The control method of the image projection device according to the embodiments of the present disclosure may further include: calculating the viewpoint information of the user relative to the image projection device on the coordinate axis by the image projection device.

[0095] The control method of the image projection device according to the embodiments of the present disclosure may further include: generating a virtual image by the image projection device based on an image projected toward a surface with minimum distance; and generating a virtual three-dimensional space by the image projection device based on an image projected toward a surface with maximum distance.

[0096] The image projection apparatus and control method according to embodiments of the present disclosure can compensate for the distortion of the projected image that occurs when the surface is irregular, has curves, or has color by considering the properties of the surface when projecting the image.

[0097] Furthermore, the image projection device and control method according to embodiments of this disclosure can project an image suitable for the user's viewpoint by taking into account the user's viewpoint, thereby providing an improved image from the user's perspective.

[0098] The methods according to embodiments of this disclosure can be embodied as program instructions executable by various computer devices and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., individually or in combination. The program commands recorded on the medium may be specifically designed and configured for this disclosure, or may be available program commands known to those skilled in the art to which this disclosure pertains, such as computer software. Examples of computer-readable recording media include magnetic media (such as hard disks, floppy disks, or magnetic tapes), optical media (such as compact ROM (CD-ROM) or digital universal disc (DVD)), magneto-optical media (such as optical floppy disks), and hardware devices (such as ROM, RAM, and flash memory) specifically configured to store and execute program instructions. Examples of program commands may include not only machine code created by a compiler, but also high-level programming languages ​​executable by a computer using an interpreter.

[0099] Embodiments of this disclosure can be implemented in the form of a recording medium including computer-executable instructions, such as program modules executed by a computer. A computer-readable storage medium can be any available medium accessible to a computer and can include all volatile and non-volatile media, as well as removable and non-removable media. Furthermore, a computer-readable medium can include all computer storage media and communication media. A computer storage medium can include all volatile and non-volatile media, as well as removable and non-removable media, embodied by some method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. A communication medium can typically include computer-readable instructions, data structures, program modules, or other data such as modulated data signals such as carrier waves or other transmission mechanisms, and may also include information transmission media. Furthermore, embodiments of this disclosure can be implemented as a computer program including computer-executable instructions, such as a computer program or computer program product executed by a computer.

[0100] Computer-readable storage media may be provided in the form of non-transitory storage media. Here, "non-transitory" means only that the storage medium does not contain signals and is tangible, but does not distinguish between data stored semi-permanently or temporarily in the storage medium. In the example, a non-transitory storage medium may include a buffer for temporarily storing data.

[0101] The control method according to embodiments of this disclosure can be provided by being included in a computer program product. The computer program product, as a commodity, can be traded between a seller and a buyer. The computer program product is distributed in the form of a machine-readable storage medium (e.g., an optical disc read-only memory (CD-ROM)) or through an app store, or it can be distributed directly or online, for example, downloaded or uploaded between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product can be at least temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an app store's server, or a relay server.

[0102] It should be understood that the various embodiments of this disclosure described in the claims and specification can be implemented in hardware, software, or a combination of hardware and software.

[0103] Any such software may be stored in a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium stores one or more computer programs (software modules), which include computer-executable instructions that, when executed individually or jointly by one or more processors of an electronic device, cause the electronic device to perform the methods of this disclosure.

[0104] Any such software may be stored in the form of volatile or non-volatile memory, such as a storage device like read-only memory (ROM), whether erasable or rewritable, or in the form of memory, such as random access memory (RAM), memory chips, devices, or integrated circuits, or stored on an optically or magnetically readable medium, such as an optical disc (CD), a digital versatile disc (DVD), a magnetic disk, or magnetic tape. It should be understood that storage devices and storage media are various embodiments of non-transitory machine-readable storage suitable for storing one or more computer programs including instructions that, when executed, implement various embodiments of this disclosure. Therefore, various embodiments provide a program and a non-transitory machine-readable storage medium for storing such a program, the program including code for implementing the means or methods claimed as any one of the claims of this specification.

Claims

1. An image projection device (110), comprising: First camera (510); Second camera (520); Light source (530); Image processor (540); A memory that stores one or more computer programs; as well as One or more processors (550) are communicatively coupled to the first camera, the second camera, the light source, the image processor, and the memory. Wherein, the one or more computer programs include computer-executable instructions that, when executed individually or jointly by the one or more processors, cause the image projection device (110): Obtain attribute information about the properties of the surface on which the image is projected. Obtain viewpoint information about the user's perspective when viewing the image. Virtual images are generated based on attribute information and viewpoint information, and Images are projected onto a surface based on virtual images and the user's location.

2. The image projection device (110) according to claim 1, wherein, The instructions, when executed individually or jointly by the one or more processors (550), cause the image projection device (110) to: Projecting a patterned image onto a surface, and Pattern image correction image projection device (110).

3. The image projection device (110) according to claim 2, wherein, The instructions, when executed individually or jointly by the one or more processors (550), cause the image projection device (110) to: Depth information about the surface depth is obtained based on the pattern image and the correction results. Surface color information and surface curve information are obtained based on pattern images and depth information.

4. The image projection device (110) according to any one of claims 1 to 3, wherein, The instructions, when executed individually or jointly by the one or more processors (550), cause the image projection device (110) to: Locate the user's location.

5. The image projection device (110) according to claim 4, wherein, The instructions, when executed individually or jointly by the one or more processors (550), cause the image projection device (110) to: A virtual field of view is generated based on depth information and the user's location, corresponding to areas that cannot be photographed.

6. The image projection device (110) according to any one of claims 1 to 5, wherein, The instructions, when executed individually or jointly by the one or more processors (550), cause the image projection device (110) to: The image is projected onto a surface so that it appears to have the optimal shape when viewed from the user's position. Detect whether the user's location has changed, and Correct the shape of the projected image to correspond to the changed position of the user.

7. The image projection device (110) according to any one of claims 1 to 6, wherein, The instructions, when executed individually or jointly by the one or more processors (550), cause the image projection device (110) to: By performing system corrections, the distance to the surface on which the image is projected is calculated.

8. The image projection device (110) according to any one of claims 1 to 7, wherein, The instructions, when executed individually or jointly by the one or more processors (550), cause the image projection device (110) to: Based on the pixel localization results of the pattern image, state information about the surface state and depth information about the surface depth are obtained, as well as... The distance to the surface is calculated based on the state information and depth information.

9. The image projection device (110) according to any one of claims 1 to 8, wherein, The instructions, when executed individually or jointly by the one or more processors (550), cause the image projection device (110) to: Calculate the user's viewpoint information relative to the image projection device (110) on the coordinate axes.

10. The image projection device (110) according to any one of claims 1 to 9, wherein, The instructions, when executed individually or jointly by the one or more processors (550), cause the image projection device (110) to: A virtual image is generated based on the image projected toward the surface with minimum distance, and A virtual 3D space is generated based on the image projected onto the surface with the maximum distance.

11. A control method for an image projection device, the control method comprising: The image projection device obtains attribute information about the properties of the surface on which the image is projected; The image projection device obtains viewpoint information about the user's viewpoint when viewing the image; The image projection device generates a virtual image based on attribute information and viewpoint information; as well as The image projection device projects an image onto a surface based on a virtual image and the user's position.

12. The control method according to claim 11, further comprising: The image projection device projects the pattern image onto the surface; as well as The image projection device is corrected based on the pattern image.

13. The control method according to claim 12, further comprising: The image projection device obtains depth information about the surface depth based on the pattern image and the correction results; as well as The image projection device obtains surface color information and surface curve information based on the pattern image and depth information.

14. The control method according to any one of claims 11 to 13, further comprising: The image projection device is used to locate the user's position.

15. The control method according to claim 14, further comprising: The image projection device generates a virtual field of view corresponding to the unphotographable area based on depth information and the user's position.

Images