Image projection device and method for operating same

Abstract

The present invention relates to an image projection device and method for correcting an image to be projected on a multi-faceted surface forming a predetermined angle in consideration of a viewing position. The image projection device may output an image to be projected on a projection area including an image display area in which the image is substantially displayed on a projection surface including at least two surfaces bent at a predetermined angle with respect to a corner. The image projection device may receive, from a remote control device, a control signal for changing a viewpoint at which the projection area is viewed, and update pixel values of the image to be projected corresponding to projection points of the projection area in response to the control signal.

Description

Image projection device and method of operation thereof

[0001] The present disclosure relates to an image projection device and method for displaying an image on a projection surface.

[0002] Video devices can be classified into analog projection devices (hereinafter referred to as “analog projection devices”) or digital projection devices (hereinafter referred to as “digital projection devices”). The analog projection devices may be projection devices that provide visual information using a medium such as film. The digital projection devices may be projection devices that provide visual information using digital signals. The digital projection devices may include beam projectors (hereinafter referred to as “projectors”). The projectors may be classified as display devices. The projectors may be classified into CRT (Cathode Ray Tube) projectors, LCD (Liquid Crystal Display) projectors, or DLP (Digital Light Processing) projectors depending on the principle of generating light.

[0003] The above projector is mainly used to display input multimedia content as is, but with the development of wired and wireless communication networks, it can also be connected to electronic devices such as digital TVs and play a supporting role in the electronic devices.

[0004] The above projector may be an electronic device capable of projecting an image, such as a photograph, picture, or text on a slide or transparent paper, onto a screen through a lens. The projector may also be referred to as an image projection apparatus. The projector may convert data regarding an image or video in the form of a file into an optical signal (or optical image) and output the converted data. The output of the optical signal may correspond to irradiation. The optical signal output by the projector may be projected onto a screen to provide an image to a viewer.

[0005] To expand the projection area of ​​the above projector, various methods are proposed to project images not only on a flat surface but also on a projection surface that includes multiple surfaces angled at a predetermined angle or various angles. The above projector outputs images to be projected on multiple surfaces by correcting them based on a single viewpoint, but in this case, distortion may occur in the projected image due to changes in viewpoint.

[0006] The embodiment of the present disclosure can provide an image projection device and method for correcting an image to be projected on a multi-faceted surface at a predetermined angle by taking into account a viewing position.

[0007] According to an exemplary embodiment, an image projection device may include a transceiver, at least one memory, an image projector, and at least one processor operably connected to the transceiver, the at least one memory, and / or the image projector, the processor including a processing circuit. The at least one processor may be configured to individually and / or collectively: control the image projector to output an image to be projected on a projection area including an image display area where an image is to be substantially displayed on a projection surface including at least two surfaces inclined at a predetermined angle with respect to an edge; receive a control signal from a remote control device via the transceiver for changing a viewpoint for viewing the projection area; and update pixel values ​​of the image to be projected corresponding to projection points of the projection area in response to the control signal. The pixel values ​​of the image to be projected may be updated such that the image to be displayed on the image display area appears rectangular from the viewpoint.

[0008] According to an exemplary embodiment, a method of operating an image projection device may include an operation of outputting an image to be projected on a projection area including an image display area where an image is to be substantially displayed on a projection surface including at least two surfaces that are bent at a predetermined angle based on an edge, an operation of receiving a control signal for changing a viewpoint for viewing the projection area from a remote control device, and an operation of updating pixel values ​​of the image to be projected corresponding to projection points of the projection area in response to the control signal. The pixel values ​​of the image to be projected may be updated such that the image to be displayed on the image display area appears as a square from the viewpoint.

[0009] According to an exemplary embodiment of the present disclosure, an image projection device capable of projecting an image on a projection surface including a plurality of surfaces bent at a predetermined angle can provide convenience in viewing a projection image that suits one's own viewpoint by allowing a user to selectively change the viewpoint.

[0010] The present disclosure is not limited to the above-mentioned contents, and various changes or modifications may be made or modified from the exemplary embodiments of the present disclosure by a person having ordinary skill in the art.

[0011] The effects that can be achieved by the exemplary embodiments of the present disclosure can be clearly derived and understood by those skilled in the art to which the exemplary embodiments of the present disclosure pertain, from the following description. In other words, unintended effects resulting from the implementation of the exemplary embodiments of the present disclosure can also be derived by those skilled in the art from the exemplary embodiments of the present disclosure.

[0012] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0013] FIG. 1 is an exemplary diagram of projecting an image onto a projection surface including a polygon at a predetermined angle in an image projection system according to one embodiment of the present disclosure.

[0014] FIG. 2 is a drawing illustrating an example in which an image projected from an image projection device according to one embodiment of the present disclosure is distorted due to a viewing position.

[0015] FIG. 3 is an exemplary configuration diagram of an image projection system according to one embodiment of the present disclosure.

[0016] FIG. 4 is a diagram illustrating an exemplary procedure for correcting an image to be projected on a projection surface including a polygon at a predetermined angle in an image projection system according to one embodiment of the present disclosure.

[0017] FIG. 5 is a flowchart illustrating an exemplary procedure performed in an image projection device according to one embodiment of the present disclosure.

[0018] FIG. 6 is a flowchart illustrating an exemplary procedure performed by a remote control device according to one embodiment of the present disclosure.

[0019] FIG. 7 is an exemplary block diagram of an image projection device according to one embodiment of the present disclosure.

[0020] FIG. 8 is an exemplary block diagram of a remote control device according to one embodiment of the present disclosure.

[0021] FIG. 9 is a flowchart illustrating an exemplary method for performing image correction in an image projection device according to one embodiment of the present disclosure.

[0022] FIG. 10 is a diagram including a table illustrating an example of obtaining a three-dimensional position sample of a projection point included in a projection surface in an image projection device according to one embodiment of the present disclosure.

[0023] FIG. 11 is a diagram illustrating an example of grouping three-dimensional position samples corresponding to projection points included in a projection surface in an image projection device according to one embodiment of the present disclosure.

[0024] FIG. 12 is a diagram illustrating an example of modeling a projection surface based on grouped three-dimensional position samples in an image projection device according to one embodiment of the present disclosure.

[0025] FIG. 13 is a diagram illustrating a relationship in which pixels of an image output from an image projection device correspond to projection points to be projected on a projection surface according to one embodiment of the present disclosure.

[0026] FIG. 14 is a diagram illustrating a relationship between a pixel of an image output by an image projection device at a viewing point and a projection point to be projected on a projection surface according to one embodiment of the present disclosure.

[0027] FIG. 15 is a diagram illustrating an example of performing image correction in an image projection device according to one embodiment of the present disclosure.

[0028] In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0029] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the drawings. However, the present disclosure may be implemented in various different forms and is not limited to the exemplified embodiments described herein. In connection with the drawings and the present disclosure, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0030] FIG. 1 is an exemplary diagram of projecting an image onto a projection surface including a polygon at a predetermined angle in an image projection system according to one embodiment of the present disclosure.

[0031] Referring to FIG. 1, an image projection system may include an image projection device (10) or a projection surface (60). The image projection device (10) may convert input image data (hereinafter referred to as “input image”) into an optical signal (hereinafter referred to as “output image”) and output the converted signal. The output image may be projected on the projection surface (60). The projection surface (60) may include a projection area (30a, 30b) on which a moving image according to a content service, such as a movie or a game, is projected. The projection area (30a, 30b) may include an image display area (40a, 40b) on which an image projected by the output image is displayed.

[0032] The projection surface (60) may include a multi-faceted surface that forms a predetermined angle with respect to at least one corner (50). The projection surface (60) may be, for example, a surface that is bent forward at a predetermined angle (e.g., less than 180 degrees) with respect to the corner (50). The predetermined angle may be an intersection angle of the multi-faceted surfaces on the projection surface (60) viewed by the viewer. In the following description, the predetermined angle will be used with the same meaning unless otherwise defined. The projection surface (60) may be, for example, a surface that is bent backward at a predetermined angle (e.g., greater than 180 degrees) with respect to the corner (50). The drawing illustrates a projection surface (60) that is bent forward at a predetermined angle (e.g., less than 180 degrees). In the detailed description of various embodiments to be described below, it will be assumed that the projection surface (60) is bent forward at a predetermined angle (e.g., less than 180 degrees). However, the various embodiments described below can be extended and applied to a projection surface that is tilted backward at a predetermined angle (e.g., more than 180 degrees). In this case, distortion may occur in the image (hereinafter referred to as “projection image”) projected by the image projection device (1) onto the projection surface (60) (see the first drawing). For example, distortion may occur in the actual image displayed in the image display area (40a, 40b) included in the projection area (30a, 30b) due to the difference in the sizes of pixels to be displayed at a first position (e.g., near the corner (50)) and a second position (e.g., near the edge line horizontal to the corner (50)) that are different from the distances at which the output image must reach based on the viewpoint of the user (20). The distortion occurring in the projection image may be different from the distortion illustrated. For example, the distortion may occur in an opposite shape to the distortion occurring in the illustrated projection image.As illustrated, a projection image projected onto a projection area (30a, 30b) from a projection surface (60) tilted forward at a predetermined angle (e.g., less than 180 degrees) may have a shape that slopes downwards while progressing in both directions from the corner (50). FIG. 2 illustrates an example of distortion that may occur for each view point in a projection image projected onto a projection area (30a, 30b) from a projection surface (60) tilted forward at a predetermined angle (e.g., less than 180 degrees). Although not illustrated, a projection image on a projection surface tilted backward at a predetermined angle (e.g., more than 180 degrees) may have a shape that slopes upwards while progressing in both directions from the corner.

[0033] The image projection device (10) can output a corrected output image through image correction (110). The corrected output image may be an image corrected with the position of the image projection device (10) as a view point. The corrected image may be an image in which pixel values ​​corresponding to each projection point are adjusted in consideration of the distance to be reached for each projection point included in the projection surface (60) output by the image projection device (10). The corrected output image may be projected on the projection area (30a, 30b). At a viewing position corresponding to the position of the image projection device (10), a user (20) can view an undistorted image (e.g., a square) displayed on the image display area (40a, 40b) of the projection area (30a, 30b) (see the second drawing).

[0034] In a situation where a projection image is displayed by an output image that has been subjected to image correction (110) based on a reference point (e.g., the position of an image projection device (10)) as described above, if a user (20) moves to a different position, a distorted projection image is seen due to the changed point (120) (see the third drawing). The distortion may appear differently depending on where the changed point is.

[0035] The above image projection device (10) can re-perform image correction (130) on the output image by taking into account the changed point of view when the viewpoint (or point of view) of the user (20) is changed (120). By re-performing the image correction (130), the output image output by the image projection device (10) can be viewed as an undistorted projection image at the changed point of view (see the fourth drawing).

[0036] FIG. 2 is a drawing illustrating an example in which an image projected from an image projection device (e.g., the image projection device (10) of FIG. 1) according to one embodiment of the present disclosure is distorted due to a viewing position.

[0037] Referring to FIG. 2, when an output image corrected by an image projection device (10) in consideration of a reference point (e.g., the position of the image projection device (10)) is projected onto a projection surface (e.g., projection surface (60) of FIG. 1) including at least two surfaces that are bent at a predetermined angle based on a corner (50), a normal projection image that is not distorted can be viewed in the image display area (40a, 40b) at the reference point (210). However, at a point other than the reference point, the projection image in the image display area (40a, 40b) may appear distorted.

[0038] For example, if the changed point of view is located relatively to the left of the reference point of view on the same horizontal axis, the first image display area (40a) on the first side of the image display area may appear relatively narrower than the second image display area (40b) on the second side. In this case, the pixel size of the first projection points included in the first image display area (40a) may be relatively smaller than the pixel size of the second projection points included in the second image display area (40b), or may be arranged more densely. This may cause distortion of the projection image viewable at the changed point of view (see 220).

[0039] For example, if the changed point of view is located relatively to the right of the reference point of view on the same horizontal axis, the second image display area (40b) on the second side of the image display area may appear relatively narrower than the first image display area (40a) on the first side. In this case, the pixel size of the second projection points included in the second image display area (40b) may be relatively smaller than the pixel size of the first projection points included in the first image display area (40a), or may be arranged more densely. This may cause distortion of the projection image viewable at the changed point of view (see 230).

[0040] For example, if the changed point of view is located relatively higher on the same vertical axis as the reference point of view, the vicinity of the edges on both sides may appear to protrude compared to the vicinity of the corner (e.g., the corner (50) of FIG. 1) (see 240). For example, if the changed point of view is located relatively lower on the same vertical axis as the reference point of view, the vicinity of the corner (e.g., the corner (50) of FIG. 1) may appear to protrude compared to the vicinity of the edges on both sides (see 250). For example, if the changed point of view is closer to the projection surface (60) than the reference point of view, the vicinity of the corner (e.g., the corner (50) of FIG. 1) may appear to protrude relatively compared to the vicinity of the edges on both sides (see 260). For example, if the changed point of view is farther from the projection surface (60) than the reference point of view, the vicinity of the edges on both sides may appear to protrude relatively compared to the vicinity of the corner (e.g., the corner (50) of FIG. 1) (see 270).

[0041] The examples of distortion described above may be used as reference for defining the mapping relationship between the pixel values ​​of the input image and the pixel values ​​of the output image when performing image correction based on the point in time described later.

[0042] FIG. 3 is an exemplary configuration diagram of an image projection system (e.g., the image projection system of FIG. 1) according to one embodiment of the present disclosure.

[0043] Referring to FIG. 3, in the case of an indoor environment, the image projection system may include at least three sensors (e.g., Anchor 0 (310), Anchor 1 (320), Anchor 2 (330)) installed at an upper portion (e.g., a ceiling). The image projection system may include an image projection device (e.g., the image projection device (10) of FIG. 1). An output image output by the image projection device (10) may be projected onto a projection surface (60). The output image projected onto the projection surface (60) may display an actual projection image on an image display area (40a, 40b) of a projection area (e.g., the projection area (30a, 30b) of FIG. 1).

[0044] Each of the at least three sensors (310, 320, 330) can measure the distance to the image projection device (10) and provide the measured distance information to the image projection device (10). The image projection device (10) can obtain its own location using the distance information provided by the at least three sensors (310, 320, 330).

[0045] Each of the at least three sensors (310, 320, 330) can measure a distance to a user (20) and / or a remote control device (30) and provide the measured distance information to the image projection device (10). The image projection device (10) can obtain the location of the user (20) and / or the remote control device (30) using the distance information provided by the at least three sensors (310, 320, 330). The image projection device (10) can obtain the location of the remote control device (30) using the information provided from the remote control device (30). The image projection device (10) can also use the information provided from the remote control device (30) to predict the point in time at which the user (20) views a projection image displayed in the image display area (40a, 40b).

[0046] The image projection device (10) can perform correction on the output image by considering the viewpoint of the user (20). For example, the image projection device (10) can map pixel values ​​included in a first output image for which image correction has been performed based on its own position as a viewpoint to pixels at other positions adjusted based on the viewpoint of the user (20). The second output image for which image correction has been performed in this way projects an actual image on the same image display area (40a, 40b) as the first output image, but the pixel values ​​projected for each projection point of the image display area (40a, 40b) may be different.

[0047] FIG. 4 is a signal flow diagram illustrating an exemplary procedure for correcting an image to be projected on a projection surface (e.g., projection surface (60) of FIG. 1) including a polygon at a predetermined angle in an image projection system (e.g., the image projection system of FIG. 1) according to one embodiment of the present disclosure.

[0048] Referring to FIG. 4, the image projection device (10) can output a reference image (410). For example, the image projection device (10) can generate an initial image to be projected on a projection area (e.g., projection area (30a, 30b) of FIG. 1) on a multi-faceted surface at a predetermined angle in response to an image output request. The initial image may be an image corrected with the position of the image projection device (10) as a view point. The image projection device (10) can output the initial image. The initial image can be projected on the projection area (30a, 30b). At a viewing position corresponding to the position of the image projection device (10), an undistorted image (e.g., a square) displayed on an image display area (40a, 40b) included in the projection area (30a, 30b) can be viewed. However, if the viewpoint of a viewer (e.g., a user (20) of FIG. 1) is different from the position of the image projection device (10), the user (20) may see the image displayed in the image display area (40a, 40b) as distorted due to the position. Examples of images displayed in the image display area (40a, 40b) appearing distorted depending on the position are as described with reference to FIG. 2.

[0049] The above remote control device (30) can monitor whether a screen adjustment request is generated according to the viewing position (420). The screen adjustment can be requested by the user. For example, the remote control device (30) can detect whether a screen adjustment request is generated by the user adjusting the viewpoint from which the image displayed on the projection area (30a, 30b) of the projection surface (60) is viewed by operating a button (e.g., a physical button or a touch button) provided on a user interface (e.g., a user I / F (830) of FIG. 8).

[0050] The remote control device (30) may generate screen adjustment information in response to the occurrence of the screen adjustment request. The screen adjustment information may be, for example, control information including directional identification information indicating a direction in which a viewpoint is to be moved. The directional identification information may include an identifier indicating one of the forward, backward, up, down, left, or right directions. The screen adjustment information may be, for example, control information including location identification information indicating a viewpoint. The screen adjustment information may be, for example, control information including identification information indicating at least one of a distance or a direction between the image projection device (10) and the remote control device (30). The remote control device (30) may transmit the generated screen adjustment information to the image projection device (10) (430).

[0051] The image projection device (10) can receive the screen adjustment information from the remote control device (30) (430). The image projection device (10) can perform image correction based on the screen adjustment information (440). For example, the image projection device (10) can update pixel values ​​of the image to be projected corresponding to projection points of the projection area (30a, 30b) based on the screen adjustment information. The image projection device (10) can update the pixel values ​​of the image to be projected so that, for example, the image to be projected on the image display area (40a, 40b) appears as a square from a viewpoint desired by the user (20). To this end, the image projection device (10) can obtain directional identification information indicating a direction in which the viewpoint is to be moved from the screen adjustment information. The directional identification information can include an identifier indicating one of the forward, backward, up, down, left, or right directions. To this end, the image projection device (10) can obtain location identification information indicating a viewpoint from the screen adjustment information. To this end, the image projection device (10) can obtain identification information indicating at least one of a distance or a direction between the image projection device (10) and the remote control device (30) from the screen adjustment information. The image projection device (10) can predict a viewpoint at which a user (20) desires adjustment based on the identification information indicating at least one of a distance or a direction between the image projection device (10) and the remote control device (30).

[0052] The image projection device (10) can output a corrected image based on the time point at which movement is requested by the user (20) (450). The image output by the image projection device (10) can be projected on the projection area (30a, 30b) of the projection surface (60). The image projected on the projection area (30a, 30b) can display the image on the image display area (40a, 40b) included in the projection area (30a, 30b). In this case, the user (20) can view the image displayed on the image display area (40a, 40b) in a rectangular shape without distortion, as when viewing it through a physical display.

[0053] According to one example, the pixel values ​​of an image output by the image projection device (10) can be updated by correction so that the image to be displayed in the image display area (40a, 40b) appears to have a standardized shape like the initial image from the user's point of view. The initial image may be an image displayed in the image display area (40a, 40b) prior to the image output by the image projection device (10).

[0054] According to the above, when an image displayed on an image display area (40a, 40b) including at least two surfaces that are bent at a predetermined angle based on a corner (e.g., corner (50) of FIG. 1) by an image projection device (10) is distorted, a user (20) can use remote control to view an undistorted image from his / her own viewpoint by operating a remote control device (30).

[0055] FIG. 5 is a flowchart illustrating an exemplary procedure performed in an image projection device (e.g., the image projection device (10) of FIG. 1) according to one embodiment of the present disclosure.

[0056] Referring to FIG. 5, the image projection device (10) can determine, in operation 511, whether an image output request has occurred. The image output request may be generated, for example, by a user (e.g., the user (20) of FIG. 1) requesting image output while the image projection device (10) is in an active mode in which normal operation is possible. The image output request may be generated by the execution of a predetermined content service, such as the playback of a video (e.g., a movie, a drama, etc.).

[0057] The image projection device (10) may output a reference image based on a first layout in operation 513. The first layout may include information regarding a screen arrangement set so that an initial image projected on a projection area (e.g., projection area (30a, 30b) of FIG. 1) on a multi-faceted surface at a predetermined angle by the image projection device (10) is not distorted when viewed from a predetermined reference point. Accordingly, the reference image projected on the projection area (30a, 30b) may provide an undistorted image to the user (20) if the user (20) views the image from the reference point. The reference point may correspond to, for example, a position of the image projection device (10).

[0058] According to one example, the reference image may be an image corrected with the position of the image projection device (10) as a view point. The image projection device (10) may output the reference image. The reference image may be projected on the projection area (30a, 30b). At a viewing position corresponding to the position of the image projection device (10), an undistorted image (e.g., a square) displayed on the image display area (40a, 40b) included in the projection area (30a, 30b) may be viewed. However, if the viewpoint of a viewer (e.g., a user (20) of FIG. 1) is different from the position of the image projection device (10), the viewer may see the image displayed on the image display area (40a, 40b) as distorted due to the position. Examples of images displayed in the image display area (40a, 40b) at each of the above locations appearing distorted are as described with reference to FIG. 2.

[0059] The image projection device (10) can monitor whether a screen adjustment event according to a viewing position occurs in operation 515. The screen adjustment can be requested by a remote control device (30). For example, when a user (20) adjusts a viewpoint for viewing an image displayed in a projection area (30a, 30b) of a projection surface (60) by manipulating a button (e.g., a physical button or a touch button) provided on the remote control device (30), the image projection device (10) can monitor the occurrence of a screen adjustment event. The image projection device (10) can obtain screen adjustment information from the remote control device (30) by the occurrence of the screen adjustment event. The screen adjustment information can be, for example, control information including directional identification information indicating a direction in which the viewpoint is to be moved. The directional identification information can include an identifier indicating one of the forward, backward, up, down, left, or right directions. The above screen adjustment information may be, for example, control information including location identification information indicating a point of view. The above screen adjustment information may be, for example, control information including identification information indicating at least one of the distance or direction between the image projection device (10) and the remote control device (30).

[0060] The image projection device (10), in operation 517, may configure a second layout for screen adjustment based on the screen adjustment information, and output a corrected image based on the second layout. For example, the image projection device (10) may update pixel values ​​of the image to be projected corresponding to projection points of the projection area (30a, 30b) based on the screen adjustment information. The image projection device (10) may update pixel values ​​of the image to be projected so that, for example, the image to be projected on the image display area (40a, 40b) appears as a square at a point in time that the user (20) desires to adjust. To this end, the image projection device (10) may obtain directional identification information indicating a direction in which the viewpoint is to be moved from the screen adjustment information. The directional identification information may include an identifier indicating at least one of the forward, backward, up, down, left, or right directions. To this end, the image projection device (10) can obtain location identification information indicating a viewpoint from the screen adjustment information. To this end, the image projection device (10) can obtain identification information indicating at least one of a distance or a direction between the image projection device (10) and the remote control device (30) from the screen adjustment information. The image projection device (10) can predict an adjusted viewpoint desired by the user (20) based on the identification information indicating at least one of a distance or a direction between the image projection device (10) and the remote control device (30). The image projection device (10) can map pixel values ​​included in an input image to corresponding pixels among pixels included in an output image based on a viewpoint at which movement is requested by the user (20).

[0061] According to one example, the pixel values ​​of an image to be projected by the image projection device (10) can be updated by correction so that the image to be displayed in the image display area (40a, 40b) appears as a standardized shape like an initial image from the viewpoint of the user (20). The initial image may be an image displayed in the image display area (40a, 40b) prior to the image to be projected by the image projection device (10). The initial image may be, for example, an image projected based on a predetermined viewpoint (e.g., a viewpoint viewed from a position where the image projection device (10) is placed in real space) in response to an image output request.

[0062] The above image projection device (10) can output the corrected output image. The image output by the image projection device (10) can be projected on the projection area (30a, 30b) of the projection surface (60). The image projected on the projection area (30a, 30b) can be displayed on the image display area (40a, 40b) included in the projection area (30a, 30b). In this case, the user (20) can view the image displayed on the image display area (40a, 40b) in a rectangular shape without distortion, as when viewing it through a physical display.

[0063] The image projection device (10) can determine whether screen output termination occurs in operation 519. If screen output termination does not occur, the image projection device (10) can repeat operations 515 and 517. If screen output termination occurs, the image projection device (10) can stop image output in operation 521.

[0064] FIG. 6 is a flowchart illustrating an exemplary procedure performed by a remote control device (e.g., the remote control device (30) of FIG. 3) according to one embodiment of the present disclosure.

[0065] Referring to FIG. 6, the remote control device (30) can determine whether a screen remote control function is activated in operation 611. The remote control function may be a function that can remotely control the operation of an electronic device (e.g., the image projection device (10) of FIG. 1) connected via a communication link based on a predetermined communication protocol (e.g., a short-range communication protocol such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA). As an example, the remote control device (30) can adjust a viewing point to be considered in order to correct an image to be projected by the image projection device (10) on the projection area (30a, 30b) of the projection surface (60) by using the screen remote control function.

[0066] If the above screen remote control function is activated, the remote control device (30) can detect a screen adjustment request according to the viewing position in operation 613. The screen adjustment can be requested by a user. For example, the user can generate an event to adjust the viewpoint of viewing the image displayed in the projection area (30a, 30b) of the projection surface (60) by manipulating a button (e.g., a physical button or a touch button) provided on a user interface (e.g., a user I / F (830) of FIG. 8).

[0067] The remote control device (30) may generate screen adjustment information in response to the event that occurred in operation 615. The screen adjustment information may be, for example, control information including directional identification information indicating a direction in which to move the viewpoint. The directional identification information may include an identifier indicating one of the forward, backward, up, down, left, or right directions. The screen adjustment information may be, for example, control information including location identification information indicating the viewpoint. The screen adjustment information may be, for example, a control signal including identification information indicating at least one of the distance or direction between the image projection device (10) and the remote control device (30).

[0068] The above remote control device (30) can transmit the generated screen adjustment information to the image projection device (10) based on the predetermined communication protocol in operation 615.

[0069] FIG. 7 is an exemplary block diagram of an image projection device (e.g., the image projection device (10) of FIG. 1) according to one embodiment of the present disclosure.

[0070] Referring to FIG. 7, the image projection device (10) may include at least one processor (710) (e.g., including a processing circuit), a transceiver (720), a memory (730), and / or an image projector (740). The image projection device (10) may include a user interface (I / F).

[0071] The above user I / F may be configured to receive information from a user. For example, the user I / F may receive a command or data to be used in a component of the image projection device (10) (e.g., at least one processor (710)) from an external source (e.g., a user) of the image projection device (10). The user I / F may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), a remote control, or a digital pen (e.g., a stylus pen).

[0072] The user interface (I / F) may be configured to transmit information to a user. For example, the user interface (I / F) may output an audio signal to the outside of the image projection device (10). The user interface (I / F) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. The receiver may be implemented separately from the speaker or as a part thereof.

[0073] The processor (710) may be implemented as one or more integrated circuit (IC) chips and may perform various data processing operations. For example, the processor (710) (or application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or chipset). The processor (710) may include sub-components including a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a display controller, a memory controller, a storage controller, a communication processor (CP), and / or a sensor interface. The sub-components are merely exemplary. For example, the processor (710) may further include other sub-components. For example, some sub-components may be omitted from the processor (710). For example, some sub-components may be included as separate components of the image projection device (10) outside the processor (710). For example, some sub-components may be incorporated into other components (e.g., a display, an image sensor). A processor (710) according to one embodiment of the present disclosure may include various processing circuits and / or multiple processors. For example, the term "processor" as used in this disclosure, including the claims, may include various processing circuits including at least one processor, wherein one or more of the at least one processor may be configured to perform various functions described in the present disclosure individually and / or collectively in a distributed manner.As used herein, when "a processor," "at least one processor," and "one or more processors" are described as being configured to perform numerous functions, these terms encompass, for example, but are not limited to, situations where one processor performs some of the recited functions and other processor(s) perform other parts of the recited functions, and also situations where a single processor can perform all of the recited functions. Additionally, the at least one processor may comprise a combination of processors that perform the various recited / disclosed functions, for example, in a distributed manner. The at least one processor may execute program instructions to achieve or perform the various functions.

[0074] The processor (710) (e.g., CPU or central processing circuit) may be configured to control sub-components based on the execution of instructions stored in the memory (730) (e.g., volatile memory and / or non-volatile memory). The GPU (or graphics processing circuit) may be configured to execute parallel operations (e.g., rendering). The NPU (or neural processing circuit) may be configured to execute operations for an artificial intelligence model (e.g., convolution computation). The ISP (or image signal processing circuit) may be configured to process a raw image acquired through an image sensor into a format suitable for a component in the image projection device (10) or a sub-component in the processor (710). The display controller (or display control circuit) may be configured to process an image acquired from the CPU, the GPU, the ISP, or the memory (730) (e.g., volatile memory) into a format suitable for projecting it onto a projection surface. The memory controller (or memory control circuit) may be configured to control reading data from the volatile memory and writing data to the volatile memory. The storage controller (or storage control circuit) may be configured to control reading data from the non-volatile memory and writing data to the non-volatile memory. The CP (communication processing circuit) may be configured to process data obtained from a sub-component within the processor (710) into a format suitable for transmission to another electronic device via the transceiver (720), or to process data obtained from another electronic device (e.g., a remote control device (e.g., the remote control device (30) of FIG. 3)) via the transceiver (720) into a format suitable for processing by the sub-component.The above sensor interface (or sensing data processing circuit, sensor hub) may be configured to process data on the status of the image projection device (10) and / or the status around the image projection device (10), obtained through an internal sensor (e.g., a distance measurement sensor (ToF (time-of-flight) sensor) or an external sensor (e.g., one or more position measurement sensors (anchors)), into a format suitable for a sub-component within the processor (710).

[0075] According to one example, the sensing data acquired through the internal sensor may include information for acquiring position coordinates of 3D samples to be referenced for correcting a projection image. The 3D samples may correspond to some or all of the projection points included in a projection area (e.g., projection area (30a, 30b) of FIG. 1) of a projection surface (e.g., projection surface (60) of FIG. 1) from a predetermined position. The information for acquiring position coordinates of the 3D samples may include, for example, information about a distance and a direction from a predetermined position to the 3D samples. The processor (710) may use the sensing data to classify 3D samples and group the classified samples. The position information acquired for the grouped samples may be used for modeling a plurality of surfaces included in the projection area (30a, 30b).

[0076] According to one example, the sensing data acquired through the external sensor may include information to be used to acquire the position of the image projection device (10). The sensing data may include distance information to the image projection device (10) measured by at least three sensors installed above (e.g., Anchor 0 (310), Anchor 1 (320), Anchor 2 (330) of FIG. 3). The processor (710) may use the sensing data to identify the position of the image projection device (10). The processor (710) may generate an image to be initially projected based on the position of the image projection device (10). The image to be initially projected may be an image corrected so that the image projected on the projection area (30a, 30b) at the position of the image projection device (10) can be displayed as a plane without distortion due to a plurality of surfaces forming a predetermined angle.

[0077] The transceiver (720) includes various communication circuitry and can be configured to exchange information with at least one electronic device (e.g., the remote control device (30) of FIG. 3 or Anchor 0 (310), Anchor 1 (320), Anchor 2 (330) of FIG. 3). The transceiver (720) can transmit and receive data or signals with the remote control device (30) or external sensors (e.g., Anchor 0 (310), Anchor 1 (320), Anchor 2 (330)) under the control of the processor (710). The above transmitter / receiver (720) may include, but is not limited to, a Bluetooth communication unit, a BLE (Bluetooth low energy) communication unit, a near field communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an infrared (IrDA, infrared data association) communication unit, a WFD (Wi-Fi Direct) communication unit, a UWB (ultrawideband) communication unit, an Ant+ communication unit, or a microwave (uWave) communication unit, depending on the performance and structure of the image projection device (10).

[0078] According to one example, the transceiver (720) may support establishment of a direct (e.g., wired) communication channel or a wireless communication channel with the remote control device (30) and performance of communication through the established communication channel. The transceiver (720) may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. The one or more communication processors may operate independently with respect to the processor (710). The transceiver (720) may include, for example, a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules can communicate with at least one remote control device (30), which is an external electronic device, via a network (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA, or a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as a plurality of separate components (e.g., multiple chips).

[0079] The above image projector (740) may be configured to output an image to be projected onto the projection area (30a, 30b) of the projection surface (60) for screen output. The image projector (740) may, for example, convert an electrical signal provided from a light processor (710) into an image to be projected, which is an optical signal, and output the converted image toward the projection area (30a, 30b). The electrical signal provided by the processor (710) may correspond to image data such as a photograph or a video.

[0080] The processor (710) may execute software to control at least one other component (e.g., hardware or software component) such as an electrically connected user I / F, the transceiver (720), or the image projector (740), or perform processing or calculation of various data. The processor (710) may store commands or data received from other components (e.g., the user I / F or the transceiver (720)) in the memory (730) (e.g., volatile memory) as at least a part of data processing or calculation, or process commands or data stored in the memory (730) and store the processed result data in the memory (730).

[0081] According to one example, the processor (710) can correct an image to be projected on a projection area (30a, 30b) on a multi-faceted surface at a predetermined angle in consideration of a viewing position. The processor (710) can generate an image to be projected on a projection area (30a, 30b) including an image display area (e.g., image display areas (40a, 40b) of FIG. 1) where an image is to be substantially displayed on a projection surface (60) including at least two surfaces that are bent at a predetermined angle based on a corner (e.g., corner (50) of FIG. 1). The processor (710) can control the image projector (740) to output the image to be projected.

[0082] The processor (710) may generate an initial image to be projected on a projection area (30a, 30b) on a multi-faceted surface at a predetermined angle in response to an image output request. The initial image to be projected may be an image corrected with the position of the image projection device (10) in real space as a view point. The processor (710) may control the image projector (740) to output the initial image to be projected. The initial image may be projected on the projection area (30a, 30b). At the position of the image projection device (10), an image in an undistorted shape (e.g., a square) may be viewed on the image display area (40a, 40b) of the projection area (30a, 30b).

[0083] The processor (710) may receive a control signal for changing a viewpoint for viewing the projection area (30a, 30b) from the remote control device (30) through the transceiver (720). The processor (710) may update pixel values ​​of the image to be projected corresponding to projection points of the projection area (30a, 30b) in response to the control signal. The processor (710) may update the pixel values ​​of the image to be projected so that the image to be displayed in the image display area (40a, 40b) appears as a square from the viewpoint. The control signal may include directional identification information indicating a direction in which the viewpoint is to be moved. The directional identification information may include an identifier indicating one of the forward, backward, up, down, left, or right directions. The control signal may include location identification information indicating the viewpoint. The control signal may include identification information indicating at least one of the distance or direction between the image projection device (10) and the remote control device (30). The processor (710) may predict a point of view based on the identification information indicating at least one of the distance or direction between the image projection device (10) and the remote control device (30) included in the control signal.

[0084] According to an example, the processor (710) can obtain coordinate values ​​indicating the positions of some or all of the projection points included in the projection area (30a, 30b). For example, the processor (710) can obtain coordinate values ​​indicating the positions of some or all of the projection points based on data sensed by an internal or external sensor (e.g., a ToF sensor). The processor (710) can model at least two surfaces included in the projection area (30a, 30b) based on the obtained coordinate values. The processor (710) can obtain first coordinate values ​​indicating the positions of the projection points on which pixel values ​​of an image to be projected on the at least two surfaces are to be projected. The processor (710) can convert the obtained first coordinate values ​​into second coordinate values ​​based on a viewpoint. The processor (710) can determine the image display area (40a, 40b) by mapping the projection points to a two-dimensional plane based on the second coordinate values. The processor (710) can set the pixel values ​​of the input image as target pixel values ​​to be projected on the projection points included in the image display area (40a, 40b) in the image to be projected.

[0085] To perform the above-described operation, the processor (710) may include a distance measurement module (711), a position recognition module (713) and / or an image correction module (715), each of which may include various processing circuits and / or executable program instructions.

[0086] The distance measurement module (711) can obtain coordinate values ​​indicating the positions of target projection points to be referenced for modeling among the projection points included in the projection area (30a, 30b). For example, the distance measurement module (711) can identify the distance and direction to the target projection points based on data sensed by an internal or external sensor (e.g., a ToF sensor), and obtain coordinate values ​​indicating the positions of the target projection points based on this.

[0087] The above location recognition module (713) can receive distance information to the image projection device (10) measured by at least three sensors installed externally (e.g., Anchor 0 (310), Anchor 1 (320), Anchor 2 (330) of FIG. 3). The location recognition module (713) can identify the location of the image projection device (10) using the sensing data.

[0088] The image correction module (715) can correct an image to be projected on a projection area (30a, 30b) on a multi-faceted surface at a predetermined angle by taking into account a viewing position. The image correction module (715) can generate an image to be projected on a projection area (30a, 30b) including an image display area (40a, 40b) where an image is actually displayed on a projection surface (60) including at least two surfaces that are bent at a predetermined angle based on a corner (e.g., a corner (50) of FIG. 1). The image correction module (715) can control the image projector (740) to output the image to be projected.

[0089] The image correction module (715) can generate an initial image to be projected on a projection area (30a, 30b) on a multi-faceted surface at a predetermined angle in response to an image output request. The initial image to be projected may be an image corrected using the position of the image projection device (10) identified by the position recognition module (713) as a view point. The image correction module (715) can control the image projector (740) to output the initial image to be projected. The initial image can be projected on the projection area (30a, 30b). At the position of the image projection device (10), an undistorted image (e.g., a square) can be viewed on the image display area (40a, 40b) of the projection area (30a, 30b).

[0090] According to one example, the image corrected by the image correction module (715) may be an image whose pixel values ​​are updated so that the image to be displayed in the image display area (40a, 40b) appears to have a standardized shape like the initial image from the user's (20) point of view. The initial image may be an output image (760) displayed in the image display area (40a, 40b) prior to the image to be projected by the image projector (740). The initial image may be, for example, an image projected by the image projector (740) based on a predetermined point of view (e.g., a point of view viewed from a position where the image projection device (10) is placed in real space) in response to an image output request.

[0091] FIG. 8 is an exemplary block diagram of a remote control device (e.g., the remote control device (30) of FIG. 3) according to one embodiment of the present disclosure.

[0092] Referring to FIG. 8, the remote control device (30) may include at least one processor (810) (e.g., including a processing circuit), a transceiver (820), and / or a user interface (I / F) (830) (e.g., including various circuits). The remote control device (30) may generate a remote control signal in response to a user's request and provide the generated remote control signal to an image projection device (e.g., the image projection device (10) of FIG. 11).

[0093] The processor (810) may be implemented as one or more IC (integrated circuit) chips and may perform various data processing. For example, the processor (810) (or AP (application processor)) may be implemented as a SoC (system on chip) (e.g., a single chip or chipset). The processor (810) (e.g., a CPU or central processing circuit) may be configured to control sub-components based on the execution of instructions stored in a storage medium such as a memory (e.g., a volatile memory and / or a non-volatile memory). The processor (810) may include various processing circuits and / or multiple processors. For example, the term “processor” as used in this disclosure, including the claims, may include various processing circuits including at least one processor, wherein one or more of the at least one processor may be configured to perform various functions described in the present disclosure individually and / or collectively in a distributed manner. As used herein, when "a processor," "at least one processor," and "one or more processors" are described as being configured to perform numerous functions, these terms encompass, for example, but are not limited to, situations where one processor performs some of the recited functions and other processor(s) perform other parts of the recited functions, and also situations where a single processor can perform all of the recited functions. Additionally, the at least one processor may comprise a combination of processors that perform the various recited / disclosed functions, for example, in a distributed manner.At least one processor may execute program instructions to achieve or perform various functions. The processor (810) may be configured to process data acquired from a sub-component into a format suitable for transmission to another electronic device (e.g., an image projection device (10)) via the transceiver (820), or to process data acquired from another electronic device (e.g., an image projection device (10)) via the transceiver (820) into a format suitable for processing by the sub-component.

[0094] The transceiver (820) may include various communication circuitry and may be configured to exchange information with at least one electronic device (e.g., the image projection device (10) of FIG. 1). The transceiver (820) may transmit and receive data or signals with the image projection device (10) under the control of the processor (810). The transceiver (820) may include, but is not limited to, a Bluetooth communication unit, a BLE (Bluetooth low energy) communication unit, a near field communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an IrDA (infrared data association) communication unit, a WFD (Wi-Fi Direct) communication unit, a UWB (ultrawideband) communication unit, an ANT+ communication unit, or a microwave (uWave) communication unit, depending on the performance and structure of the remote control device (30).

[0095] According to one example, the transceiver (820) may support establishment of a direct (e.g., wired) communication channel or a wireless communication channel with the image projection device (10) and communication through the established communication channel. The transceiver (820) may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. The one or more communication processors may operate independently with respect to the processor (810). The transceiver (820) may include, for example, a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules can communicate with at least one image projection device (10), which is an external electronic device, via a network (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA, or a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as a plurality of separate components (e.g., multiple chips).

[0096] The user I / F (830) may include various circuits and be configured to receive information from a user. For example, the user I / F (830) may generate commands or data to be used in a component of the remote control device (30) (e.g., at least one processor (810)) in response to a user's operation. The user I / F (830) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0097] The user interface (830) may be configured to transmit information to the user. For example, the user interface (830) may output an audio signal to the outside. The user interface (830) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. The receiver may be implemented separately from the speaker or as part of the speaker.

[0098] According to one example, the processor (810) may detect the occurrence of an event that adjusts the viewpoint of viewing an image displayed on a projection area (e.g., projection area (30a, 30b) of FIG. 1) of a projection surface (e.g., projection surface (60) of FIG. 1) by manipulating a button (e.g., a physical button or a touch button) provided on the user I / F (830) by the user. The processor (810) may generate screen adjustment information in response to the generated event. The screen adjustment information may be control information including, for example, directional identification information indicating a direction in which the viewpoint is to be moved. The directional identification information may include an identifier indicating one of the forward, backward, up, down, left, or right directions. The screen adjustment information may be control information including, for example, location identification information indicating a viewpoint. The above screen adjustment information may be control information including, for example, identification information indicating at least one of the distance or direction between the image projection device (10) and the remote control device (30). The processor (810) may control the transceiver (820) to transmit the generated screen adjustment information to the image projection device (10).

[0099] FIG. 9 is a flowchart illustrating an exemplary method for performing image correction in an image projection device (e.g., the image projection device (10) of FIG. 1) according to one embodiment of the present disclosure.

[0100] Referring to FIG. 9, the image projection device (10) can acquire a plurality of samples in a projection area (e.g., the projection area (30a, 30b) of FIG. 1) in operation 910. When the projection surface (60) includes at least two surfaces that are bent at a predetermined angle, the samples may be three-dimensional samples having different separation distances at a specific point in time rather than a plane. The position of the three-dimensional sample may be defined by three-dimensional coordinates (x, y, z). For example, the image projection device (10) can measure the distance and direction of some or all of the projection points included in the projection area (30a, 30b) using a distance sensor (e.g., a ToF sensor). The image projection device (10) can acquire projection points corresponding to the three-dimensional samples based on the measured distance and direction.

[0101] FIG. 10 is a diagram including a table illustrating an example of obtaining a three-dimensional position sample of a projection point included in a projection area (e.g., projection area (30a, 30b) of FIG. 1) in an image projection device (e.g., image projection device (10) of FIG. 1) according to one embodiment of the present disclosure.

[0102] Referring to FIG. 10, the image projection device (10) senses distance and direction information of projection points corresponding to samples included in the projection area (30a and 30b) using a sensor (ToF sensor), and can define 3D coordinate values ​​(x, y, z) for each sample based on the sensed distance and direction information. As an example, the projection area (30a and 30b) is divided into two areas based on one corner (50). The number of samples may not be specified. For example, the number of samples may be adaptively applied in consideration of the number of surfaces included in the projection area (30a and 30b) and / or the curvature of the projection surface (60). The image projection device (10) can relatively increase the density of samples in a part in the projection area (30a and 30b) where a position measurement error is likely to occur.

[0103] The image projection device (10) can, in operation 920, classify and group a plurality of three-dimensional samples into at least two groups. The number of groups for the grouping can be determined based on a number that makes it easy to distinguish the positions of the plurality of three-dimensional samples. For example, if the projection area (30) includes two surfaces that are bent at a predetermined angle, the number of groups for the grouping can be two. In operation 920, the image projection device (10) can model a plurality of projection surfaces by classifying and grouping samples distributed at similar positions.

[0104] FIG. 11 includes a graph illustrating an example of grouping three-dimensional position samples corresponding to projection points included in a projection area (e.g., projection areas (30a, 30b) of FIG. 1) in an image projection device (e.g., image projection device (10) of FIG. 1), according to an embodiment of the present disclosure. FIG. 12 includes a graph illustrating an example of modeling faces included in a projection area (e.g., projection areas (30a, 30b) of FIG. 1) based on grouped three-dimensional position samples in an image projection device (e.g., image projection device (10) of FIG. 1), according to an embodiment of the present disclosure.

[0105] Referring to FIGS. 11 and 12, the image projection device (10) sets a reference value (e.g., 0) on the x-axis corresponding to the horizontal axis in the coordinate values ​​(x, y, z) of the acquired 3D samples (1100), and can classify first samples (1110) whose x-values ​​are less than the reference value, second samples (1120) whose x-values ​​exceed the reference value, and other ambiguous third samples (1130). The image projection device (10) can group the first samples (1110) into a first group (1210) corresponding to the first projection surface, the second samples (1120) into a second group (1220) corresponding to the second projection surface, and the third samples (1130) into a third group (1230) that cannot be classified into either the first or second projection surfaces. The first samples (1110) grouped into the first group (1210) will be included in the first projection surface. The second samples (1120) grouped into the second group (1220) will be included in the second projection surface. The image projection device (10) can model the number of projection surfaces included in the projection area (30a and 30b) as two.

[0106] The above image projection device (10) can, in operation 930, obtain the positions of projection points where each pixel of the output image it outputs meets the projection area (30a, 30b).

[0107] FIG. 13 is a graph showing a relationship between pixels of an image output from an image projection device (e.g., image projection device (10) of FIG. 1) and projection points to be projected on a projection area (e.g., projection area (30a, 30b) of FIG. 1) according to one embodiment of the present disclosure.

[0108] Referring to FIG. 13, the image projection device (10) can obtain a first coordinate value of a projection point (1321) corresponding to a position where a pixel (1311) included in an output image (1310) is to be projected onto the projection area (1320). The output image (1310) may be an uncorrected image. The pixels (1311) included in the output image (1310) may be pixels to be projected onto the projection area (1320) by the image projection device (10). Each of the projected pixels will be projected onto one of the projection points (1321) included in the projection area (1320). The above image projection device (10) can obtain first coordinate values ​​that define the positions of projection points (1321) where pixels (1311) included in the output image (1310) meet the projection area (1320).

[0109] The above image projection device (10) can, in operation 940, determine a second coordinate value corresponding to the first coordinate value in a coordinate system that allows viewing of the projection area (30a and 30b) from the user's viewpoint. The coordinate system that allows viewing of the projection area (30a and 30b) can be determined based on the user's viewpoint.

[0110] FIG. 14 is a graph showing a relationship between a pixel of an image output by an image projection device (e.g., an image projection device (10) of FIG. 1) at a viewing point and a projection point to be projected on a projection area (e.g., a projection area (30a, 30b) of FIG. 1) according to one embodiment of the present disclosure.

[0111] Referring to FIG. 14, the image projection device (10) can predict the viewpoint of the user (20), and determine a coordinate system (1410) corresponding to the arrangement of pixels of an expected output image when viewing the projection area (30a or 30b) based on the predicted viewpoint. The image projection device (10) can determine the second coordinate value of a projection point (1421) that will meet in the projection area (1420) when pixels (1411) in the determined coordinate system are projected.

[0112] The image projection device (10) can, in operation 950, obtain pixel values ​​for each projection point of the projection area (30a and 30b) based on the second coordinate values. The pixel values ​​for each projection point can be obtained from corresponding pixels in an input image. The pixels of the input image for which the pixel values ​​are to be obtained can be determined using the second coordinate values. In order to determine the pixels of the input image for which the pixel values ​​are to be obtained, the projection points corresponding to the second coordinate values ​​can be converted into coordinate values ​​on a two-dimensional plane.

[0113] FIG. 15 is a diagram including a graph illustrating an example of performing image correction in an image projection device (e.g., the image projection device (10) of FIG. 1) according to one embodiment of the present disclosure.

[0114] Referring to FIG. 15, the image projection device (10) can convert projection points included in a coordinate system (1510) based on the viewpoint of the user (20) into coordinate values ​​of a two-dimensional coordinate system (1520). The image projection device (10) can determine a pixel to be projected on a projection point (1521) included in the two-dimensional coordinate system (1520) in an output image (1540) that is an image to be projected (1501).

[0115] The above image projection device (10) can read the pixel value in the input image (1530) corresponding to the pixel to be projected (1503). The image projection device (10) can map the read pixel value to the pixel value of the pixel determined in the output image (1540) (1505). By the above-described operation, correction for the image to be projected can be completed by mapping the pixel value of the input image to all pixels of the output image.

[0116] The above image projection device (10) can output the output image generated by the pixel values ​​acquired for each projection point in operation 960 so that it can be projected onto the projection area (30a and 30b).

[0117] According to one embodiment, a computer-readable non-transitory storage medium may record a program for executing a method of switching an activation function in the image projection device (10) described above.

[0118] According to one embodiment, an image projection apparatus may include a transceiver, at least one memory, an image projector, and at least one processor operably connected to the transceiver, the at least one memory and / or the image projector, the processor including processing circuitry. The at least one processor may be individually and / or collectively configured to control the image projector to output a projected image to be projected on a projection region, the projection region including an image display region on which an image is to be substantially displayed, on a projection plane comprising at least two surfaces angled at a predetermined angle relative to an edge. The at least one processor may be individually and / or collectively configured to receive a control signal from a remote control apparatus via the transceiver for changing a view point of the projection region. At least one processor may be configured, individually and / or collectively, to update pixel values ​​of the to-be-projected image corresponding to projection points of the projection area in response to the control signal. Here, the pixel values ​​of the to-be-projected image may be updated such that the image to be displayed in the image display area appears to have a standardized shape like an initial image at the point in time. The initial image may be an image displayed in the image display area prior to the image to be projected.

[0119] According to one embodiment, at least one processor may be configured to, individually and / or collectively, control the image projector to project the initial image, which is corrected based on a position of the image projector, onto the image display area in response to an image output request.

[0120] According to one embodiment, the control signal may include directional identification information indicating the direction in which the viewpoint is to be moved.

[0121] According to one embodiment, the direction identification information may include an identifier indicating at least one of the forward, backward, up, down, left, or right directions.

[0122] According to one embodiment, the control signal may include location identification information indicating the point in time.

[0123] According to one embodiment, the control signal may include identification information indicating at least one of a distance and / or a direction between the image projection device and the remote control device.

[0124] According to one embodiment, at least one processor may be configured, individually and / or collectively, to predict the point in time based on the identification information.

[0125] According to one embodiment, at least one processor may be configured to, individually and / or collectively, obtain coordinate values ​​indicating positions of some or all of the projection points. At least one processor may be configured, individually and / or collectively, to model the at least two surfaces based on the obtained coordinate values. At least one processor may be configured, individually and / or collectively, to obtain first coordinate values ​​indicating positions of projection points on which the pixel values ​​of the image to be projected are to be projected on the at least two surfaces. At least one processor may be configured, individually and / or collectively, to convert the obtained first coordinate values ​​into second coordinate values ​​based on the viewpoint.

[0126] According to one embodiment, at least one processor may be configured, individually and / or collectively, to map the projection points onto a two-dimensional plane based on the second coordinate values ​​to determine the image display area. At least one processor may be configured, individually and / or collectively, to set pixel values ​​of the input image as target pixel values ​​to be projected onto projection points included in the image to be projected.

[0127] According to one embodiment, the at least one processor (710) may be configured, individually and / or collectively, to receive a sensing signal provided by at least one sensor installed in a real space via the transceiver. The at least one processor may be configured, individually and / or collectively, to identify a location of the image projection device using the sensing signal.

[0128] According to one embodiment, a first projection area including a first image display area in which a first image with updated pixel values ​​is projected and displayed on the projection surface may be substantially identical to a position of a second projection area including a second image display area in which a second image before the pixel values ​​are updated is projected and displayed on the projection surface.

[0129] According to one embodiment, a method of operating an image projection apparatus may be provided. The method may include an operation of outputting a projected image to a projection area, which includes an image display area on which an image is substantially displayed, on a projection plane including at least two surfaces that are inclined at a predetermined angle with respect to an edge. The method may include an operation of receiving a control signal for changing a view point of the projection area from a remote control apparatus (30). The method may include an operation of updating pixel values ​​of the projected image corresponding to projection points of the projection area in response to the control signal. Here, the pixel values ​​of the projected image may be updated such that the image to be displayed in the image display area appears to have a standardized shape like an initial image at the view point. The above initial image may be an image displayed in the image display area prior to the image to be projected.

[0130] According to one embodiment, the method may include an operation of projecting the initial image, which is corrected based on a position of the image projection device, onto the image display area in response to an image output request.

[0131] According to one embodiment, the control signal may include directional identification information indicating the direction in which the viewpoint is to be moved.

[0132] According to one embodiment, the direction identification information may include an identifier indicating at least one of the forward, backward, up, down, left, or right directions.

[0133] According to one embodiment, the control signal may include location identification information indicating the point in time.

[0134] According to one embodiment, the method may include an operation of predicting the point in time based on identification information indicating at least one of a distance and / or a direction between the image projection device and the remote control device included in the control signal.

[0135] According to one embodiment, the operation of outputting an image to be projected may include an operation of obtaining coordinate values ​​indicating positions of some or all of the projection points. The operation of outputting the image to be projected may include an operation of modeling the at least two surfaces based on the obtained coordinate values. The operation of outputting the image to be projected may include an operation of obtaining first coordinate values ​​indicating positions of projection points on which the pixel values ​​of the image to be projected are to be projected on the at least two surfaces. The operation of outputting the image to be projected may include an operation of converting the obtained first coordinate values ​​into second coordinate values ​​based on the viewpoint.

[0136] According to one embodiment, the method may include an operation of determining the image display area by mapping the projection points onto a two-dimensional plane based on the second coordinate values. The method may include an operation of setting pixel values ​​of the input image as target pixel values ​​to be projected onto projection points included in the image display area in the image to be projected.

[0137] According to one embodiment, the method may include an operation of receiving a sensing signal provided by at least one sensor installed in a real space. The method may include an operation of identifying a position of the image projection device using the sensing signal.

[0138] According to one embodiment, a first projection area including a first image display area in which a first image with updated pixel values ​​is projected and displayed on the projection surface may be substantially identical to a position of a second projection area including a second image display area in which a second image before the pixel values ​​are updated is projected and displayed on the projection surface.

[0139] Electronic devices according to various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, home appliances, or other devices. Electronic devices according to embodiments of this document are not limited to the aforementioned devices.

[0140] The various embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" can each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish the corresponding components from other corresponding components and do not limit the corresponding components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as being “coupled” or “connected” to another component (e.g., a second component), with or without the terms “functionally” or “communicatively,” that component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0141] The term "module" as used in various embodiments of this document may include a unit implemented using hardware, software, firmware, or any combination thereof, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or portion of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0142] Various embodiments of the present document may be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., a memory (730)) readable by a machine (e.g., an image projection device (10)). For example, a processor (e.g., the processor (710)) of the machine (e.g., the image projection device (10)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, a 'non-transitory storage medium' is a tangible device and may not contain signals (e.g., electromagnetic waves), but the term does not distinguish between cases where data is stored semi-permanently on the storage medium and cases where it is stored temporarily.

[0143] According to one embodiment, the methods according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0144] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0145] While the present disclosure has been illustrated and described with reference to various exemplary embodiments, it is to be understood that the various exemplary embodiments are illustrative and not limiting. Those skilled in the art will further appreciate that various modifications in form and detail are possible without departing from the spirit and scope of the present disclosure, including the appended claims and their equivalents. It is also to be understood that any embodiment(s) described in the present disclosure may be used in conjunction with other embodiments described in the present disclosure.

Claims

1. In an image projection apparatus (10), Transceiver (720); At least one memory (730) for storing instructions; Image projector (740); and At least one processor (710) operably connected to the above transmitter / receiver (720), the at least one memory (730) and / or the image projector (740), and including a processing circuit; The above instructions, when individually and / or collectively executed by at least one processor (710), cause the image projection device (10) to perform at least one operation, At least one of the above actions, An operation of controlling the image projector (730) to output a projected image to a projection area including an image display area where an image is substantially displayed on a projection surface including at least two surfaces bent at a predetermined angle based on an edge (50); An operation of receiving a control signal for changing the view point of the projection area from a remote control apparatus (30) through the above transmitter / receiver (720); and Including an operation of updating pixel values ​​of the image to be projected corresponding to projection points of the projection area in response to the control signal, Here, the pixel values ​​of the image to be projected are updated so that the image to be displayed in the image display area appears as a standardized shape of the initial image at the point in time, An image projection device (10), wherein the initial image includes an image displayed in the image display area prior to the image to be projected.

2. In paragraph 1, When the above instructions are individually or collectively executed by the at least one processor (210), the electronic device (100) causes: An image projection device (10) that causes an operation to be performed to control the image projector (730) to project the initial image corrected based on the position of the image projection device (10) onto the image display area in response to an image output request.

3. In paragraph 1, The above control signal includes directional identification information indicating the direction in which the point in time is to be moved and / or location identification information indicating the point in time, The above direction identification information is an image projection device (10) including an identifier indicating at least one of the forward, backward, up, down, left, or right directions.

4. In paragraph 1, The above control signal includes identification information indicating at least one of the distance and / or direction between the image projection device (10) and the remote control device (30), An image projection device (10), which causes the electronic device (100) to perform an operation of predicting the point in time based on the identification information when the above instructions are individually or collectively executed by the at least one processor (210).

5. In paragraph 1, When the above instructions are individually or collectively executed by the at least one processor (210), the electronic device (100) causes: An action of obtaining coordinate values ​​indicating the positions of some or all of the above projection points; An operation of modeling at least two surfaces based on the acquired coordinate values; An operation of obtaining first coordinate values ​​indicating the position of a projection point on which the pixel values ​​of the image to be projected are to be projected on at least two surfaces; An operation of converting the first coordinate values ​​obtained above into second coordinate values ​​based on the above point in time; An operation of determining the image display area by mapping the projection points onto a two-dimensional plane based on the second coordinate values; and An image projection device (10) that causes an operation to be performed to set pixel values ​​of an input image as target pixel values ​​to be projected on projection points included in the image display area of ​​the image to be projected.

6. In paragraph 1, When the above instructions are individually or collectively executed by the at least one processor (210), the electronic device (100) causes: An operation of receiving a sensing signal provided by at least one sensor installed in an actual space through the above transceiver (720); and An image projection device (10) that causes an operation of identifying a position of the image projection device (10) using the sensing signal.

7. In paragraph 1, An image projection device (10), wherein a first projection area including a first image display area in which a first image with updated pixel values ​​is projected and displayed on the projection surface (10) is substantially identical in position to a second projection area including a second image display area in which a second image before the pixel values ​​are updated is projected and displayed on the projection surface (10).

8. In the operating method of the image projection apparatus (10), An operation of outputting an image to be projected on a projection area including an image display area where an image is actually displayed on a projection surface including at least two surfaces bent at a predetermined angle based on an edge (50); An operation of receiving a control signal for changing the view point of the projection area from a remote control apparatus (30); and Including an operation of updating pixel values ​​of the image to be projected corresponding to projection points of the projection area in response to the control signal, Here, the pixel values ​​of the image to be projected are updated so that the image to be displayed in the image display area appears as a standardized shape of the initial image at the point in time, A method wherein the initial image is an image displayed in the image display area prior to the image to be projected.

9. In paragraph 8, A method comprising an operation of projecting the initial image, which is corrected based on the position of the image projection device (10), onto the image display area in response to an image output request.

10. In paragraph 8, The above control signal includes directional identification information indicating the direction in which the point in time is to be moved and / or location identification information indicating the point in time, A method wherein the above direction identification information includes an identifier indicating at least one of the forward, backward, up, down, left, or right directions.

11. In paragraph 8, A method including an operation of predicting the point in time based on identification information indicating at least one of the distance or direction between the image projection device (10) and the remote control device (30) included in the control signal.

12. In paragraph 8, The operation of outputting the image to be projected above is: An action of obtaining coordinate values ​​indicating the positions of some or all of the above projection points; An operation of modeling at least two surfaces based on the acquired coordinate values; An operation of obtaining first coordinate values ​​indicating the position of a projection point on which the pixel values ​​of the image to be projected are to be projected on at least two surfaces; and A method including an operation of converting the first coordinate values ​​obtained above into second coordinate values ​​based on the above point in time.

13. In paragraph 12, An operation of determining the image display area by mapping the projection points onto a two-dimensional plane based on the second coordinate values; and A method comprising an operation of setting pixel values ​​of an input image as target pixel values ​​to be projected at projection points included in the image display area of ​​the image to be projected.

14. In paragraph 8, An operation of receiving a sensing signal provided by at least one sensor installed in a real space; and A method comprising an operation of identifying a position of the image projection device (10) using the sensing signal.

15. In paragraph 8, A method in which a first projection area including a first image display area in which a first image with updated pixel values ​​is projected and displayed on the projection surface (60) is substantially the same as a second projection area including a second image display area in which a second image before the pixel values ​​are updated is projected and displayed on the projection surface (60).

Images