Image projection apparatus, method, and recording medium

Abstract

The present disclosure relates to an image projection apparatus, method and recording medium for displaying an image on a projection surface. The image projection apparatus may comprise: at least one distance sensor; an image projector; at least one memory including a non-volatile recording medium for storing instructions; and at least one processor including a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the image projection apparatus to perform at least one operation. The at least one operation may comprise an operation of determining a luminance change coefficient for an input pixel for a gradation conversion model and an output pixel corresponding thereto. The at least one operation may comprise an operation of obtaining a projection surface deviation correction coefficient reflecting the characteristics of a projection surface that are obtained from the at least one distance sensor. The at least one operation may comprise an operation of generating an output image to be projected onto a projection area of the projection surface by the image projector, by correcting the luminance of a pixel of an input image in consideration of at least one of the luminance change coefficient or the projection surface deviation correction coefficient.

Description

Image projection device, method, and recording medium

[0001] The present disclosure relates to an image projection device, method, and recording medium for displaying an image on a projection surface.

[0002] The video device may be an analog projection device (hereinafter referred to as an “analog projection device”) or a digital projection device (hereinafter referred to as a “digital projection device”). An analog projection device may provide visual information using a medium such as film. A digital projection device may be a projection device that provides visual information using digital signals. Digital projection devices may include a beam projector (hereinafter referred to as a “projector”). A projector may be classified as a display device. Depending on the principle of generating light, a projector may be classified into a CRT (Cathode Ray Tube) projector, an LCD (Liquid Crystal Display) projector, or a DLP (Digital Light Processing) projector.

[0003] Projectors are primarily used to display multimedia content input directly into the projector. When connected to an electronic device (e.g., a digital TV) via a wired or wireless communication network, the projector can display the multimedia content received from the electronic device.

[0004] A projector can be an electronic device that can project photographs, drawings, text, images, or videos onto a screen through a lens. A projector is called an image projection apparatus and can convert data (in the form of a file) regarding an image or video into an optical signal (or optical image) and output it. The output of the optical signal can correspond to irradiation. The output image displayed on the screen by the image projection apparatus may have uneven brightness due to the characteristics of the projection surface or the characteristics of the lens that radiates the optical signal.

[0005] The above information may be provided as background information to aid in understanding this document. None of the above is claimed to be prior art related to this document or can be used to determine prior art.

[0006] According to one embodiment, an image projection device may include at least one distance sensor. The image projection device may include an image projector. The image projection device may include at least one memory including a non-volatile recording medium storing instructions. The image projection device may include at least one processor operatively connected to the at least one distance sensor (420), the image projector (440), and the at least one memory, the processor including a processing circuit. When the instructions are individually or collectively executed by the at least one processor, the instructions may cause the image projection device to perform at least one operation. The at least one operation may include determining a brightness variation coefficient for an input pixel for a grayscale transformation model and an output pixel corresponding to the input pixel. The at least one operation may include obtaining a projection surface deviation correction coefficient in which a feature of the projection surface obtained from the at least one distance sensor is reflected. The at least one operation may include an operation of correcting the brightness of a pixel of an input image by taking into account at least one of the brightness change coefficients to generate an output image to be projected by the image projector onto a projection area of ​​the projection surface.

[0007] According to one embodiment, a method of operating an image projection apparatus may include an operation of determining a brightness variation coefficient for an input pixel for a grayscale transformation model and an output pixel corresponding to the input pixel. The method may include an operation of obtaining a projection surface deviation correction coefficient in which a characteristic of a projection surface is reflected, obtained from at least one distance sensor. The method may include an operation of correcting the brightness of a pixel of an input image by considering at least one of the brightness variation coefficient and the projection surface deviation correction coefficient, thereby generating an output image to be projected by the image projector on a projection area of ​​the projection surface.

[0008] According to one embodiment, computer-readable instructions stored on a recording medium, when executed by at least a portion of at least one processor included in an image projection device, may cause the image projection device to perform at least one operation. The at least one operation may include determining a brightness variation coefficient for an input pixel for a grayscale transformation model and an output pixel corresponding to the input pixel. The at least one operation may include obtaining a projection surface deviation correction coefficient reflecting a characteristic of a projection surface obtained from at least one distance sensor. The at least one operation may include correcting a brightness of a pixel of an input image by taking into account at least one of the brightness variation coefficient and the projection surface deviation correction coefficient, thereby generating an output image to be projected by the image projector onto a projection area of ​​the projection surface.

[0009] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0010] FIG. 1 is a drawing for explaining an operation of projecting an image on a projection surface in an image projection system according to one embodiment.

[0011] FIG. 2 is a drawing for explaining an example of an image being projected on a projection surface in an image projection system according to one embodiment.

[0012] FIG. 3 is a drawing for explaining correction of brightness deviation in an image projection device according to one embodiment.

[0013] FIG. 4 is a block diagram for projecting image data in an image projection device according to one embodiment.

[0014] FIG. 5 is a control flowchart for obtaining position data of an area on which image data is to be projected in an image projection device according to one embodiment.

[0015] Figure 6a is a diagram for explaining brightness deviation compensation that increases the brightness of input pixels using a grayscale conversion model.

[0016] Figure 6b is a diagram for explaining brightness deviation compensation that lowers the brightness of input pixels using a grayscale conversion model.

[0017] Figure 6c is a drawing for explaining brightness correction using a grayscale conversion model.

[0018] FIG. 7a and FIG. 7b are drawings for explaining obtaining an optical deviation correction coefficient by taking into account the vignetting characteristics of the lens.

[0019] Figure 8a is a drawing for explaining that brightness deviation occurs on a non-planar projection surface depending on the incident angle of the projection beam.

[0020] Figure 8b shows the second projection surface deviation correction coefficient (G) obtained by considering the incident angle (θ) for each sensing measurement point. θ ) is a drawing to explain how to compensate for brightness deviation.

[0021] Figure 8c shows the first projection surface deviation correction coefficient (G) obtained by considering the reaching distance (d) for each sensing measurement point. d ) is a drawing to explain how to compensate for brightness deviation.

[0022] FIG. 9 is a diagram for explaining an operation of generating an output image by performing brightness compensation and / or brightness deviation compensation by applying weights to an input image.

[0023] FIG. 10 is a block diagram of an electronic device within a network environment according to one or more embodiments.

[0024] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0025] FIG. 1 is a drawing for explaining an operation of projecting an image onto a projection surface (110) in an image projection system (10) according to one embodiment, and FIG. 2 is a drawing for explaining an example of an image being projected onto a projection surface (110) in an image projection system (10) according to one embodiment.

[0026] In FIGS. 1 and 2, it is assumed that the projection surface (110) is non-planar, but this is only an example, and it is obvious that various embodiments proposed in the present disclosure can be equally applied to a projection surface (110) that is planar or multi-faceted.

[0027] Referring to FIG. 1 or FIG. 2, an image projection system (10) may include an image projection device (100) (e.g., a beam projector) or a projection plane (110). The projection plane (110) may correspond to, for example, a white or silver film for projecting an optical signal on a screen. The projection plane (110) may be, for example, a flat or multi-faceted surface, such as a wall, onto which an optical signal may be projected, or may be a non-flat surface, such as a curtain.

[0028] As an example, the projection surface (110) may be a non-planar curved surface having a predetermined curvature characteristic. The projection surface (110) may be referred to as a 'curved projection surface'. The curved projection surface is a projection surface that may be formed by, for example, a curtain, a tent, or a banner. Here, the 'projection surface' may be used to refer to any one of a planar projection surface, a multi-faceted projection surface, or a non-planar projection surface. In the drawings or detailed description of the present disclosure, a non-planar projection surface is assumed as an example, and thus the 'projection surface' will be regarded as a 'non-planar projection surface', but various embodiments of the present disclosure are not limited thereto.

[0029] The curvature characteristic on a non-planar projection surface (110) may be a characteristic related to the shape in which the projection surface (110) is bent or curved. The curvature characteristic may include, for example, a wave characteristic formed by crests and roots in a certain direction, such as horizontal (or left-right), vertical (or up-down), or diagonal. In this case, the curvature characteristic may have a slope due to the formation of crests and valleys. The slope may have a predetermined inclination.

[0030] The projection surface (110) may include a projection region (120). The projection region (120) may be, for example, an area where projection beams corresponding to optical signals transmitted by the image projection device (100) from the projection surface (110) can reach. The projection region (120) may have curvature characteristics that are substantially the same as or similar to those of the projection surface (110). The projection region (120) may include pixel projection points where the projection beams will reach. An image corresponding to a specific pixel may be displayed at a pixel projection point by the corresponding projection beam. A pixel is an element that is the smallest unit that constitutes an image to be displayed on the projection region (120) and may have a predetermined unit area. A pixel may have a color value including R (red), G (green), and B (blue), which are subpixels corresponding to the three elements of light that actually represent color, and / or a luminance value indicating brightness.

[0031] The projection area (120) may include a non-screen display area and a screen display area (130). The non-screen display area may be an area where pixel projection points where projection beams reach are distributed, but where no actual image is displayed. For example, pixels displayed by projection beams projected on the non-screen display area may not have a color value. The screen display area (130) may be an area where an actual image is displayed by projection beams that reach pixel projection points. For example, pixels displayed by projection beams projected on the screen display area (130) may have specific color values ​​and specific luminance values. In other words, the screen display area (130) may be an area where an image is actually displayed by projection beams transmitted to the projection area (120) by the image projection device (100). The projection area (120) or the screen display area (130) may have curvature characteristics that are substantially the same as or similar to those of the projection surface (110).

[0032] The image projection device (100) can generate output image data (hereinafter, referred to as “output image”) by correcting input image data (hereinafter, referred to as “input image”). According to one example, the image projection device (100) can perform brightness deviation correction on the input image to generate an output image with corrected brightness deviation. For example, the image projection device (100) can perform brightness correction on the input image to improve the contrast ratio of the output screen. For example, the image projection device (100) can perform brightness correction to increase the brightness of pixels with high brightness among the pixels of the input image and decrease the brightness of pixels with low brightness so that the outline of the image can be made clear in the output image. For example, the image projection device (100) can perform brightness deviation correction on the input image by taking into account the characteristics of the projection lens that may cause optical deviation and / or the characteristics of the projection surface with respect to the shape and / or curvature of the projection surface (110). A detailed description of this will be provided later. In addition to brightness deviation correction, the image projection device (100) can perform distortion correction to compensate for image distortion. For example, if the projection surface (110) is flat, such as a screen, the image projection device (100) can perform flat distortion correction, such as keystone correction, on the input image to generate an output image. For example, if the projection surface (110) is multi-faceted, the image projection device (100) can perform multi-faceted distortion correction on the input image to generate an output image. For example, if the projection surface (110) is non-flat, such as a curved surface, the image projection device (100) can perform non-flat distortion correction on the input image to generate an output image. That is, the image projection device (100) can perform automatic correction (auto keystone) on the input image so that the image displayed on the screen display area (130) can appear as a flat image without distortion by taking into account the curvature characteristics of the projection surface (110).The application of automatic correction to display an undistorted image like a planar image on a projection surface (110) having curvature characteristics corresponding to a non-planar surface in an image projection device (100) may not be directly related to at least one embodiment of the present disclosure, and therefore, a detailed description thereof will be omitted.

[0033] For example, the image projection device (100) may convert a brightness deviation-corrected output image into an optical signal and transmit a plurality of projection beams toward a projection surface (110). The input image may be image data input according to a content service such as a movie or a game, for example. The optical signal transmitted by the image projection device (100) may be projected onto a projection area (120) of the projection surface (110).

[0034] For example, the image projection device (100) can correct the output image to reflect the viewpoint of the viewer (125). For example, the image projection device (100) can generate an output image by performing brightness deviation correction on the input image in consideration of the viewpoint of the viewer (125), thereby providing a screen with an optimized contrast ratio to the viewer (125).

[0035] FIG. 3 is a drawing for explaining correction of brightness deviation in an image projection device (e.g., image projection device (100) of FIG. 1) according to one embodiment.

[0036] Although FIG. 3 assumes that the projection surface (e.g., the projection surface (110) of FIG. 1 or 2) is non-planar, brightness deviation may also occur in the output screen displayed on a projection surface of a planar or multi-faceted type, and the brightness deviation correction method of the present disclosure can be applied to this as well. In addition, the brightness deviation correction method to be proposed in the present disclosure will be described assuming an output screen to be displayed on the projection surface (110) by the image projection device (100), but this is only an example, and may also be partially applied to electronic devices (e.g., notebook PCs, desktop PCs, tablet devices, smartphones, televisions, set-top boxes) that have their own display or can be connected to a monitor based on a specific wired / wireless communication standard. However, since electronic devices that display an output screen through a display such as a monitor are not based on a projection method, only a method of defining a grayscale conversion model in advance and correcting the brightness of an input pixel based on the model can be applied.

[0037] Referring to FIG. 3, the first screen (310) is an output screen displayed in a screen display area (e.g., a screen display area (130) of FIG. 1 or 2) by projecting an output image (e.g., an output image (403) of FIG. 4) generated without brightness correction (330) (hereinafter, referred to as 'brightness deviation correction (330)') for an input image (e.g., an input image (401) of FIG. 4) onto a projection area (e.g., a projection area (120) of FIG. 1 or 2). Brightness deviation may refer to the degree to which brightness of each area of ​​the output screen is uneven due to loss caused by optical deviation and / or projection surface deviation. Brightness correction refers to adjusting the brightness of some or all pixels of the input image in order to obtain a desired contrast ratio on the output screen. Brightness deviation compensation, which is one of the brightness compensation methods, means adjusting the brightness by pixel or by partial area (hereinafter referred to as 'partial display area') (e.g., the first to second partial display areas (731, 733, 735, 737) of FIG. 7a) so that the output screen has uniform brightness overall by compensating for brightness loss that may occur on the output screen in advance.

[0038] In the first screen (310), since the brightness deviation correction (330) for the input image (401) is not performed, the brightness may not be consistent and an uneven brightness deviation phenomenon may occur overall. This brightness deviation phenomenon may occur because the inherent characteristics (e.g., projection lens characteristics) of the projection beams transmitted by the image projection device (100) are not the same, and / or the characteristics (e.g., projection surface characteristics) of the projection surface (110) on which the projection beams transmitted by the image projection device (100) are to be projected are not the same. The projection lens characteristics may be, for example, inherent optical characteristics of the image projection device (100), such as a lens vignetting phenomenon. Items representing the projection surface characteristics may include, for example, the distance that the projection beams transmitted by the image projection device (100) travel to reach the projection surface (110) (hereinafter, referred to as 'reach distance (d)'). The reach of the projection beams can be defined as the distance from the image projection device (100) to the point on the projection surface (110) where the corresponding projection beams are to be projected. An item representing the characteristics of the projection surface may be, for example, the angle of incidence on the projection surface (110) due to the projection direction of the projection beams transmitted by the image projection device (100).

[0039] For example, the reach of the projection beams may be influenced by the position of the point where the projection beam is to be projected on the projection surface (110) (or the screen display area (130)). For example, the reach of a projection beam whose projection point is near the center of the screen display area (130) may be relatively shorter than the reach of a projection beam whose projection point is near the edge of the screen display area (130). A projection beam with a short reach may incur relatively less loss in brightness than a projection beam with a long reach. As a result, a predetermined brightness deviation may occur between pixels displayed near the center of the screen display area (130) and pixels displayed near the edge of the screen display area (130).

[0040] For example, when the projection surface (110) is a non-planar surface having a predetermined curvature characteristic, the arrival distance of the projection beams may be affected by the predetermined curvature characteristic of the projection surface (110). The curvature characteristic may be a characteristic related to the shape in which the projection surface (110) is bent or curved. The curvature characteristic of the projection surface (110) may include, for example, a wave characteristic formed by crests and roots in a certain direction, such as horizontal (or left-right), vertical (or up-down), or diagonal direction. In this case, the curvature characteristic may have a slope due to the formation of crests and valleys. The slope may have a predetermined inclination. In this case, the arrival distance of the projection beam whose projection point is near the crest of the screen display area (130) may be relatively short compared to the arrival distance of the projection beam whose projection point is near the valley of the screen display area (130). A projection beam with a short reach may experience relatively less brightness loss than a projection beam with a long reach. As a result, a certain brightness difference may occur between pixels displayed near the top of the screen display area (130) and pixels displayed near the bottom of the screen display area (130).

[0041] For the reasons described above, brightness deviations may occur in the first screen (310), with the brightness being displayed relatively bright or relatively dark depending on the display area.

[0042] The second screen (320) is an output screen displayed in the screen display area (130) by projecting an output image (403) generated by performing brightness deviation correction (330) on the input image (401) onto the projection area (120). In the second screen (320), brightness can be expressed uniformly overall by performing brightness deviation correction (330) on the input image (401). That is, it can be confirmed that brightness deviation does not substantially occur in the second screen (320).

[0043] According to an example, the image projection device (100) improves the contrast ratio of the second screen (320) by adjusting the brightness of the input pixel (hereinafter referred to as 'input pixel brightness (B) in (i))', where i is a pixel index) to correct the brightness of the output pixel (hereinafter referred to as 'output pixel brightness (B out (i)'reference, where i is a pixel index) can be determined. The image projection device (100) can perform contrast ratio improvement to brighten the bright part of the output image and darken the dark part of the output image so that the outline of the image can be clearly seen in the output image.

[0044] According to one example, the image projection device (100) applies a weight (e.g., a brightness change coefficient (δ)) to input pixels to be projected on a display area (e.g., near the edge or valley of the screen display area (130)) that was darkened on the first screen (310). curve ), optical deviation correction factor (G Φ), projection surface deviation correction coefficient (G)) can be determined to be relatively higher than the weights (e.g., brightness change coefficient, optical deviation correction coefficient, projection surface deviation correction coefficient) to be applied to the input pixels to be projected on the display area (e.g., near the center or near the floor of the screen display area (130)) that were brightly displayed on the first screen (310). For example, the image projection device (100) can determine the weights (e.g., brightness change coefficient, optical deviation correction coefficient, projection surface deviation correction coefficient) to be applied to the pixels to be projected on the display area (e.g., near the center or near the floor of the screen display area (130)) that were brightly displayed on the first screen (310) to be relatively lower than the weights (e.g., brightness change coefficient, optical deviation correction coefficient, projection surface deviation correction coefficient) to be applied to the pixels to be projected on the display area (e.g., near the edge or near the valley of the screen display area (130)) that were darkly displayed on the first screen (310). The image projection device (100) can perform brightness deviation correction (330) on input pixels based on weights determined for each input pixel. In the second screen (320), brightness can be made consistent overall by performing brightness deviation correction (330) on the input image (401).

[0045] FIG. 4 is a block diagram for projecting image data in an image projection device (e.g., the image projection device (100) of FIG. 1) according to one embodiment.

[0046] Referring to FIG. 4, the image projection device (100) may include at least one processor (410) (hereinafter, referred to as 'processor (410)'), at least one sensor, at least one memory (430) (hereinafter, referred to as 'memory (430)'), or an image projector (440). The at least one sensor may include a distance sensor (420). The distance sensor (420) may be a time of flight (ToF) sensor, a depth camera, or a ToF camera.

[0047] The distance sensor (420) can obtain position data corresponding to a plurality of sensing measurement points included in a projection surface (e.g., the projection surface (110) of FIG. 1) and / or a projection area (e.g., the projection area (120) of FIG. 1). The sensing measurement points can be distributed on the projection surface (110) and / or the projection area (120). The sensing measurement points can be distributed regularly or irregularly on the projection surface (110) and / or the projection area (120). The sensing measurement points can be points where projection beams transmitted by the distance sensor (420) reach the projection surface (110) and / or the projection area (120). For example, the distance sensor (420) can obtain position data of the corresponding sensing measurement points by receiving IR signals that are reflected from the sensing measurement points when the transmitted IR signal is received. The position data may include a space orthogonal coordinate (or three-dimensional (3D) orthogonal coordinate system) (hereinafter referred to as a 'space orthogonal coordinate system') corresponding to the position of each sensing measurement point in a coordinate space (3D). As an example, the space orthogonal coordinate system corresponding to the position of each sensing measurement point may be referred to as a 'coordinate value (P(x,y,z))'. The projection surface (110) may include, for example, about 250 sensing measurement points. In this case, the position data may include about 250 coordinate values.

[0048] For example, when the projection surface (110) is non-planar, the sensing measurement points may be irregularly distributed on the projection surface (110) and / or the projection area (120). For example, assuming a distance sensor (420) that transmits beams so that the sensing measurement points are evenly distributed on the projection surface of the plane, the sensing measurement points that the projection beams transmitted by the distance sensor (420) will reach may be distributed in proportion to the slope of the projection surface (110) that is non-planar. That is, the sensing measurement points that exist in an area with a steep slope (hereinafter, referred to as a 'first slope') on the non-planar projection surface (110) may be distributed at relatively wider intervals than the sensing measurement points that exist in an area with a relatively mild slope (hereinafter, referred to as a 'second slope'). Therefore, the density of sensing measurement points on the first slope may be relatively lower than the density of sensing measurement points on the second slope. The first slope may be distinguished based on the difference in the degree of inclination relative to the second slope due to the curvature of the projection surface (110), which may be an exemplary assumption.

[0049] At least one sensor may provide sensing data related to the position of the image projection device (100) and / or the position of a viewer (e.g., the viewer (125) of FIG. 1). The sensing data acquired by the at least one sensor may include information to be used to acquire the position of the viewer (125). The processor (410) may predict the point in time at which the viewer (125) looks at the projection surface (110) by considering the position of the viewer (125) acquired based on the sensing data. The processor (410) may identify the position of the image projection device (100) based on the sensing data.

[0050] The memory (430) can store various data used by at least one component (e.g., the processor (410) or the distance sensor (420)) of the image projection device (100). The data can include, for example, software (e.g., a program) and input data or output data for commands related thereto. The memory (430) can include volatile memory or non-volatile memory. The program can be stored as software in the memory (430), for example. In one example, the memory (430) can include an operating system, middleware, or an application.

[0051] The memory (430) can store data for brightness deviation correction. The memory (430) can store data for correcting brightness deviation of an output image due to contrast ratio improvement, optical deviation, and / or projection surface deviation in an image projection device (100), for example.

[0052] According to an example, the memory (430) may include one or more grayscale conversion models for improving the contrast ratio of the output image (e.g., see FIG. 6c). One or more grayscale conversion models may be stored in advance in the memory (430) during the production process of the image projection device (100). The grayscale conversion model may be used to determine the brightness of an input pixel (hereinafter, referred to as 'input pixel brightness (B in (i))', where i is the pixel index) and the brightness change coefficient (δ) to correct the brightness for contrast ratio improvement. curve )(Example: The first brightness change coefficient (δ) of Fig. 6a curve1 )(630) or the second brightness change coefficient (δ) of Fig. 6b curve2)(660)) can be defined (e.g., see FIG. 6a or FIG. 6b). For example, the memory (430) can store a grayscale conversion model for improving the contrast ratio by considering the characteristics of the projection lens. For example, the memory (430) can store grayscale conversion models to be selectively applied in response to a content type (e.g., movie mode, game mode, or sports mode). For example, the memory (430) can store grayscale conversion models to be selectively applied in response to an image output mode (e.g., natural image, soft image, bright image, dark image). The brightness change coefficient (δ) of the grayscale conversion model stored in the memory (430) curve ) can be adjusted in response to the contrast ratio of the output image changed by the user. A new grayscale conversion model can also be registered in the memory (430) according to the user's request.

[0053] According to an example, the memory (430) can store data regarding the characteristics of a projection lens that may cause optical deviation in the image projection device (100). The data regarding the characteristics of the projection lens can include at least one look-up table (LUT). The LUT includes an optical deviation correction coefficient (G) for correcting optical deviation due to the characteristics of the projection lens for each pixel of the input image. Φ ) can be defined. For example, the memory (430) may include at least one LUT for correcting brightness deviation due to vignetting characteristics of the lens (e.g., see FIG. 7a or FIG. 7b) for each pixel. For example, the LUT may include an optical deviation correction coefficient (G) for correcting optical deviation due to vignetting characteristics for each partial display area of ​​the input image (e.g., the first to second partial display areas (731, 733, 735, 737) of FIG. 7a). Φ) can be defined. For example, the LUT may define an optical deviation correction factor (G) to correct the optical deviation for each partial display area (731, 733, 735, 737) where optical deviation due to vignetting characteristics occurs in the output image. Φ ) can be defined. The optical deviation correction coefficient (G) of the LUT stored in the memory (430) Φ ) may be changed or adjusted according to the user's request.

[0054] According to an example, a new LUT may be registered in the memory (430) according to a user's request. For example, the image projection device (100) may check whether a change in the characteristics of the projection lens has occurred periodically (e.g., a period for checking the characteristics of the projection lens) and / or aperiodically (e.g., a request for checking the characteristics of the projection lens by the user). As an example, the image projection device (100) (or the processor (410) transmits a test image (e.g., a single-tone image) to the projection surface (110), and calculates an optical deviation correction coefficient (G) for a plurality of pixels included in the output image displayed on the projection surface (110). Φ ) can be obtained. For example, the image projection device (100) (or processor (410) can obtain an optical deviation correction coefficient (G) for all pixels included in the output image. Φ ) can be obtained. For example, the image projection device (100) (or processor (410) may obtain an optical deviation correction coefficient (G) for specific pixels distributed by partial display areas (731, 733, 735, 737) that divide the screen display area (130) where the output image is displayed. Φ ) can be obtained. In this case, the image projection device (100) (or processor (410) can obtain an optical deviation correction coefficient (G) for each partial display area (731, 733, 735, 737) Φ ) can be obtained. The image projection device (100) (or processor (410) obtains an optical deviation correction coefficient (G) for each pixel or each partial display area. Φ) can generate a new LUT. The memory (430) can store a new LUT generated by the image projection device (100) (or the processor (410).

[0055] According to one example, the memory (430) may store data regarding the characteristics of the projection surface. The data regarding the characteristics of the projection surface may include a projection surface deviation correction coefficient (G). The data regarding the characteristics of the projection surface may include position data corresponding to a plurality of sensing measurement points acquired by the distance sensor (420). The position data may include, for example, a spatial Cartesian coordinate system (e.g., a coordinate value (P(x,y,z))) corresponding to the position of each of the sensing measurement points in the coordinate space. The position data may include, for example, a reach distance (d) for each sensing measurement point. The position data may include, for example, a first projection surface deviation correction coefficient (G) acquired in consideration of the reach distance (d) for each sensing measurement point. d ) may be included. Data on the characteristics of the projection surface are the incident angle (θ) corresponding to the projection beam at the sensing measurement point. i ) may include (e.g., see Fig. 8b). For example, the angle of incidence (θ i ) can be defined by the angle between the vector of the projection beam (hereinafter referred to as the 'beam vector') and the normal vector at the corresponding sensing measurement point. Data on the characteristics of the projection surface can be defined by the incident angle (θ) for each projection beam at the sensing measurement points. i ) obtained by considering the second projection surface deviation correction coefficient (G) θ ) may be included.

[0056] The image projector (440) can convert the output image (403) for which brightness deviation compensation has been performed by the processor (410) into an optical signal (405) to be projected onto the projection area (120) of the projection surface (110). The image projector (440) can, for example, convert the output image (403), which is an electrical signal provided from the processor (410), into an optical signal (405) and transmit it toward the projection area (120). The output image (403), which is an electrical signal provided by the processor (410), can correspond to image data such as a photograph or a video. The optical signal (405) projected by the image projector (440) can display an output screen on a screen display area (e.g., a screen display area (130) of FIG. 1) included in the projection area (120). For example, the optical signal (405) transmitted by the image projector (440) may be composed of projection beams. The projection beams may, for example, correspond to each pixel constituting the output image (403). Accordingly, the projection beams may be projected onto pixel projection points distributed in the projection area (120) to display pixels constituting the output screen. The pixel projection points may be points where the projection beams transmitted by the image projector (100) reach the projection area (120). For example, among the projection beams projected onto the pixel projection points in the projection area (120), only some of the projection beams projected onto the screen display area (130) may have color values ​​to actually display an image.

[0057] The processor (410) may execute software to control at least one other component (e.g., hardware or software component) electrically connected to the processor (410), such as a distance sensor (420) or an image projector (440), or to perform processing or calculation of various data. As at least a part of data processing or calculation, the processor (410) may store commands or data received from other components (e.g., a distance sensor (420), a user I / F, or a transceiver) in a memory (430) (e.g., a volatile memory), or process commands or data stored in the memory (430) and store processed result data in the memory (430).

[0058] The processor (410) may be implemented as one or more integrated circuit (IC) chips and may perform various data processing operations. For example, the processor (410) (or application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or chipset). The processor (410) may include sub-components including a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a display controller, a memory controller, a storage controller, a communication processor (CP), and / or a sensor interface. The sub-components are merely exemplary. For example, the processor (410) may further include other sub-components. For example, some sub-components may be omitted from the processor (410). For example, some sub-components may be included as separate components of the image projection device (100) outside the processor (410). For example, some subcomponents may be contained within other components (e.g., a display, an image sensor).

[0059] The processor (410) (e.g., CPU or central processing circuit) may be configured to control sub-components based on the execution of instructions stored in the memory (430) (e.g., volatile memory and / or non-volatile memory). In one example, the GPU (or graphics processing circuit) included in the processor (410) may be configured to execute parallel operations (e.g., rendering). In one example, the NPU (or neural processing circuit) included in the processor (410) may be configured to execute operations for an artificial intelligence model (e.g., convolution computation). In one example, the ISP (or image signal processing circuit) included in the processor (410) may be configured to process a raw image (401) acquired through an image sensor into a format suitable for a component in the image projection device (100) or a sub-component in the processor (410). In one example, a display controller (or display control circuit) included in the processor (410) may be configured to process an image (401) acquired from a CPU, a GPU, an ISP, or a memory (430) (e.g., a volatile memory) into a format suitable for projecting onto a projection surface (e.g., the projection surface (110) of FIG. 1 or 2). In one example, a memory controller (or memory control circuit) included in the processor (410) may be configured to control reading data from a volatile memory and writing data to the volatile memory. In one example, a storage controller (or storage control circuit) included in the processor (410) may be configured to control reading data from a nonvolatile memory and writing data to the nonvolatile memory.In one example, the CP (communication processing circuit) included in the processor (410) may be configured to process data acquired from a sub-component within the processor (410) into a format suitable for transmission to another electronic device via a transceiver (not shown), or to process data acquired from another electronic device (e.g., a remote controller) via a transceiver into a format suitable for processing by the sub-component. In one example, the sensor interface (or sensing data processing circuit, sensor hub) included in the processor (410) may be configured to process data on the status of the image projection device (100) and / or the status of the surroundings of the image projection device (100), acquired via an internal sensor (e.g., a distance sensor (time-of-flight sensor) (420)) or an external sensor (e.g., one or more position measurement sensors (anchors)), into a format suitable for the sub-component within the processor (410).

[0060] The processor (410) can perform brightness correction (e.g., brightness deviation correction (330) of FIG. 3) on the input image (401) to generate an output image (403). The processor (410) can transmit the output image (403) to the image projector (440). The processor (410) can control the image projector (440) to convert the output image (403) into an optical signal (405) and project it toward the projection surface (110). Since the optical signal (405) transmitted by the image projector (440) has undergone brightness correction, the brightness of the output screen displayed in the screen display area (130) can be expressed uniformly overall. For example, the processor (410) can perform brightness correction on the input image to improve the contrast ratio of the output screen. For example, the processor (410) may perform brightness correction to increase the brightness of pixels with high brightness among the pixels of the input image and decrease the brightness of pixels with low brightness so that the outline of the image in the output image can be made clear. The processor (410) may perform brightness deviation correction for the input image by taking into account the characteristics of the projection lens that may cause optical deviation and / or the characteristics of the projection surface regarding the shape and / or curvature of the projection surface (110).

[0061] For example, the processor (410) may determine a weight (e.g., a brightness change coefficient, an optical deviation correction coefficient, a projection surface deviation correction coefficient) to be applied to input pixels to be projected on a display area that may be displayed dark on the output screen (e.g., near the edge or near the valley of the screen display area (130)) to be relatively higher than a weight (e.g., a brightness change coefficient, an optical deviation correction coefficient, a projection surface deviation correction coefficient) to be applied to input pixels to be projected on a display area that may be displayed bright on the output screen (e.g., near the center or near the floor of the screen display area (130)).

[0062] For example, the processor (410) may determine a weight (e.g., a brightness change coefficient, an optical deviation correction coefficient, a projection surface deviation correction coefficient) to be applied to input pixels to be projected on a display area that may be displayed brightly on the output screen (e.g., near the center or near the floor of the screen display area (130)) to be relatively lower than a weight (e.g., a brightness change coefficient, an optical deviation correction coefficient, a projection surface deviation correction coefficient) to be applied to input pixels to be projected on a display area that may be displayed darkly on the output screen (e.g., near the edge or near the valley of the screen display area (130)).

[0063] In addition to brightness deviation correction, the processor (410) can perform distortion correction to compensate for image distortion. For example, if the projection surface (110) is flat, such as a screen, the processor (410) can perform flat distortion correction, such as keystone correction, on the input image to generate an output image. For example, if the projection surface (110) is multi-faceted, the processor (410) can perform multi-faceted distortion correction on the input image to generate an output image. For example, if the projection surface (110) is non-flat, such as a curved surface, the processor (410) can perform non-flat distortion correction on the input image to generate an output image. That is, the processor (410) can perform automatic correction on the input image so that the image displayed on the screen display area (130) can appear as a flat image without distortion by taking into account the curvature characteristics of the projection surface (110).

[0064] According to an example, the processor (410) uses a grayscale conversion model (e.g., the first grayscale conversion model (620) of FIG. 6a or the second grayscale conversion model (650) of FIG. 6b) to convert an input pixel (b) in or b in' ) corresponding to the brightness change coefficient (δ) curve )(Example: The first brightness change coefficient (δ) of Fig. 6a curve1 )(630) or the second brightness change coefficient (δ) of Fig. 6bcurve2 )(660)) can be determined. The processor (410) determines the input pixel (b) by the brightness change coefficient (630, 660). in or b in' ) for the output pixel (b) with brightness deviation corrected out2 or b out2' ) can be generated. The gradation conversion model (620 or 650) can be set or preset at the production process stage of the image projection device (100). One or more gradation conversion models (620, 650) can be set in the image projection device (100). For example, a gradation conversion model reflecting the characteristics of the projection lens can be set or preset, or registered in the image projection device (100). For example, gradation conversion models (620, 650) to be selectively applied in response to a content type (e.g., movie mode, game mode, or sports mode) can be set or preset, or registered in the image projection device (100). For example, gradation conversion models (620, 650) to be selectively applied in response to an image output mode (e.g., natural image, soft image, bright image, dark image) can be set or preset, or registered in the image projection device (100). In this case, the processor (410) may adjust or change the characteristics of a preset (or set) grayscale conversion model (620, 650) in response to a user's request. For example, the processor (410) may select an appropriate grayscale conversion model (620, 650) in response to the user's content type or video output mode settings and apply the selected grayscale conversion model to correct for brightness deviation.

[0065] According to one example, the processor (410) calculates an optical deviation correction factor (G) that reflects the vignetting characteristics of the lens included in the image projector (440). Φ) can be obtained. The processor (410) may obtain, for example, an optical deviation correction coefficient (G) corresponding to each of a plurality of partial display areas (e.g., the first to second partial display areas (731, 733, 735, 737) of FIG. 7A) from a look-up table preset based on vignetting characteristics. Φ ) can be obtained. A plurality of partial display areas (731, 733, 735, 737) can be defined by dividing the projection area (120) based on the change in the vignetting characteristic. For example, the plurality of partial display areas can be areas that can be divided based on the boundary where brightness deviation occurs due to the vignetting characteristic of the lens in the projection area (120) (see FIG. 7a). For example, an optical deviation correction coefficient (G Φ ) may have a relatively smaller value in the first partial display area (e.g., the first partial display area (731) of FIG. 7A) at a short distance from the center point of the projection area (120) compared to the second partial display area (e.g., the second partial display area (733) of FIG. 7A) at a long distance.

[0066] According to one example, the processor (410) calculates an optical deviation correction coefficient (G) for a specific pixel based on an angle (Φ1 or Φ2) between a projection beam (e.g., a first projection beam (751) of FIG. 7B) that will display a substantially central pixel (e.g., a central pixel (741) of FIG. 7B) of the projection area (120) among the display pixels of the projection area (120) and a projection beam (e.g., a second projection beam (753) or a third projection beam (755) of FIG. 7B) that will display a specific pixel (e.g., a first target pixel (743) or a second target pixel (745) of FIG. 7B). Φ ) can be obtained (see FIG. 7b). The processor (410) may, for example, obtain an optical deviation correction coefficient (G) for all pixels in the projection area (120). Φ ) can be obtained. The processor (410) may, for example, obtain an optical deviation correction coefficient (G) for some pixels of the projection area (120). Φ) can be obtained. For example, the optical deviation correction coefficient (G Φ ) may be a predetermined number of pixels distributed in each partial display area where brightness deviation occurs due to vignetting characteristics. For example, the processor (410) may set a relatively small optical deviation correction coefficient (G) for a target pixel (743) located at a short distance from the center pixel (741) of the projection area (120) and having a small angle (Φ2) compared to a target pixel (745) located at a long distance and having a large angle (Φ1). Φ ) can be set.

[0067] According to an example, the processor (410) can obtain a projection surface deviation correction coefficient (G) that reflects the characteristics of the projection surface. The processor (410) may, for example, obtain a first projection surface deviation correction coefficient (G) by considering the arrival distance (d) for each sensing measurement point. d ) can be obtained (see FIG. 8b). The processor (410) can determine the reaching distance (d) for each sensing measurement point based on the sensing data obtained by the distance sensor (420) (e.g., the time difference between the time when the projection beam is transmitted and the time when the corresponding projection beam is fed back). For example, the processor (410) can determine an optical deviation correction coefficient (G) that is relatively smaller for a pixel with a short reaching distance (d) than for a pixel with a long reaching distance (d). Φ ) can be set. The processor (410) can, for example, set the incident angle (θ) for each projection beam at the sensing measurement points. i ) considering the second projection surface deviation correction factor (G) θ ) can be obtained (see Fig. 8c). The processor (410) can obtain the incident angle (θ) for each projection beam. i ) can be determined based on the sensing data (e.g., spatial orthogonal coordinate system of sensing measurement points) acquired by the distance sensor (420). For example, the incident angle (θ i) can be defined as the angle between the beam vector and the normal vector corresponding to the sensing measurement point. For example, the processor (410) may be configured to determine the incident angle (θ i ) for this small pixel, the angle of incidence (θ) i ) has a relatively small optical deviation correction factor (G) compared to large pixels. Φ ) can be set.

[0068] According to one example, the processor (410) may include a data collection module (411), a correction intensity calculation module (413), and / or a pixel brightness adjustment module (415).

[0069] The data collection module (411) can collect data necessary to perform brightness correction and / or brightness deviation correction. In one example, the data collection module (411) can obtain a grayscale transformation model to be applied to improve the contrast ratio by taking into account the characteristics of the projection lens. In one example, the data collection module (411) can project a single grayscale image onto a projection surface (1100) and collect data on optical deviation from an image captured by a camera of the output screen displayed on the projection surface (110) by the camera. The data collection module (411) can generate at least one LUT using the collected data on optical deviation. The LUT can include an optical deviation correction coefficient (G) for correcting optical deviation due to the characteristics of the projection lens for each pixel of the output image. Φ ) can be defined. According to an example, the data acquisition module (411) can generate at least one LUT for correcting brightness deviation due to vignetting characteristics of a lens (e.g., see FIG. 7a or FIG. 7b). For example, the LUT may include an optical deviation correction coefficient (G) for correcting optical deviation due to vignetting characteristics for each pixel of the output image. Φ) can be defined. For example, the LUT may define an optical deviation correction coefficient (G) for correcting the optical deviation for each partial display area (e.g., the first to second partial display areas (731, 733, 735, 737) of FIG. 7a) in which optical deviation due to vignetting characteristics occurs in the output image. Φ ) can be defined. According to an example, the data collection module (411) can collect data on the characteristics of the projection surface. The data collection module (411) can collect position data corresponding to a plurality of sensing measurement points acquired by the distance sensor (420) as data on the characteristics of the projection surface. The position data may include, for example, a spatial orthogonal coordinate system (e.g., coordinate value (P(x,y,z))) corresponding to the position of each of the sensing measurement points in the coordinate space. The position data may include, for example, a reach distance (d) for each sensing measurement point. The data collection module (411) considers the reach distance (d) for each sensing measurement point and calculates a first projection surface deviation correction coefficient (G d ) can be determined. According to an example, the data acquisition module (411) determines the incident angle (θ) corresponding to the projection beam at the sensing measurement point. i ) can be collected as data on the characteristics of the projection surface. For example, the angle of incidence (θ i ) can be defined as the angle between the beam vector and the normal vector at the sensing measurement point. The data acquisition module (411) can measure the incident angle (θ) for each projection beam. i ) considering the second projection surface deviation correction factor (G) θ ) can be determined.

[0070] The correction intensity calculation module (413) applies a weight (e.g., a brightness change coefficient (δ)) to each pixel of the input image (401) for brightness correction. curve ), optical deviation correction factor (G Φ), projection surface deviation correction coefficient (G)) can be determined. The correction strength calculation module (413) can determine, for example, a weight (e.g., a brightness change coefficient (δ)) to be applied to pixels to be projected in a display area that may be displayed dark on the output screen (e.g., near the edge or near the valley of the screen display area (130)). curve ), optical deviation correction factor (G Φ ), projection surface deviation correction coefficient (G)) can be determined to be relatively higher than the weights (e.g., brightness change coefficient, optical deviation correction coefficient, projection surface deviation correction coefficient) to be applied to input pixels to be projected on a display area that can be displayed brightly on the output screen (e.g., near the center or near the floor of the screen display area (130)). The correction strength calculation module (413) may determine, for example, the weights (e.g., brightness change coefficient, optical deviation correction coefficient, projection surface deviation correction coefficient) to be applied to pixels to be projected on a display area that can be displayed brightly on the output screen (e.g., near the center or near the floor of the screen display area (130)) to be compared with the weights (e.g., brightness change coefficient (δ) to be applied to pixels to be projected on a display area that can be displayed darkly on the output screen (e.g., near the edge or near the valley of the screen display area (130)). curve ), optical deviation correction factor (G Φ ), the projection surface deviation correction coefficient (G) can be determined relatively low.

[0071] According to an example, the correction intensity calculation module (413) uses a grayscale conversion model (e.g., the first grayscale conversion model (620) of FIG. 6a or the second grayscale conversion model (650) of FIG. 6b) to apply a brightness change coefficient (δ) to each pixel when correcting brightness for an input image. curve ) can be determined. The grayscale model is, for example, the input pixel brightness (B in (i) Brightness change coefficient (δ) to correct brightness to improve contrast ratio curve )(Example: The first brightness change coefficient (δ) of Fig. 6a curve1)(630) or the second brightness change coefficient (δ) of Fig. 6b curve2 )(660)) can be defined (e.g., see FIG. 6a or FIG. 6b). The grayscale conversion model can be modeled to improve the contrast ratio, for example, by considering the characteristics of the projection lens. The grayscale conversion model can be modeled to be selectively applied, for example, in response to a content type (e.g., movie mode, game mode, or sports mode). The grayscale conversion model can be modeled to be selectively applied, for example, in response to an image output mode (e.g., natural image, soft image, bright image, dark image).

[0072] For example, the brightness change coefficient is the difference between the input pixel brightness and the desired output pixel brightness (B out (i) Difference (e.g. B) out (i) - B in(i)). For example, if the correction intensity calculation module (413) knows the input pixel brightness for a specific pixel, it can obtain the brightness change coefficient from the grayscale transformation model (620 or 650). For example, the correction intensity calculation module (413) can obtain the brightness change coefficient for all pixels of the input image (401) using the grayscale transformation model (620 or 650). For example, the correction intensity calculation module (413) can obtain the brightness change coefficient for some pixels of the input image (401) using the grayscale transformation model (620 or 650), and predict the brightness change for the remaining pixels based on the brightness change coefficient obtained for some pixels. The correction intensity calculation module (413) can determine the brightness change coefficient for each pixel so that brightness correction can be performed for each pixel of the input image (401). In this case, the output pixel brightness can be differentially adjusted for each pixel. For example, the correction intensity calculation module (413) can determine a brightness change coefficient to be applied to the brightness of an input pixel of a specific grayscale to be relatively higher or lower than a brightness change coefficient to be applied to the brightness of an input pixel of a different grayscale.

[0073] For example, the correction intensity calculation module (413) may determine an optical deviation correction coefficient to correct brightness deviation for the input image (401) by considering the characteristics (e.g., vignetting characteristics) of the lens that projects the optical signal (405) converted from the output image (403). If the correction intensity calculation module (413) knows the optical deviation of the corresponding lens due to a cause such as the vignetting characteristic, the correction intensity calculation module (413) may determine an optical deviation correction coefficient to differentially correct brightness deviation for each pixel and / or each sub-region by considering the optical deviation. For example, the correction intensity calculation module (413) may determine an optical deviation correction coefficient to be applied to pixels to be projected near the center of the screen to be relatively higher or lower than an optical deviation correction coefficient to be applied to pixels to be projected near the edge.

[0074] In one example, the correction intensity calculation module (413) can determine a projection surface deviation correction coefficient to correct brightness deviation for the input image (401) by considering characteristics of the projection surface (110) (e.g., curvature characteristics due to non-planarity). For example, the characteristics of the projection surface (110) may be a distance at which a projection beam emitted by the image projection device (100) reaches the projection surface (110) and / or an incident angle at the projection surface (110) due to a direction of the beam emitted by the image projection device (100). For example, the distance or incident angle per pixel may differ depending on the type of the projection surface (110) (e.g., planar, multi-faceted, or non-planar). If the distance or incident angle per pixel is known, the correction intensity calculation module (413) can determine a deviation correction coefficient to differentially correct brightness deviation for each pixel and / or each sub-region by considering the distance or incident angle per pixel. For example, the correction strength calculation module (413) may determine a deviation correction coefficient to be applied to pixels having a close arrival distance to be relatively higher than a deviation correction coefficient to be applied to pixels having a far arrival distance. For example, the correction strength calculation module (413) may determine a deviation correction coefficient to be applied to pixels having a far arrival distance to be relatively lower than a deviation correction coefficient to be applied to pixels having a close arrival distance. For example, the correction strength calculation module (413) may determine a deviation correction coefficient to be applied to pixels having a large incidence angle to be relatively higher than a deviation correction coefficient to be applied to pixels having a small incidence angle. For example, the correction strength calculation module (413) may determine a deviation correction coefficient to be applied to pixels having a small incidence angle to be relatively lower than a deviation correction coefficient to be applied to pixels having a large incidence angle.

[0075] The pixel brightness adjustment module (415) can perform brightness correction and / or brightness deviation correction on the input image by applying at least one of the brightness change coefficient, the optical deviation correction coefficient, or the projection surface deviation correction coefficient obtained by the correction intensity calculation module (413). The pixel brightness adjustment module (415) transmits the output image (403) generated through the brightness correction and / or brightness deviation correction to the image projector (440) so that the optical signal (405) is transmitted. The output image (403) generated through the brightness correction and / or brightness deviation correction allows an output image with consistent brightness overall to be displayed in the projection area (120).

[0076] In one example, the image projection device (100) may include additional components such as a user interface (I / F). For example, the user I / F may be configured to receive information from a user. The user I / F may receive commands or data to be used in a component of the image projection device (100) (e.g., the processor (410)) from an external source of the image projection device (100) (e.g., a user). The user I / F may include, for example, a microphone, a mouse, a keyboard, keys (e.g., buttons), a remote control, or a digital pen (e.g., a stylus pen). In one example, the user I / F may be configured to transmit information to the user. The user I / F may output an audio signal to the external source of the image projection device (100) through a component such as a speaker. For example, the speaker may be used for general purposes such as multimedia playback or recording playback.

[0077] In one example, the image projection device (100) may include additional components such as a transceiver. The transceiver may be configured to exchange information with at least one electronic device. The transceiver may transmit and receive data or signals with a remote control device or external sensors under the control of the processor (410).

[0078] According to an example, the transmitter and receiver may include, but are not limited to, a Bluetooth communication unit, a BLE (Bluetooth low energy) communication unit, a near field communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an infrared (IrDA, infrared data association) communication unit, a WFD (Wi-Fi Direct) communication unit, a UWB (ultra-wideband) communication unit, an Ant+ communication unit, or a microwave (uWave) communication unit, depending on the performance and structure of the image projection device (100).

[0079] According to one example, the transceiver may support establishing a direct (e.g., wired) communication channel or a wireless communication channel with a remote control device and performing communication through the established communication channel. The transceiver may include one or more CPs that support direct (e.g., wired) communication or wireless communication. One or more CPs may operate independently with respect to the processor (410). The transceiver may include, for example, a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules may communicate with at least one remote control device, which is an external electronic device, via a network (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or IrDA, or a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a WAN)). These different types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips).

[0080] According to one example, the image projection device (100) may include an external sensor as an additional component. The sensing data acquired through the external sensor may include information to be used to obtain the position of the image projection device (100). The sensing data acquired through the external sensor may include information to be used to obtain the position of a viewer (e.g., the viewer (125) of FIG. 1). The image projection device (100) may predict the point in time at which the viewer (125) looks at the projection surface (110) by considering the position of the viewer (125) acquired based on the sensing data. The processor (410) may identify the position of the image projection device (100) using the sensing data.

[0081] FIG. 5 is a control flowchart (500) for obtaining position data of an area (e.g., projection area (120) of FIG. 1 or FIG. 2) on which image data is to be projected in an image projection device (e.g., image projection device (100) of FIG. 1) according to one embodiment.

[0082] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0083] Referring to FIG. 5, the image projection device (100) can receive image data in operation 510. The image data can be input according to a content service such as a movie or a game, for example.

[0084] The image projection device (100) may, in operation 520, collect data necessary to perform brightness correction and / or brightness deviation correction. According to an example, the image projection device (100) may obtain a grayscale transformation model to be applied to improve the contrast ratio by taking into account the characteristics of the projection lens. According to an example, the image projection device (100) may project a single grayscale image onto a projection surface (1100), and thereby collect data on optical deviation from an image captured by a camera of the output screen displayed on the projection surface (110). The image projection device (100) may generate at least one LUT using the collected data on optical deviation. The LUT may include an optical deviation correction coefficient (G) for correcting optical deviation due to the characteristics of the projection lens for each pixel of the output image. Φ ) can be defined. According to an example, the image projection device (100) can generate at least one LUT for correcting brightness deviation due to vignetting characteristics of a lens (e.g., see FIG. 7a or FIG. 7b). For example, the LUT may include an optical deviation correction coefficient (G) for correcting optical deviation due to vignetting characteristics for each pixel of the output image. Φ ) can be defined. For example, the LUT may define an optical deviation correction coefficient (G) for correcting the optical deviation for each partial display area (e.g., the first to second partial display areas (731, 733, 735, 737) of FIG. 7a) in which optical deviation due to vignetting characteristics occurs in the output image. Φ) can be defined. According to an example, the image projection device (100) can collect data on the characteristics of the projection surface. The image projection device (100) can collect position data corresponding to a plurality of sensing measurement points acquired by the distance sensor (420) as data on the characteristics of the projection surface. The position data may include, for example, a spatial orthogonal coordinate system (e.g., coordinate value (P(x,y,z))) corresponding to the position of each of the sensing measurement points in the coordinate space. The position data may include, for example, a reaching distance (d) for each sensing measurement point. The image projection device (100) considers the reaching distance (d) for each sensing measurement point and calculates a first projection surface deviation correction coefficient (G d ) can be determined. According to an example, the image projection device (100) can determine the incident angle (θ) corresponding to the projection beam at the sensing measurement point. i ) can be collected as data on the characteristics of the projection surface. For example, the angle of incidence (θ i ) can be defined as the angle between the beam vector and the normal vector at the sensing measurement point. The data acquisition module (411) can measure the incident angle (θ) for each projection beam. i ) considering the second projection surface deviation correction factor (G) θ ) can be determined.

[0085] The image projection device (100) applies a weight (e.g., brightness change coefficient (δ)) to each pixel of the input image (401) for brightness correction in operation 530. curve ), optical deviation correction factor (G Φ ), projection surface deviation correction coefficient (G)) can be determined. The image projection device (100) can determine, for example, a weight (e.g., a brightness change coefficient (δ)) to be applied to pixels to be projected in a display area that may be displayed darkly on the output screen (e.g., near the edge or valley of the screen display area (130) curve ), optical deviation correction factor (G Φ), projection surface deviation correction coefficient (G)) can be determined to be relatively high compared to the weights (e.g., brightness change coefficient, optical deviation correction coefficient, projection surface deviation correction coefficient) to be applied to input pixels to be projected on a display area (e.g., near the center or near the floor of the screen display area (130)) that can be displayed brightly on the output screen. The image projection device (100) may, for example, determine the weights (e.g., brightness change coefficient, optical deviation correction coefficient, projection surface deviation correction coefficient) to be applied to pixels to be projected on a display area (e.g., near the center or near the floor of the screen display area (130)) that can be displayed brightly on the output screen, and the weights (e.g., brightness change coefficient (δ) to be applied to pixels to be projected on a display area (e.g., near the edge or near the valley of the screen display area (130)) that can be displayed darkly on the output screen. curve ), optical deviation correction factor (G Φ ), the projection surface deviation correction coefficient (G) can be determined relatively low.

[0086] According to an example, the image projection device (100) uses a grayscale conversion model (e.g., the first grayscale conversion model (620) of FIG. 6a or the second grayscale conversion model (650) of FIG. 6b) to apply a brightness change coefficient (δ) to each pixel when correcting brightness for an input image. curve ) can be determined. The grayscale model is, for example, the input pixel brightness (B in (i) Brightness change coefficient (δ) to correct brightness to improve contrast ratio curve )(Example: The first brightness change coefficient (δ) of Fig. 6a curve1 )(630) or the second brightness change coefficient (δ) of Fig. 6b curve2)(660)) can be defined (e.g., see FIG. 6a or FIG. 6b). The grayscale conversion model can be modeled to improve the contrast ratio, for example, by considering the characteristics of the projection lens. The grayscale conversion model can be modeled to be selectively applied, for example, in response to a content type (e.g., movie mode, game mode, or sports mode). The grayscale conversion model can be modeled to be selectively applied, for example, in response to an image output mode (e.g., natural image, soft image, bright image, dark image).

[0087] For example, the brightness change coefficient (δ curve ) is the input pixel brightness B in (i) and the output pixel brightness (B) to be obtained out (i) Difference (e.g. B) out (i) - B in(i)). For example, if the image projection device (100) knows the input pixel brightness for a specific pixel, it can obtain the brightness change coefficient from the grayscale conversion model (620 or 650). For example, the image projection device (100) can obtain the brightness change coefficient for all pixels of the input image (401) using the grayscale conversion model (620 or 650). For example, the image projection device (100) can obtain the brightness change coefficient for some pixels of the input image (401) using the grayscale conversion model (620 or 650), and predict the brightness change for the remaining pixels based on the brightness change coefficient obtained for some pixels. The image projection device (100) can determine the brightness change coefficient for each pixel so that brightness correction can be performed for each pixel of the input image (401). In this case, the output pixel brightness can be differentially adjusted for each pixel. For example, the image projection device (100) can determine a brightness change coefficient to be applied to the brightness of an input pixel of a specific grayscale to be relatively higher or lower than a brightness change coefficient to be applied to the brightness of an input pixel of a different grayscale.

[0088] According to an example, the image projection device (100) calculates an optical deviation correction coefficient (G) to correct brightness deviation for the input image (401) by considering the characteristics (e.g., vignetting characteristics) of the lens that will project the optical signal (405) converted into the output image (403). Φ) can be determined. If the image projection device (100) knows the optical deviation of the corresponding lens due to a cause such as a vignetting characteristic, the image projection device (100) can determine an optical deviation correction coefficient for differentially correcting the brightness deviation for each pixel and / or each partial area by considering the optical deviation. For example, the image projection device (100) can determine an optical deviation correction coefficient to be applied to pixels to be projected near the center of the screen to be relatively higher or lower than an optical deviation correction coefficient to be applied to pixels to be projected near the edge.

[0089] In one example, the image projection device (100) can determine a brightness correction factor (G) to correct brightness deviation for the input image (401) by considering characteristics of the projection surface (110) (e.g., curvature characteristics due to non-planarity). For example, the characteristics of the projection surface (110) may be a distance at which a projection beam emitted by the image projection device (100) reaches the projection surface (110) and / or an incident angle at the projection surface (110) due to a direction of the beam emitted by the image projection device (100). For example, the reaching distance or incident angle per pixel may differ depending on the type of the projection surface (110) (e.g., planar, multi-faceted, or non-planar). If the reaching distance or the incident angle per pixel is known, the image projection device (100) can determine a brightness correction coefficient (G) to differentially correct the brightness deviation for each pixel and / or each subarea by considering the reaching distance or the incident angle per pixel.

[0090] According to an example, the image projection device (100) applies a second projection surface deviation correction coefficient (G) to pixels having a large incident angle (θ). θ ) is applied to pixels with small incident angle (θ) and a second projection plane deviation correction coefficient (G) is applied. θ ) can be determined relatively high compared to the incident angle (θ). For example, the image projection device (100) may have a second projection surface deviation correction coefficient (G) to be applied to pixels having a small incident angle (θ). θ) is applied to pixels with a large incident angle (θ) and a second projection plane deviation correction coefficient (G) is applied. θ ) can be determined relatively low compared to the image projection device (100). For example, the image projection device (100) is '(1 - cos(θ)) α 'The second projection surface deviation correction factor (G) θ ) can be determined.

[0091] For example, the image projection device (100) may determine a deviation correction coefficient to be applied to pixels having a short reach distance to be relatively higher than a deviation correction coefficient to be applied to pixels having a long reach distance. For example, the image projection device (100) may determine a deviation correction coefficient to be applied to pixels having a long reach distance to be relatively lower than a deviation correction coefficient to be applied to pixels having a short ... 2 ) as the first projection surface deviation correction factor (G) d ) can be determined.

[0092] According to an example, the image projection device (100) is a random distribution (GΦ = N(μ,σ) centered on a reference point (e.g., the center point of the brightness distribution of the output screen or the center point of the output screen) 2 )) can also be used to determine a deviation correction coefficient for differentially correcting brightness deviation for each pixel and / or each sub-area.

[0093] According to an example, the image projection device (100) determines the brightness change coefficient (δ) previously determined. curve ), optical deviation correction factor (G Φ ), first projection surface deviation correction factor (G d ) and the second projection surface deviation correction coefficient (G θ ) can be used to determine the weight to be applied to each pixel of the input image (401) for brightness correction. For example, the image projection device (100) may be configured to use 'δ curve × G Φ × G θ × G d' can determine the weight.

[0094] The image projection device (100) applies a weight ('δ) to each pixel of the input image (401) for brightness correction in operation 540. curve × G Φ × G θ × G d ') can be applied to perform brightness correction and / or brightness deviation correction on pixels of the input image (401).

[0095] The image projection device (100) can, in operation 550, convert an output image (403) generated through brightness correction and / or brightness deviation correction into an optical signal (405) and transmit it through the image projector (440). The output image (403) generated through brightness correction and / or brightness deviation correction allows an output image with consistent brightness to be displayed in the projection area (120).

[0096] FIG. 6a is a drawing for explaining brightness deviation compensation that increases brightness for input pixels using a grayscale conversion model, FIG. 6b is a drawing for explaining brightness deviation compensation that decreases brightness for input pixels using a grayscale conversion model, and FIG. 6c is a drawing for explaining brightness compensation that uses a grayscale conversion model.

[0097] The first tone conversion model (620) in FIG. 6a is modeled to increase the brightness of the input pixel, and the second tone conversion model (650) in FIG. 6b is modeled to decrease the brightness of the input pixel.

[0098] Referring to Fig. 6a, the first grayscale conversion model (620) is designed to perform differential brightness deviation correction to increase brightness for each grayscale of the input pixel brightness. For example, the first grayscale conversion model (620) has a relatively high brightness change coefficient (δ) for grayscales that are relatively high compared to intermediate grayscales. curve(630)) is applied, and a relatively low brightness change coefficient (δ) is applied to relatively low grayscales compared to intermediate grayscales. curve (630)) is applied. Here, the brightness change coefficient (δ) curve (630)) is the brightness of a specific input pixel (b in ) when applying the reference graph (610) to the first output pixel brightness (b) out1 ) and the second output pixel brightness (b) that can be obtained when applying the graph (620) corresponding to the first tone conversion model. out2 ) can be defined as the difference value. In this case, the first output pixel brightness (b out' ) is the second output pixel brightness (b out2 ) than the brightness change coefficient (δ) curve (630)) can be as high as that.

[0099] Referring to Fig. 6b, the second tone conversion model (650) is designed to perform differential brightness deviation compensation to reduce brightness for each tone of the input pixel brightness. For example, the second tone conversion model (650) has a relatively high brightness change coefficient (δ) for relatively high tone levels compared to intermediate tone levels. curve' (660)) is applied, and a relatively low brightness change coefficient (δ) is applied to relatively low grayscales compared to intermediate grayscales. curve' (660)) is applied. Here, the brightness change coefficient (δ) curve' (660)) is the brightness of a specific input pixel (b in ) when applying the reference graph (640) to the first output pixel brightness (b) out1' ) and the second output pixel brightness (b) that can be obtained when applying the graph (650) corresponding to the second tone conversion model. out2' ) can be defined as the difference value. In this case, the first output pixel brightness (b out1' ) is the second output pixel brightness (b out2' ) than the brightness change coefficient (δ) curve'(660)) can be as low as that.

[0100] Referring to FIG. 6C, an image projection device (e.g., the image projection device (100) of FIG. 1) can generate an output image by correcting the brightness of input pixels of an input image (670) based on a grayscale conversion model (e.g., the first grayscale conversion model (620) of FIG. 6A). Objects (e.g., the sun (671), the roof (673), and the wall (675)) included in the input image (670) illustrated above may have different brightnesses. The image projection device (100) can obtain a correction intensity to be applied to each object having different brightnesses in the input image (670) based on the grayscale conversion model (620). A correction intensity image (680) illustrated below shows a correction intensity corresponding to objects (e.g., the sun (671), the roof (673), and the wall (675)) included in the input image (670). For example, in the input image (670), the sun (671) may have the highest brightness value, the wall (675) may have an intermediate brightness value, and the roof (673) may have the darkest brightness value. The image projection device (100) may assign a low correction intensity (e.g., the lowest correction intensity in the gradation bar) (681) to the brightest sun (671) based on the gradation conversion model (620) so that the amount of brightness change after correction may be the smallest. The image projection device (100) may assign a medium correction intensity (e.g., the intermediate correction intensity in the gradation bar) (685) to the wall (675) having an intermediate brightness based on the gradation conversion model (620) so that the amount of brightness change after correction may be intermediate. The image projection device (100) can assign a low correction intensity (e.g., a relatively low correction intensity in the gradient bar) (683) to the darkest roof (673) based on the gradation conversion model (620) so that the amount of brightness change after correction is not large.For example, it can be seen that the correction strength for brightness deviation can be assigned low so that the brightness change can be relatively small for specific objects that are very bright or very dark in the input image (670), such as the sun (671) or the roof (673).

[0101] FIG. 7a and FIG. 7b are drawings for explaining obtaining an optical deviation correction coefficient by taking into account the vignetting characteristics of the lens.

[0102] In Fig. 7a, brightness deviation may occur in four partial display areas due to the vignetting phenomenon, but this is only one example, and the same or similar optical deviation correction coefficient may be determined for more or less partial display areas.

[0103] Referring to FIG. 7a, the image projection device (100) can obtain an optical deviation correction coefficient that reflects the vignetting characteristics of the lens included in the image projector (e.g., the image projector (440) of FIG. 4). The vignetting phenomenon is a phenomenon in which light is not fully transmitted to the edges (720a, 720b, 720c, 720d) of the projection area (120) due to refraction that occurs when the optical signal (405) passes through the lens. As a result, on the output screen projected on the projection surface (e.g., the projection surface (110) of FIG. 1) by the optical signal (e.g., the optical signal (405) of FIG. 4) transmitted by the image projection device (100), the edges (720a, 720b, 720c, 720d) may be displayed with a darker brightness than the center (710).

[0104] The image projection device (100) can perform brightness deviation correction for input pixels to correct brightness deviation that occurs on the output screen due to the vignetting phenomenon. According to an example, the image projection device (100) can preset an LUT based on vignetting characteristics. For example, in the LUT, an optical deviation correction coefficient (G) for correcting optical deviation due to the characteristics of the projection lens for each pixel of the input image Φ ) can be defined. For example, the LUT may define an optical deviation correction factor (G) to correct the optical deviation for each partial display area (731, 733, 735, 737) where optical deviation due to vignetting characteristics occurs in the output image. Φ ) can be defined. For example, for the edges (720a, 720b, 720c, 720d) where brightness may be relatively dark due to vignetting, the optical deviation correction factor (G) can be set so that brightness deviation correction can be performed by a relatively high correction strength. Φ )(737) can be determined relatively high. For example, for the area near the center (710) where the brightness is relatively bright due to the vignetting phenomenon, the optical deviation correction coefficient (G) can be set so that brightness deviation correction can be performed by a relatively low correction intensity. Φ )(731) can be determined relatively low. For example, because the brightness becomes relatively darker as it gets further away from the center (710) due to the vignetting phenomenon, the optical deviation correction coefficient (G) can be set so that brightness deviation correction can be performed by a relatively higher correction strength. Φ )(733, 735) can be determined.

[0105] Referring to FIG. 7b, the image projection device (100) can obtain an angle (Φ1 or Φ2) between a projection beam (e.g., a first projection beam (751) of FIG. 7b) that will display a substantially central pixel (e.g., a central pixel (741) of FIG. 7b) of the projection area (120) among the display pixels of the projection area (120) and a projection beam (e.g., a second projection beam (753) or a third projection beam (755) of FIG. 7b) that will display a specific pixel (e.g., a first target pixel (743) or a second target pixel (745) of FIG. 7b). The image projection device (100) can determine an optical deviation correction coefficient for the corresponding pixel based on the angle (Φ1 or Φ2) obtained for each specific pixel (743, 745). The image projection device (100) can obtain, for example, an optical deviation correction coefficient for all pixels in the projection area (120). The image projection device (100) can obtain, for example, an optical deviation correction coefficient for some pixels in the projection area (120). For example, some pixels for which the optical deviation correction coefficient is to be obtained may be a predetermined number of pixels distributed in each partial display area where a brightness deviation occurs due to a vignetting phenomenon. For example, a target pixel (743) located at a short distance from the center pixel (741) of the projection area (120) and having a small angle (Φ2) has a relatively small optical deviation correction coefficient (G) compared to a target pixel (745) located at a long distance and having a large angle (Φ1). Φ ) can be determined.

[0106] For example, a target pixel (745) located far from the center pixel (741) of the projection area (120) and having a large angle (Φ1) has a relatively large optical deviation correction factor (G) compared to a target pixel (743) located far from the center pixel (741) and having a small angle (Φ2). Φ ) can be determined. For example, the optical deviation correction factor (G) corresponding to a relatively high correction intensity the farther away from the center pixel (741) Φ )(751, 753, 755) can be assigned.

[0107] FIG. 8a is a drawing for explaining that brightness deviation occurs on a non-planar projection surface (e.g., projection surface (110) of FIG. 1) depending on the incident angle of the projection beam, and FIG. 8b is a second projection surface deviation correction coefficient (G) obtained by considering the incident angle (θ) for each sensing measurement point. θ ) is a drawing for explaining compensation of brightness deviation, and Fig. 8c shows the first projection surface deviation correction coefficient (G) obtained by considering the reaching distance (d) for each sensing measurement point. d ) is a drawing to explain how to compensate for brightness deviation.

[0108] Referring to FIG. 8A, the image projection device (100) can display an output screen (820) in the screen display area (130) by projecting an output image (e.g., an output image (403) in FIG. 4) generated without brightness correction (e.g., a brightness deviation correction (330) in FIG. 3) for an input image (810) (e.g., an input image (401) in FIG. 4) onto the projection area (120). In this case, a deviation may occur in the brightness of a corresponding pixel in the output screen (820) depending on the incident angle (θ) at which the projection beam is projected onto the projection area (120). For example, the intensity of the reflected light may decrease as the projection beam is obliquely incident onto the projection area (120). That is, the intensity of the light reflected from the projection surface (110) may have a different attenuation amount depending on the incident angle of the projection beam. Due to this, the resulting brightness deviation causes an output screen (820) to be displayed with uneven brightness.

[0109] Referring to FIG. 8b, if the image projection device (100) can obtain the incident angle (θ1 (851), θ2 (852), θ3 (853), θ4 (854), θ5 (855), θ6 (856)) for each projection beam (801) or pixel projection point (821, 823, 825, 827, 829), the second projection surface deviation correction coefficient (G) for the corresponding pixel θ ) can be determined. For example, a specific angle of incidence (θ i(850)) is the beam vector (r) of the corresponding projection beam (801). i )(840) and the normal vector (n) at the corresponding pixel projection point (821) i )(830) can be determined by the angle between the beam vectors (841, 842, 843, 844, 845, 846) and the normal vectors (831, 832, 833, 834, 835, 836) at the corresponding pixel projection points (821, 823, 825, 827, 829). The image projection device (10 ... i ) for this small pixel, the angle of incidence (θ) i ) is a relatively small second optical deviation correction factor (G) compared to large pixels. Φ ) can be assigned. For example, the image projection device (100) can be assigned '(1 - cos(θ)) α 'The second projection surface deviation correction factor (G) θ ) can be determined.

[0110] Referring to FIG. 8c, if the image projection device (100) can obtain the arrival distance (d1 (871), d2 (872), d3 (873), d4 (874)) for each projection beam (861, 862, 863, 864) projected on the pixel projection point, the first projection surface deviation correction coefficient (G) for the corresponding pixel d ) can be determined. The image projection device (100) provides a small first optical deviation correction coefficient (G) corresponding to a relatively lower correction intensity for pixels with a short reach distance compared to pixels with a long reach distance. d ) can be assigned. The image projection device (100) has a large first optical deviation correction coefficient (G) corresponding to a relatively high correction intensity for pixels with long reach distances compared to pixels with short reach distances. d ) can be assigned. For example, the image projection device (100) may be assigned 'd 2'The first projection surface deviation correction factor (G) d ) can be determined.

[0111] As described above, the first projection surface deviation correction coefficient (G d ) and the second projection surface deviation correction coefficient (G θ ) is determined, the image projection device (100) determines the first projection surface deviation correction coefficient (G d ) and the second projection surface deviation correction coefficient (G θ ) can be used to determine the projection surface deviation correction strength, i.e., the projection surface deviation correction coefficient. For example, the first projection surface deviation correction coefficient (G d ) and the second projection surface deviation correction coefficient (G θ ) product (G) d × G θ ) can determine the projection surface deviation correction strength corresponding to the corresponding pixel, i.e., the projection surface deviation correction coefficient. The image projection device (100) can adjust the output pixel brightness by performing brightness correction on the input pixel brightness by the projection surface deviation correction strength determined for each pixel. The output image by the output pixels whose brightness has been adjusted in this way can be displayed as an output screen (880) in the projection area (120). The lower drawing in FIG. 8c visually expresses the deviation correction strength (890) corresponding to the output screen displayed in the projection area (120).

[0112] Figure 9 shows the weights (e.g., brightness change coefficient (δ)) applied to the input image (940). curve )(910), optical deviation correction factor (G Φ )(920), brightness correction coefficient (G)(930)) to perform brightness compensation and / or brightness deviation compensation to generate an output image (950).

[0113] Referring to FIG. 9, an image projection device (e.g., the image projection device (100) of FIG. 1) uses a specific grayscale conversion model to calculate a brightness change coefficient (δ) per pixel of an input image (940). curve)(910) can be obtained. The image projection device (100) can obtain an optical deviation correction coefficient (G) for each pixel of the input image (940) by considering the optical deviation, which is a characteristic of the projection lens. Φ )(920) can be obtained.

[0114] The image projection device (100) considers the incident angle (θ), which is a characteristic of the projection surface, and calculates a second projection surface deviation correction coefficient (G) for each pixel of the input image (940). θ ) can be obtained. The image projection device (100) considers the reaching distance (d), which is a characteristic of the projection surface, and calculates the first projection surface deviation correction coefficient (G) for each pixel of the input image (940). d ) can be obtained.

[0115] The image projection device (100) applies a weight (δ) to each pixel of the input image (401) for brightness correction. curve × G Φ × G θ × G d ') can be determined. The image projection device (100) determines the weight (δ) for each input pixel. curve × G Φ × G θ × G d ') can be used to perform brightness correction and / or brightness deviation correction for the corresponding input pixel. The image projection device (100) can perform brightness correction and / or brightness deviation correction for each input pixel to generate an output image (950).

[0116] FIG. 10 is a block diagram of an electronic device (1001) (e.g., an image projection device (100) of FIG. 2) within a network environment (1000) according to one or more embodiments.

[0117] Referring to FIG. 10, in a network environment (1000), an electronic device (1001) may communicate with an electronic device (1003) via a first network (1098) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (1005) or a server (1007) via a second network (1096) (e.g., a long-range wireless communication network). In one example, the electronic device (1001) may communicate with the electronic device (1005) via the server (1007). In one example, the electronic device (1001) may include a processor (1010), a memory (1020), an audio module (1040), an image module (1050), a sensor module (1060), a power management module (1070), an input module (1082), an interface (1084), a connection terminal (1086), or a communication module (1090). In some examples, the electronic device (1001) may omit at least one of these components (e.g., the input module (1082)), or may have one or more other components added. In some examples, some of these components may be integrated into a single component.

[0118] The processor (1010) may, for example, execute software (e.g., a program (1030)) to control at least one other component (e.g., a hardware or software component) of the electronic device (1001) connected to the processor (1010) and perform various data processing or operations. In one example, as at least a part of the data processing or operations, the processor (1010) may store commands or data received from other components (e.g., a sensor module (1060) or a communication module (1090)) in a volatile memory (1022), process the commands or data stored in the volatile memory (1022), and store result data in a non-volatile memory (1024). In one example, the processor (1010) may include a main processor (1012) (e.g., a central processing unit or an application processor) or an auxiliary processor (1014) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (1012). For example, when the electronic device (1001) includes the main processor (1012) and the auxiliary processor (1014), the auxiliary processor (1014) may be configured to use less power than the main processor (1012) or to be specialized for a given function. The auxiliary processor (1014) may be implemented separately from the main processor (1012) or as a part thereof.

[0119] The auxiliary processor (1014) may control at least a portion of functions or states associated with at least one component (e.g., a sensor module (1060) or a communication module (1090)) of the electronic device (1001), for example, on behalf of the main processor (1012) while the main processor (1012) is in an inactive (e.g., sleep) state, or together with the main processor (1012) while the main processor (1012) is in an active (e.g., application execution) state. In one example, the auxiliary processor (1014) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a communication module (1090)). In one example, the auxiliary processor (1014) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (1001) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (1007)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above models, but is not limited to the examples described above. In addition to, or as an alternative to, a hardware structure, the artificial intelligence model may include a software structure.

[0120] The memory (1020) can store various data used by at least one component (e.g., the processor (1010) or the sensor module (1060)) of the electronic device (1001). The data can include, for example, software (e.g., the program (1030)) and input data or output data for commands related to the software. The memory (1020) can include volatile memory (1022) or non-volatile memory (1024).

[0121] The program (1030) may be stored as software in memory (1020) and may include, for example, an operating system (1036), middleware (1034), or an application (1032).

[0122] The input module (1082) can receive commands or data to be used in a component of the electronic device (1001) (e.g., a processor (1010)) from an external source (e.g., a user) of the electronic device (1001). The input module (1082) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0123] The audio module (1040) may include an audio processing module (1042) or an audio output module (1044). The audio output module (1044) may output audio signals to the outside of the electronic device (1001). The audio output module (1044) may include, for example, a speaker. The speaker may be used for general purposes such as multimedia playback or recording playback. The audio processing module (1042) may convert sound into an electrical signal, or vice versa. In one example, the audio module (1040) may acquire sound through the input module (1082), or output sound through the audio output module (1044), or an external electronic device (e.g., electronic device (1003)) (e.g., speaker or headphone) that is directly (e.g., wired or wirelessly) or wirelessly connected to the electronic device (1001).

[0124] The image module (1050) may include an image processing module (1052) or an image output module (1054). The image processing module (1052) may output a video signal to the outside of the electronic device (1001). The image output module (1054) may include, for example, a display and / or an optical projector. The optical projector may convert an electrical video signal into an optical signal and output it. The image processing module (1052) may convert an image into an electrical signal or, conversely, convert an electrical signal into an image. According to an example, the image module (1050) may acquire an image through the input module (1082), or output an image through the aspect output module (1054), or an external electronic device (e.g., the electronic device (1003)) directly or wirelessly connected to the electronic device (1001). The image module (1050) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device.

[0125] The sensor module (1060) can detect the operating status (e.g., power or temperature) of the electronic device (1001) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to an example, the sensor module (1060) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0126] The interface (1084) may support one or more designated protocols that may be used to directly (e.g., wired or wirelessly) or wirelessly connect the electronic device (1001) to an external electronic device (e.g., electronic device (1003)). The interface (1084) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface (e.g., Bixby).

[0127] The connection terminal (1086) may include a connector through which the electronic device (1001) may be physically connected to an external electronic device (e.g., the electronic device (1003)). In one example, the connection terminal (1086) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0128] The power management module (1070) can manage power supplied to the electronic device (1001). According to one embodiment, the power management module (1070) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0129] The communication module (1090) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (1001) and an external electronic device (e.g., electronic device (1003), electronic device (1005), or server (1007)), and the performance of communication through the established communication channel. The communication module (1090) may operate independently from the processor (1010) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. In one example, the communication module (1090) may include a wireless communication module (1092) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (1094) (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules may communicate with an external electronic device (1005) via a first network (1098) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (1096) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1092) may use subscriber information (e.g., an international mobile subscriber identity (IMSI)) to verify or authenticate the electronic device (1001) within a communication network such as the first network (1098) or the second network (1096).

[0130] The wireless communication module (1092) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimizing terminal power and connecting multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (1092) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (1092) may support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (1092) may support various requirements specified in the electronic device (1001), an external electronic device (e.g., the electronic device (1005)), or a network system (e.g., the second network (1096)). According to one embodiment, the wireless communication module (1092) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0131] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0132] In one example, commands or data may be transmitted or received between the electronic device (1001) and an external electronic device (1005) via a server (1007) connected to a second network (1096). Each of the external electronic devices (1003 or 1005) may be the same or a different type of device as the electronic device (1001). In one example, all or part of the operations executed in the electronic device (1001) may be executed in one or more of the external electronic devices (1003, 1005, or 1007). For example, when the electronic device (1001) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (1001) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (1001). The electronic device (1001) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (1001) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one example, the external electronic device (1005) may include an Internet of Things (IoT) device. The server (1007) may be an intelligent server utilizing machine learning and / or a neural network.In one example, an external electronic device (1005) or server (1007) may be included within the second network (1096). The electronic device (1001) may be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology and IoT-related technology.

[0133] According to an example, an image projection apparatus (100) may include at least one distance sensor (ToF) (420). The image projection apparatus (100) may include an image projector (440). The image projection apparatus (100) may include at least one memory (430) including a non-volatile recording medium storing instructions. The image projection apparatus (100) may include at least one processor (410) operatively connected to the at least one distance sensor (420), the image projector (440), and the at least one memory, the processor including a processing circuit. When the instructions are individually or collectively executed by the at least one processor (410), the instructions may cause the image projection apparatus (100) to perform at least one operation. At least one of the above operations is performed on an input pixel (b) for a grayscale transformation model (521, 620 or 650). in or b in' ) and the above input pixel (b in or b in' ) corresponding to the output pixel (b) out2 or b out2') may include an operation (operation 530) of determining a brightness change coefficient (531, 630 or 660) regarding the projection surface (110). The at least one operation may include an operation (operation 530) of obtaining a projection surface deviation correction coefficient (535) in which a characteristic (525) of the projection surface (110) obtained from the at least one distance sensor (420) is reflected. The at least one operation may include an operation (operation 540) of correcting the brightness of a pixel of an input image (401) by considering at least one of the brightness change coefficient (531, 630 or 660) or the projection surface deviation correction coefficient (535) to generate an output image (403) to be projected by the image projector (440) on the projection area (120) of the projection surface (110).

[0134] For example, the tone conversion model (620 or 650) may be preset during the production process.

[0135] According to an example, when the instructions are individually or collectively executed by at least one processor (410), the image projection device (100) may be caused to perform an operation (operation 530) of obtaining an optical deviation correction coefficient (533) reflecting a vignetting characteristic (523) of a lens of the image projector (440).

[0136] For example, when the instructions are individually or collectively executed by at least one processor (410), the image projection device (100) may be caused to perform an operation (operation 530) of determining a first projection surface deviation correction coefficient based on length information (d1, d2, d3, d4) of projection beams for displaying display pixels of the projection area (120).

[0137] For example, when the instructions are individually or collectively executed by at least one processor (410), the image projection device (100) may be caused to perform an operation (operation 540) of correcting brightness for a pixel of the input image (401) by taking into account the first projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient.

[0138] For example, when the instructions are individually or collectively executed by at least one processor (410), the image projection device (100) may be caused to perform an operation (operation 530) of determining a second projection surface deviation correction coefficient based on direction information of projection beams for displaying display pixels of the projection area (120).

[0139] For example, when the instructions are individually or collectively executed by at least one processor (410), the image projection device (100) may be caused to perform an operation (operation 540) of correcting brightness for a pixel of the input image (401) by taking into account the second projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient.

[0140] For example, when the instructions are individually or collectively executed by at least one processor (410), the image projection device (100) causes the target pixel (P) on the projection surface (820) to i ) corresponding to the position of the normal vector (n) i ) and the target pixel (P i ) beam vector (r i ) by the angle of incidence (θ) i ) can cause an operation (operation 530) to be performed to determine a second projection surface deviation correction coefficient based on the second projection surface deviation correction coefficient.

[0141] For example, when the instructions are individually or collectively executed by at least one processor (410), the image projection device (100) may be caused to perform an operation (operation 540) of correcting brightness for a pixel of the input image (401) by taking into account the second projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient.

[0142] For example, when the instructions are individually or collectively executed by at least one processor (410), the image projection device (100) may be caused to perform an operation (operation 530) of determining a first projection surface deviation correction coefficient based on length information (d1, d2, d3, d4) of projection beams that will display display pixels of the projection area (120).

[0143] For example, when the instructions are individually or collectively executed by at least one processor (410), the image projection device (100) causes the target pixel (P) on the projection surface (820) to i ) corresponding to the position of the normal vector (n) i ) and the target pixel (P i ) beam vector (r i ) by the angle of incidence (θ) i ) can cause an operation (operation 530) to be performed to determine a second projection surface deviation correction coefficient based on the second projection surface deviation correction coefficient.

[0144] According to an example, when the instructions are individually or collectively executed by at least one processor (410), the image projection device (100) may be caused to perform an operation (operation 540) of correcting brightness for a pixel of the input image (401) by taking into account the first projection surface deviation correction coefficient, the second projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient.

[0145] According to an example, when the instructions are individually or collectively executed by at least one processor (410), the image projection device (100) may be caused to perform an operation (operation 530) of obtaining the optical deviation correction coefficient corresponding to each of a plurality of partial display areas (731, 733, 735, 737) from a look-up table preset based on the vignetting characteristic.

[0146] For example, the plurality of partial display areas (731, 733, 735, 737) may be projection areas that divide the projection area (120) based on changes in the vignetting characteristics.

[0147] For example, the optical deviation correction coefficient may have a relatively larger value in the first partial display area (737) at a distance from the center point of the projection area (120) than in the second partial display area (731) at a close distance.

[0148] For example, when the instructions are individually or collectively executed by at least one processor (410), the image projection device (100) may be caused to perform an operation (operation 530) of determining an angle (Φ1 or Φ2) between a first projection beam (751) for displaying a center pixel (741) of the projection area (120) and a second projection beam (753 or 755) for displaying a specific pixel (743 or 745).

[0149] According to an example, when the instructions are individually or collectively executed by at least one processor (410), the image projection device (100) may be caused to perform an operation (operation 530) of obtaining the optical deviation correction coefficient corresponding to at least one target pixel of the display pixels based on the between values ​​(Φ1 or Φ2).

[0150] For example, the optical deviation correction coefficient may have a relatively larger value for a first target pixel (743) having a narrow interval value (Φ2) than for a second target pixel (745) having a wide interval value (Φ1).

[0151] According to an example, the operation method of the image projection apparatus (100) is to input pixels (b) for a grayscale conversion model (521, 620 or 650). in or b in' ) and the above input pixel (b in or b in' ) corresponding to the output pixel (b) out2 or b out2' ) may include an operation (operation 530) of determining a brightness change coefficient (531, 630 or 660) regarding the projection surface (110). The method of operation may include an operation (operation 530) of obtaining a projection surface deviation correction coefficient (535) in which a characteristic (525) of the projection surface (110) obtained from at least one distance sensor (420) is reflected. The method of operation may include an operation (operation 540) of correcting the brightness of a pixel of an input image (401) by considering at least one of the brightness change coefficient (531, 630 or 660) or the projection surface deviation correction coefficient (535) to generate an output image (403) to be projected by the image projector (440) on the projection area (120) of the projection surface (110).

[0152] For example, the tone conversion model (620 or 650) may be preset during the production process.

[0153] According to an example, the operating method may further include an operation (operation 530) of obtaining an optical deviation correction coefficient (533) reflecting a vignetting characteristic (523) of a lens of an image projector (440).

[0154] According to an example, the operation of obtaining the projection surface polarization correction coefficient (535) may include an operation (operation 530) of determining a first projection surface deviation correction coefficient based on length information (d1, d2, d3, d4) of projection beams for displaying display pixels of the projection area (120).

[0155] According to an example, the operation of generating the output image (403) may include an operation (operation 540) of correcting brightness for a pixel of the input image (401) by taking into consideration the first projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient.

[0156] According to an example, the operation of obtaining the projection surface polarization correction coefficient (535) may include an operation (operation 530) of determining a second projection surface deviation correction coefficient based on direction information of projection beams that will display display pixels of the projection area (120).

[0157] According to an example, the operation of generating the output image (403) may include an operation (operation 540) of correcting the brightness for a pixel of the input image (401) by taking into account the second projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient.

[0158] According to an example, the operation of obtaining the projection surface polarization correction coefficient (535) is performed by obtaining the target pixel (P) in the projection surface (820). i ) corresponding to the position of the normal vector (n) i ) and the target pixel (P i ) beam vector (r i ) by the angle of incidence (θ) i ) may include an operation (operation 530) of determining a second projection surface deviation correction coefficient based on the second projection surface deviation correction coefficient.

[0159] According to an example, the operation of generating the output image (403) may include an operation (operation 540) of correcting brightness for a pixel of the input image (401) by taking into consideration the second projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient.

[0160] According to an example, the operation of obtaining the projection surface polarization correction coefficient (535) may include an operation (operation 530) of determining a first projection surface deviation correction coefficient based on length information (d1, d2, d3, d4) of projection beams that will display display pixels of the projection area (120).

[0161] According to an example, the operation of generating the output image (403) is to generate a target pixel (P) on the projection surface (820). i ) corresponding to the position of the normal vector (n) i ) and the target pixel (P i ) beam vector (r i ) by the angle of incidence (θ) i ) may include an operation (operation 530) of determining a second projection surface deviation correction coefficient based on the

[0162] According to an example, the operation (operation 530) of obtaining the optical deviation correction coefficient (533) may include an operation (operation 540) of correcting brightness for a pixel of the input image (401) by taking into consideration the first projection surface deviation correction coefficient, the second projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient.

[0163] According to an example, the operation (operation 530) of obtaining the optical deviation correction coefficient (533) may include the operation (operation 530) of obtaining the optical deviation correction coefficient corresponding to each of a plurality of partial display areas (731, 733, 735, 737) from a look-up table preset based on the vignetting characteristic.

[0164] For example, the plurality of partial display areas (731, 733, 735, 737) may be projection areas obtained by dividing the projection area (120) based on changes in the vignetting characteristics.

[0165] For example, the optical deviation correction coefficient may have a relatively larger value in the first partial display area (737) at a distance from the center point of the projection area (120) than in the second partial display area (731) at a close distance.

[0166] According to an example, the operation (operation 530) of obtaining the optical deviation correction coefficient (533) may include the operation (operation 530) of determining an angle (Φ1 or Φ2) between a first projection beam (751) to display a center pixel (741) of the projection area (120) and a second projection beam (753 or 755) to display a specific pixel (743 or 745).

[0167] According to an example, the operation (operation 530) of obtaining the optical deviation correction coefficient (533) may include the operation (operation 530) of obtaining the optical deviation correction coefficient corresponding to at least one target pixel of the display pixels based on the between values ​​(Φ1 or Φ2).

[0168] For example, the optical deviation correction coefficient may have a relatively larger value for a first target pixel (743) having a narrow interval value (Φ2) than for a second target pixel (745) having a wide interval value (Φ1).

[0169] According to one example, computer-readable instructions stored on a recording medium, when executed by at least a part of at least one processor (410) included in an image projection apparatus (100), may cause the image projection apparatus (100) to perform at least one operation. The at least one operation may include: converting an input pixel (b) to a grayscale conversion model (620 or 650) in or b in' ) and the above input pixel (b in or b in' ) corresponding to the output pixel (b) out2 or b out2' ) may include an operation (operation 530) of determining a brightness variation coefficient (630 or 660) regarding the image projector (440). The at least one operation may include an operation (operation 530) of obtaining an optical deviation correction coefficient that reflects a vignetting characteristic of a lens included in the image projector (440). The at least one operation may include an operation (operation 540) of correcting brightness for a pixel of an input image (401) by considering at least one of the brightness variation coefficient (630 or 660) and the optical deviation correction coefficient to generate an output image (403) to be projected by the image projector (440) on a projection area (120) of a projection surface (110).

[0170] According to one example, a recording medium storing computer-readable instructions may be provided. The instructions, when executed by at least a part of at least one processor included in an image projection apparatus (100), may cause the image projection apparatus (100) to perform at least one operation. The at least one operation may include: performing an input pixel (b) for a grayscale conversion model (521, 620, or 650) in or b in' ) and the above input pixel (b in or bin' ) corresponding to the output pixel (b) out2 or b out2' ) may include an operation (operation 530) of determining a brightness change coefficient (531, 630 or 660) regarding the projection surface (110). The at least one operation may include an operation (operation 530) of obtaining a projection surface deviation correction coefficient (535) in which a characteristic (525) of the projection surface (110) obtained from at least one distance sensor (420) is reflected. The at least one operation may include an operation (operation 540) of correcting the brightness of a pixel of an input image (401) by considering at least one of the brightness change coefficient (531, 630 or 660) or the projection surface deviation correction coefficient (535) to generate an output image (403) to be projected by the image projector (440) on the projection area (120) of the projection surface (110).

[0171] Electronic devices according to various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, display devices (e.g., TVs, monitors, optical projectors), portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the aforementioned devices.

[0172] The various embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" can each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish the corresponding components from other corresponding components and do not limit the corresponding components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as being "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0173] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0174] Various embodiments of the present document may be implemented as software (e.g., a program) including one or more commands stored in a storage medium (e.g., a memory (430)) readable by a machine (e.g., an image projection device (100)). For example, a processor (e.g., a processor (410)) of the machine (e.g., an image projection device (100)) may call at least one command among the one or more commands stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one command called. The one or more commands may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0175] According to one embodiment, the methods according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0176] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In the image projection apparatus (100), At least one distance sensor (ToF) (420); Image projector (440); At least one memory (430) including a non-volatile recording medium storing instructions; and At least one processor (410) operatively connected to at least one distance sensor (420), an image projector (440), and at least one memory, and including a processing circuit; When the above instructions are individually or collectively executed by at least one processor (410), they cause the image projection device (100) to perform at least one operation, At least one of the above actions: Input pixels (b) for the grayscale transformation model (521, 620 or 650) in or b in' ) and the above input pixel (b in or b in' ) corresponding to the output pixel (b) out2 or b out2' ) for determining the brightness change coefficient (531, 630 or 660) (action 530); An operation (operation 530) of obtaining a projection surface deviation correction coefficient (535) reflecting the characteristics (525) of the projection surface (110) obtained from at least one distance sensor (420); and An operation (operation 540) of generating an output image (403) to be projected by the image projector (440) on the projection area (120) of the projection surface (110) by correcting the brightness of the pixels of the input image (401) by considering at least one of the brightness change coefficient (531, 630 or 660) or the projection surface deviation correction coefficient (535). A video projection device (100) including:

2. In paragraph 1, When the above instructions are individually or collectively executed by at least one processor (410), the image projection device (100) causes: An operation (operation 530) of obtaining an optical deviation correction coefficient (533) reflecting the vignetting characteristic (523) of the lens of the above image projector (440); An operation (operation 530) of determining a first projection surface deviation correction coefficient based on length information (d1, d2, d3, d4) of projection beams for displaying display pixels of the projection area (120); and An operation for correcting the brightness of a pixel of the input image (401) by considering the first projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient (operation 540). An image projection device (100) that causes the image to be projected.

3. In paragraph 1, When the above instructions are individually or collectively executed by at least one processor (410), the image projection device (100) causes: An operation (operation 530) of obtaining an optical deviation correction coefficient (533) reflecting the vignetting characteristic (523) of the lens of the above image projector (440); An operation (operation 530) of determining a second projection surface deviation correction coefficient based on direction information of projection beams for displaying display pixels of the projection area (120); and An operation for correcting the brightness of a pixel of the input image (401) by considering the second projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient (operation 540). An image projection device (100) that causes the image to be projected.

4. In paragraph 1, When the above instructions are individually or collectively executed by at least one processor (410), the image projection device (100) causes: An operation (operation 530) of obtaining an optical deviation correction coefficient (533) reflecting the vignetting characteristic (523) of the lens of the above image projector (440); The target pixel (P) on the above projection surface (820) i ) corresponding to the position of the normal vector (n) i ) and the target pixel (P i ) beam vector (r i ) by the angle of incidence (θ) i ) to determine the second projection surface deviation correction coefficient (Operation 530); and An operation for correcting the brightness of a pixel of the input image (401) by considering the second projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient (operation 540). An image projection device (100) that causes the image to be projected.

5. In paragraph 1, When the above instructions are individually or collectively executed by at least one processor (410), the image projection device (100) causes: An operation (operation 530) of obtaining an optical deviation correction coefficient (533) reflecting the vignetting characteristic (523) of the lens of the above image projector (440); An operation (operation 530) of determining a first projection surface deviation correction coefficient based on length information (d1, d2, d3, d4) of projection beams that display display pixels of the projection area (120); The target pixel (P) on the above projection surface (820) i ) corresponding to the position of the normal vector (n) i ) and the target pixel (P i ) beam vector (r i ) by the angle of incidence (θ) i ) to determine the second projection surface deviation correction coefficient (Operation 530); and An operation for correcting the brightness of a pixel of the input image (401) by considering the first projection surface deviation correction coefficient, the second projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient (operation 540). An image projection device (100) that causes the image to be projected.

6. In any one of paragraphs 2 to 5, When the above instructions are individually or collectively executed by at least one processor (410), the image projection device (100) causes: An operation of obtaining the optical deviation correction coefficient corresponding to each of a plurality of partial display areas (731, 733, 735, 737) from a look-up table set based on the above vignetting characteristics (operation 530) causes it to perform, Here, the plurality of partial display areas (731, 733, 735, 737) are projection areas obtained by dividing the projection area (120) based on changes in the vignetting characteristics, in an image projection device (100).

7. In any one of paragraphs 2 to 5, When the above instructions are individually or collectively executed by at least one processor (410), the image projection device (100) causes: An operation (operation 530) of determining an angle (Φ1 or Φ2) between a first projection beam (751) for displaying a center pixel (741) of the projection area (120) and a second projection beam (753 or 755) for displaying a specific pixel (743 or 745); and An operation (operation 530) of obtaining the optical deviation correction coefficient corresponding to at least one target pixel of the display pixels based on the above-mentioned value (Φ1 or Φ2). An image projection device (100) that causes the image to be projected.

8. In the operating method of the image projection apparatus (100), Input pixels (b) for the grayscale transformation model (521, 620 or 650) in or b in' ) and the above input pixel (b in or b in' ) corresponding to the output pixel (b) out2 or b out2' ) for determining the brightness change coefficient (531, 630 or 660) (action 530); An operation (operation 530) of obtaining a projection surface deviation correction coefficient (535) reflecting the characteristics (525) of the projection surface (110) obtained from at least one distance sensor (420); and An operation (operation 540) of generating an output image (403) to be projected by the image projector (440) on the projection area (120) of the projection surface (110) by correcting the brightness of the pixels of the input image (401) by considering at least one of the brightness change coefficient (531, 630 or 660) or the projection surface deviation correction coefficient (535). A method of operation, comprising:

9. In paragraph 8, An operation (operation 530) of obtaining an optical deviation correction coefficient (533) reflecting the vignetting characteristics (523) of the lens of the image projector (440) Including more than, The operation of obtaining the above projection surface polarization correction coefficient (535) is: An operation (operation 530) of determining a first projection surface deviation correction coefficient based on length information (d1, d2, d3, d4) of projection beams for displaying display pixels of the above projection area (120) Including, The operation of generating the above output image (403) is as follows: An operation for correcting the brightness of a pixel of the input image (401) by considering the first projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient (operation 540). A method of operation, comprising:

10. In paragraph 8, An operation (operation 530) of obtaining an optical deviation correction coefficient (533) reflecting the vignetting characteristics (523) of the lens of the image projector (440) Including more than, The operation of obtaining the above projection surface polarization correction coefficient (535) is: An operation (operation 530) of determining a second projection surface deviation correction coefficient based on direction information of projection beams for displaying display pixels of the above projection area (120) Including, The operation of generating the above output image (403) is as follows: An operation for correcting the brightness of a pixel of the input image (401) by considering the second projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient (operation 540). A method of operation, comprising:

11. In paragraph 8, An operation (operation 530) of obtaining an optical deviation correction coefficient (533) reflecting the vignetting characteristics (523) of the lens of the image projector (440) Including more than, The operation of obtaining the above projection surface polarization correction coefficient (535) is: The target pixel (P) on the above projection surface (820) i ) corresponding to the position of the normal vector (n) i ) and the target pixel (P i ) beam vector (r i ) by the angle of incidence (θ) i ) to determine the second projection surface deviation correction coefficient (Operation 530) Including, The operation of generating the above output image (403) is as follows: An operation for correcting the brightness of a pixel of the input image (401) by considering the second projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient (operation 540). A method of operation, comprising:

12. In paragraph 8, An operation (operation 530) of obtaining an optical deviation correction coefficient (533) reflecting the vignetting characteristics (523) of the lens of the image projector (440) Including more than, The operation of obtaining the above projection surface polarization correction coefficient (535) is: An operation (operation 530) of determining a first projection surface deviation correction coefficient based on length information (d1, d2, d3, d4) of projection beams that display display pixels of the projection area (120); The target pixel (P) on the above projection surface (820) i ) corresponding to the position of the normal vector (n) i ) and the target pixel (P i ) beam vector (r i ) by the angle of incidence (θ) i ) to determine the second projection surface deviation correction coefficient (Operation 530) Including, The operation of generating the above output image (403) is as follows: An operation (operation 540) for correcting the brightness of a pixel of the input image (401) by considering the first projection surface deviation correction coefficient, the second projection surface deviation correction coefficient, the brightness change coefficient, and the optical deviation correction coefficient. A method of operation, comprising:

13. In any one of paragraphs 9 to 12, The operation (operation 530) of obtaining the optical deviation correction coefficient (533) is as follows: An operation (operation 530) of obtaining the optical deviation correction coefficient corresponding to each of a plurality of partial display areas (731, 733, 735, 737) from a look-up table set based on the above vignetting characteristics, Here, the plurality of partial display areas (731, 733, 735, 737) are projection areas obtained by dividing the projection area (120) based on changes in the vignetting characteristics, the operating method.

14. In any one of paragraphs 9 to 12, The operation (operation 530) of obtaining the optical deviation correction coefficient (533) is as follows: An operation (operation 530) of determining an angle (Φ1 or Φ2) between a first projection beam (751) for displaying a center pixel (741) of the projection area (120) and a second projection beam (753 or 755) for displaying a specific pixel (743 or 745); and An operation (operation 530) of obtaining the optical deviation correction coefficient corresponding to at least one target pixel of the display pixels based on the above-mentioned value (Φ1 or Φ2). A method of operation, comprising:

15. In any one of paragraphs 2 to 7 or paragraphs 9 to 14, The above tone conversion model (620 or 650) is set during the production process, The above optical deviation correction coefficient has a relatively larger value in the first partial display area (737) at a distance from the center point of the projection area (120) than in the second partial display area (731) at a close distance. The above optical deviation correction coefficient is an image projection device (100) or an operating method, wherein a first target pixel (743) having a narrow interval value (Φ2) has a relatively larger value than a second target pixel (745) having a wide interval value (Φ1).

Images