Image projection apparatus, method, and recording medium

Abstract

This method for operating an image projection apparatus comprises: an operation in which at least one sensor of the image projection apparatus detects first position data in a coordinate space corresponding to a plurality of sensing measurement points distributed on a projection surface onto which an optical signal corresponding to an output image is to be projected; an operation of determining second position data on a coordinate plane reflecting bending characteristics of the projection surface based on the first position data; and an operation of acquiring third position data corresponding to a plurality of pixel projection points, wherein the optical signal is projected onto a plurality of pixel projection points of a projection area of the projection surface by performing area-weighted interpolation of the second position data.

Description

Image projection device, method, and recording medium

[0001] The present disclosure relates to an image projection device, method, and recording medium for displaying an image on a projection surface.

[0002] The video device may be an analog projection device (hereinafter referred to as an “analog projection device”) or a digital projection device (hereinafter referred to as a “digital projection device”). An analog projection device may provide visual information using a medium such as film. A digital projection device may provide visual information using a digital signal. A digital projection device may include a beam projector (hereinafter referred to as a “projector”). A projector may be classified as a display device. Depending on the principle of generating light, a projector may be classified into a CRT (Cathode Ray Tube) projector, an LCD (Liquid Crystal Display) projector, or a DLP (Digital Light Processing) projector.

[0003] Projectors are primarily used to display multimedia content input directly into the projector. When connected to an electronic device (e.g., a digital TV) via a wired or wireless communication network, the projector can display multimedia content received from the electronic device.

[0004] A projector can be an electronic device that projects photographs, drawings, text, images, or videos onto a screen through a lens. Projectors are also called image projection devices. Projectors can convert image or video data in file form into optical signals (or optical images) and output them. The output of these optical signals can correspond to irradiation. The optical signals output from the projector can be projected onto a screen to display images to the viewer.

[0005] Projectors can expand their projection area by projecting images onto non-planar projection surfaces, not just flat surfaces. However, this can cause distortion in the image projected onto these non-planar projection surfaces.

[0006] The above information may be provided as background information to aid in understanding this document. None of the above is claimed to be prior art related to this document or can be used to determine prior art.

[0007] According to one embodiment, an image projection device comprises: at least one sensor; at least one memory including a nonvolatile recording medium storing instructions; an image projector configured to project an optical signal corresponding to an output image onto a projection surface; and at least one processor operatively connected to the at least one sensor, the at least one memory and the image projector, the processor including a processing circuit, wherein the instructions, when individually or collectively executed by the at least one processor, cause the image projection device to perform at least one operation, the at least one operation including: detecting first position data in a coordinate space corresponding to a plurality of sensing measurement points on the projection surface using the at least one sensor; determining second position data in a coordinate plane in which a curvature characteristic of the projection surface is reflected based on the first position data; and obtaining third position data corresponding to a plurality of pixel projection points, wherein the optical signal can be projected onto a plurality of pixel projection points in a projection area of ​​the projection surface by performing area-weighted interpolation of the second position data.

[0008] According to one embodiment, an image projection device includes an operation of detecting first position data in a coordinate space corresponding to a plurality of sensing measurement points distributed on a projection surface on which an optical signal corresponding to an output image is to be projected by at least one sensor of the image projection device; an operation of determining second position data in a coordinate plane in which a curvature characteristic of the projection surface is reflected based on the first position data; and an operation of obtaining third position data corresponding to a plurality of pixel projection points, wherein the optical signal can be projected onto a plurality of pixel projection points of a projection area of ​​the projection surface by performing area-weighted interpolation of the second position data.

[0009] According to one embodiment, a non-transitory recording medium storing at least one computer-readable instruction, the instruction, when executed by at least a part of at least one processor of an image projection device, causes the image projection device to perform at least one operation, the at least one operation including: detecting first position data in a coordinate space corresponding to a plurality of sensing measurement points distributed on a projection surface on which an optical signal corresponding to an output image is to be projected by at least one sensor of an image projection sensor; determining second position data in a coordinate plane reflecting a curvature characteristic of the projection surface based on the first position data; and obtaining third position data corresponding to a plurality of pixel projection points, wherein the optical signal can be projected onto a plurality of pixel projection points of a projection area of ​​the projection surface by performing area-weighted interpolation of the second position data.

[0010] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0011] FIG. 1 is a drawing for explaining an example of projecting an image on a curved projection surface in an image projection system according to one embodiment.

[0012] FIG. 2 is a drawing for explaining an operation of projecting an image onto a curved projection surface in an image projection system according to one embodiment.

[0013] FIG. 3 is a drawing for explaining acquisition of projection points based on measurement points in an image projection device according to one embodiment.

[0014] FIG. 4 is a block diagram for projecting image data in an image projection device according to one embodiment.

[0015] FIG. 5 is a control flowchart for obtaining position data of an area on which image data is to be projected in an image projection device according to one embodiment.

[0016] Figure 6a is a drawing for explaining the gradient distribution of sensing measurement points in the first coordinate plane.

[0017] Figure 6b is a drawing for explaining obtaining a new coordinate axis (u, v) based on the gradient distribution in the second coordinate plane.

[0018] Figure 6c is a drawing for explaining the result of performing Voronoi fragmentation.

[0019] Figure 6d is a drawing for explaining obtaining pixel projection points by performing data interpolation.

[0020] Figure 7 is a drawing for explaining an example of measuring the distance between second coordinates for Voronoi fragmentation.

[0021] Figures 8a, 8b, 8c, 8d, or 8e are drawings for explaining changes in cell shape according to the curvature characteristics of the projection surface.

[0022] Figure 9a is a drawing showing an undistorted input image input to an image projection device.

[0023] Figure 9b is a drawing showing an image displayed when an image projection device projects an input image onto a curved projection surface without correction.

[0024] Figure 9c is a drawing showing an image displayed when an image projection device corrects an input image and projects it onto a projection surface without considering the curvature characteristics of the projection surface.

[0025] Figure 9d is a drawing showing an image displayed when an image projection device corrects an input image by considering the curvature characteristics of the projection surface and then projects the image onto the projection surface.

[0026] FIG. 10 is a block diagram of an electronic device within a network environment according to one or more embodiments.

[0027] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0028] FIG. 1 is a drawing for explaining an example of projecting an image on a curved projection surface in an image projection system according to one embodiment, and FIG. 2 is a drawing for explaining an operation of projecting an image on a curved projection surface in an image projection system according to one embodiment.

[0029] Referring to FIG. 1 or FIG. 2, an image projection system may include an image projection device (100) (e.g., a beam projector) or a projection plane (110). The image projection device (100) may convert input image data (hereinafter referred to as “input image”) into an optical signal (hereinafter referred to as “output image”) and output the converted image. The output image output from the image projection device (100) may be projected on a projection plane (110) that includes a projection area (120) on which an image according to a content service such as a movie or a game is projected. The projection area (120) may include an image display area on which an image projected by the output image is actually displayed. The projection plane (110) on which the output image is projected from the image projection device (100) may be a flat surface or a non-planar surface. The image projection device (100) can convert an input image into an output image, which is an optical signal, without any special correction, and output it, if the projection surface (110) is flat, such as a screen. If the projection surface (110) is non-flat, such as a curved surface, the image projection device (100) can project an output image by performing signal processing to automatically correct (auto keystone) the input image so that the image displayed on the non-flat projection surface (110) appears as a flat image without distortion. Hereinafter, a projection surface (110) having a predetermined curvature characteristic rather than a flat surface is referred to as a 'curved projection surface (110)', and may mean, for example, a curved surface such as a curtain, a tent, or a banner. In addition, a projection surface that is not specifically referred to as a 'flat projection surface' in the present disclosure may also refer to a curved projection surface (110).

[0030] According to one example, the image projection device (100) can perform processing on image data to be output as an optical signal by taking into account the curvature characteristics of the projection surface (110). The curvature characteristics of the projection surface (110) may be related to the shape in which the projection surface (110) is bent or curved. The curvature characteristics of the projection surface (110) may include, for example, the characteristics of a wave that forms a crest and a root in a predetermined direction, such as a horizontal (or left-right), vertical (or up-down), or diagonal direction.

[0031] FIG. 3 is a drawing for explaining acquisition of projection points based on measurement points in an image projection device (e.g., image projection device (100) of FIG. 2) according to one embodiment.

[0032] Referring to FIG. 3, the image projection device (100) can obtain position data (hereinafter, referred to as “first position data”) corresponding to a plurality of measurement points (311) (hereinafter, referred to as “sensing measurement points (311)”) included in a curved projection surface (e.g., projection surface (110) of FIG. 1) using a distance sensor such as a ToF (time of flight) sensor (e.g., the distance sensor (420) of FIG. 4). The first position data may include, for example, a space orthogonal coordinate (or three-dimensional (3D) orthogonal coordinate system) (hereinafter, referred to as “space orthogonal coordinate system”) corresponding to the position of each of the sensing measurement points (311) in a coordinate space. The space orthogonal coordinate system corresponding to the position of each of the sensing measurement points (311) may be referred to as a “first coordinate value.”

[0033] For example, the projection surface (110) may include about 250 sensing measurement points (311). In this case, the first position data may include about 250 first coordinate values. The first graph (310) is a diagram of the first coordinate values ​​acquired corresponding to the sensing measurement points (311). According to the first graph (310), the sensing measurement points (311) may be irregularly distributed due to the curvature characteristic of the projection surface (110). For example, the distance between the measurement points distributed near the peak and / or valley of the projection surface (110) may be relatively narrow compared to the distance between the measurement points distributed on the slope. Considering this characteristic, the first position data acquired by the image projection device (100) for the sensing measurement points (311) may be scattered data that is not standardized. Here, 'formalized' means that there is a specific structure or order established between relative positions.

[0034] The image projection device (100) can perform data interpolation (330) to obtain position data corresponding to a plurality of projection points (321) (hereinafter, referred to as “pixel projection points”) based on first position data corresponding to a small number of sensing measurement points (311). According to one example, the position data corresponding to the pixel projection points (321) can correspond to pixel projection points (321) on which pixels of the output image (403) are to be projected onto a projection area (e.g., projection area (120) of FIG. 1 or 2) by the image projection device (100) performing data interpolation (330) using the first position data. The second graph (320) represents second coordinate values ​​corresponding to the pixel projection points (321).

[0035] According to an example, the image projection device (100) may perform a preprocessing process on the first position data before performing data interpolation (330). For example, the image projection device (100) may determine second position data on a coordinate plane reflecting the curvature characteristics of the projection surface (110) based on the first position data. The image projection device (100) may perform data interpolation using the second position data to obtain third position data corresponding to pixel projection points (321) on which an optical signal is to be projected in a projection area (120) of the projection surface (110). The third position data may include a spatial orthogonal coordinate system corresponding to the position of each pixel projection point (321) in the coordinate space. Detailed operations according to the preprocessing process by the image projection device (100) will be described later.

[0036] FIG. 4 is a block diagram (400) for projecting image data in an image projection device (e.g., the image projection device (100) of FIG. 2) according to one embodiment.

[0037] Referring to FIG. 4, the image projection device (100) may include at least one processor (410) (hereinafter, referred to as 'processor (410)'), at least one sensor, at least one memory (430) (hereinafter, referred to as 'memory (430)'), or an image projector (440). The at least one sensor may include a distance sensor (420). The distance sensor (420) may be a ToF sensor or a ToF camera.

[0038] The distance sensor (420) can obtain position data (hereinafter, referred to as “first position data” or “first coordinate value”) corresponding to a plurality of measurement points (e.g., sensing measurement points (311) of FIG. 3) included in a projection surface (e.g., projection surface (110) of FIG. 1 or 2). The sensing measurement points (311) may be distributed on the projection surface (110). In one example, when the projection surface (110) is curved, the sensing measurement points (311) may not be uniformly distributed on the projection surface (110), but may be irregularly distributed. For example, assuming a distance sensor (420) that transmits beams so that the measurement points are evenly distributed on a flat projection surface, the beams transmitted by the distance sensor (420) can provide a distribution of measurement points whose spacing becomes narrower in proportion to the inclination of the curved projection surface (110). That is, the measurement points existing in an area with a steep slope (hereinafter referred to as the 'first slope') on the curved projection surface (110) can be distributed at relatively wider intervals than the measurement points existing in an area with a relatively shallow slope (hereinafter referred to as the 'second slope'). Therefore, the density of the measurement points on the first slope can be relatively lower than the density of the measurement points on the second slope. For example, the first slope can be distinguished based on the difference in the degree of relative inclination with respect to the second slope due to the curvature of the projection surface (110), and this can be an exemplary assumption. An example in which the sensing measurement points (331) are distributed on the projection surface (110) can be referred to FIG. 3.

[0039] The memory (430) can store various data used by at least one component (e.g., the processor (410) or the distance sensor (420)) of the image projection device (100). The data can include, for example, input data or output data for software (e.g., a program) and software-related commands. The memory (430) can include volatile memory or non-volatile memory. The program can be stored as software in the memory (430), for example. In one example, the memory (430) can include an operating system, middleware, or an application.

[0040] The image projector (440) may be configured to output an output image (403) to be projected onto a projection area (120) of a projection surface (110) as an optical signal for screen output. The image projector (440) may, for example, convert an electrical signal provided from a processor (410) into an output image (403) to be projected as an optical signal and output it toward the projection area (120). The electrical signal provided by the processor (410) may correspond to image data such as a photograph or a video.

[0041] The processor (410) may execute software to control at least one other component (e.g., hardware or software component), such as an electrically connected distance sensor (420) or an image projector (440), or to perform processing or calculation of various data. As at least a part of data processing or calculation, the processor (410) may store commands or data received from other components (e.g., a distance sensor (420), a user interface (I / F), or a transceiver) in a memory (430) (e.g., a volatile memory), or process commands or data stored in the memory (430) and store processed result data in the memory (430).

[0042] The processor (410) may be implemented as one or more integrated circuit (IC) chips and may perform various data processing operations. For example, the processor (410) (or application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or chipset). The processor (410) may include sub-components including a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a display controller, a memory controller, a storage controller, a communication processor (CP), and / or a sensor interface. The sub-components are merely exemplary. For example, the processor (410) may further include other sub-components. For example, some sub-components may be omitted from the processor (410). For example, some sub-components may be included as separate components of the image projection device (100) outside the processor (410). For example, some subcomponents may be contained within other components (e.g., a display, an image sensor).

[0043] The processor (410) (e.g., CPU or central processing circuit) may be configured to control sub-components based on the execution of instructions stored in the memory (430) (e.g., volatile memory and / or non-volatile memory). In one example, the GPU (or graphics processing circuit) included in the processor (410) may be configured to execute parallel operations (e.g., rendering). In one example, the NPU (or neural processing circuit) included in the processor (410) may be configured to execute operations for an artificial intelligence model (e.g., convolution computation). In one example, the ISP (or image signal processing circuit) included in the processor (410) may be configured to process a raw image (401) acquired through an image sensor into a format suitable for a component in the image projection device (100) or a sub-component in the processor (410). In one example, a display controller (or display control circuit) included in a processor (410) may be configured to process an image (401) obtained from a CPU, GPU, ISP, or memory (430) (e.g., volatile memory) into a format suitable for projecting onto a projection surface (e.g., projection surface (110) of FIG. 1 or 2).

[0044] In one example, a memory controller (or memory control circuit) included in the processor (410) may be configured to control reading data from volatile memory and writing data to volatile memory. In one example, a storage controller (or storage control circuit) included in the processor (410) may be configured to control reading data from non-volatile memory and writing data to non-volatile memory.

[0045] In one example, the CP (communication processing circuit) included in the processor (410) may be configured to process data acquired from a sub-component within the processor (410) into a format suitable for transmission to another electronic device via a transceiver, or to process data acquired from another electronic device (e.g., a remote controller) via a transceiver into a format suitable for processing by the sub-component. In one example, the sensor interface (or sensing data processing circuit, sensor hub) included in the processor (410) may be configured to process data on the status of the image projection device (100) and / or the status of the surroundings of the image projection device (100), acquired via an internal sensor (e.g., a distance sensor (ToF (time-of-flight) sensor) (420)) or an external sensor (e.g., one or more position measurement sensors (anchors)), into a format suitable for the sub-component within the processor (410).

[0046] According to one example, the processor (410) may process image data to be output as an optical signal by considering the curvature characteristic of the projection surface (110) (or projection area (120)) by at least one sensor (420). The curvature characteristic of the projection surface (110) may be related to the shape in which the projection surface (110) is bent or curved. The curvature characteristic of the projection surface (110) may be the same as described above with reference to FIG. 3. For example, if the projection surface (110) is non-planar (e.g., a curved surface as shown in FIG. 2), the processor (410) may perform an operation to automatically correct (auto keystone) the input image (401) so that the image displayed on the non-planar projection surface (110) can appear as a planar image without distortion.

[0047] To explain in detail, the processor (410) can detect first position data (or first coordinate value (P1(x,y,z))) in a coordinate space corresponding to a plurality of sensing measurement points (311) distributed on a projection surface (110) by at least one sensor (420).

[0048] The processor (410) can determine second position data on a coordinate plane that reflects the curvature characteristic of the projection surface (110) based on the detected first position data. For example, the curvature characteristic of the projection surface (110) can include information about the directionality of a wave that propagates in a single direction on the projection surface (110). According to an example, the processor (410) can obtain the inclination (gx, gy) of the sensing measurement points (331) on a first coordinate plane (e.g., the coordinate plane (610) of FIG. 6A) along a predetermined coordinate axis (x, y) by the first position data (see FIG. 6A). The processor (410) can determine the coordinate axes (e.g., v (625), u (627) of FIG. 6B) of the second coordinate plane that will determine the second position data by reflecting the distribution of the obtained inclination (see FIG. 6B). For example, the processor (410) can check the distribution of the acquired slope for the sensing measurement points (331) on a coordinate plane (e.g., the coordinate plane (620) of FIG. 6B) with the slopes (gx, gy) of the sensing measurement points (331) as coordinate axes (e.g., gy(621), gx(623) of FIG. 6B). For example, the distribution of the slope on a coordinate plane (e.g., the coordinate plane (620) of FIG. 6B) with the slopes (gx, gy) as coordinate axes (e.g., gy(621), gx(623) of FIG. 6B) may have a distribution slanted from the upper left to the lower right (see FIG. 6B). According to an example, the distribution of the technique may be affected by the curvature characteristics of the projection surface (110), i.e., the direction (or slope) in which the wave propagates on the projection surface (110). The processor (410) can determine an eigenvector regarding direction and an eigenvalue regarding slope based on the distribution of the acquired slope. The processor (410) can obtain the coordinate axes (u, v) of the second coordinate plane based on the eigenvector and eigenvalue.The image projection device (100) can determine the size of the coordinate axes (u, v) on the second coordinate plane, for example, by the ratio of the eigenvalues. The processor (410) can determine the coordinate values ​​(u value, v value)) of the planar orthogonal coordinate system corresponding to the first coordinate value (P1(x,y,z)), which is the first position data, on the second coordinate plane, as the second position data.

[0049] The processor (410) may perform area-weighted interpolation using the determined second position data to obtain third position data corresponding to a plurality of pixel projection points (321) on which the light signal (403) is to be projected onto the projection area (120) of the projection surface (110). According to an example, the processor (410) may identify a planar orthogonal coordinate system (u, v) of the sensing measurement points (311) in the second coordinate plane based on the second position data. The processor (410) may determine a distance between the sensing measurement points (311) in the second coordinate plane using the identified planar orthogonal coordinate system (u, v). The processor (410) may perform Voronoi fragmentation based on the distance between the determined sensing measurement points (e.g., pixel projection points (sensing measurement points) (611) of FIG. 6C). The processor (410) may perform scattered data interpolation on the first cells (e.g., cell (631) of FIG. 6c) obtained as a result of Voronoi fragmentation (e.g., 630 of FIG. 6c) to obtain second cells corresponding to pixel projection points (e.g., pixel projection points (643) of FIG. 6d) (see FIG. 6d). For example, the shapes of the first cells and / or the second cells may be narrowed in a direction in which a slope exists on the projection surface (110) depending on the curvature characteristics (see FIG. 8b, FIG. 8c, FIG. 8d, or FIG. 8e).

[0050] According to one example, the processor (410) may perform data interpolation (e.g., interpolation (330) of FIG. 3) to obtain second position data corresponding to a plurality of pixel projection points (e.g., projection points (321) of FIG. 3 or projection points (643) of FIG. 6D) from first position data corresponding to a small number of sensing measurement points (e.g., measurement points (311) of FIG. 3 or measurement points (641) of FIG. 6D). The first position data may be obtained by the distance sensor (420) for the sensing measurement points (311, 641) included in the curved projection surface (110). A plurality of second position data can be obtained for pixel projection points (321, 643) at which pixels of an output image (403) are to be projected in a projection area (e.g., projection area (120) of FIG. 1 or FIG. 2) by performing data interpolation using the first position data by the processor (410).

[0051] The processor (410) can process the input image (401) based on the second position data to generate an output image (403) so that the image to be projected on the curved projection area (120) can be viewed as a flat image to the viewer (e.g., the viewer (130) of FIG. 2).

[0052] In one example, the processor (410) may include a measurement point acquisition module (411), a projection point acquisition module (413), and / or an image correction module (415). The term "module" (e.g., measurement point acquisition module (411), projection point acquisition module (413), and image correction module (415)) as used in the present disclosure may include units implemented in hardware, software, or firmware, and may be used interchangeably with other terms such as "logic," "logic block," "component," or "circuit." A "module" may be a single integrated component configured to perform one or more functions, or may be a minimum unit or a portion thereof. For example, in one example, the module may be implemented in the form of an application-specific integrated circuit (ASIC). As illustrated in FIG. 4, the measurement point acquisition module (411), the projection point acquisition module (413), and the image correction module (415) may be included in the processor (410). Accordingly, the measurement point acquisition module (411), the projection point acquisition module (413), and the image correction module (415) may have a hardware structure corresponding to the structure of the processor (410).

[0053] As an example, the measurement point acquisition module (411), the projection point acquisition module (413), and the image correction module (415) may be computer codes loaded into the processor (410). Accordingly, the measurement point acquisition module (411), the projection point acquisition module (413), and the image correction module (415) may have a structure corresponding to the computer code. The measurement point acquisition module (411), the projection point acquisition module (413), and the image correction module (415) may be replaced with or interchanged with the measurement point acquisition code, the projection point acquisition code, and the image correction code, respectively.

[0054] The measurement point acquisition module (411) can acquire first position data corresponding to the sensing measurement points (311) included in the projection surface (110) based on the sensing value measured by the distance sensor (420). The first position data can include first coordinate values ​​(P1(x,y,z)) corresponding to each of the sensing measurement points (311). The first coordinate values ​​(P1(x,y,z)) can be a spatial orthogonal coordinate system acquired targeting a coordinate space. For example, the first coordinate values ​​(P1(x,y,z)) can be defined as position data (x value, y value) corresponding to a planar orthogonal coordinate system (or a two-dimensional orthogonal coordinate system) corresponding to a coordinate plane and position data (z value) corresponding to a depth or distance.

[0055] The projection point acquisition module (413) can determine second position data reflecting the curvature characteristic of the projection surface (110). In one example, the curvature characteristic of the projection surface (110) can include information about the directionality of a wave that propagates in a single direction on the projection surface (110). For example, the direction in which the wave propagates on the projection surface (110) can be a horizontal direction from left to right or from right to left. For example, the direction in which the wave propagates on the projection surface (110) can be a vertical direction from top to bottom or from bottom to top. For example, the direction in which the wave propagates on the projection surface (110) can be a first diagonal direction from the lower left corner to the upper right corner or from the upper right corner to the lower left corner. For example, the direction in which the wave propagates on the projection surface (110) can be a second diagonal direction from the upper left corner to the lower right corner or from the lower right corner to the upper left corner. For example, in FIG. 3, the wave can proceed in a horizontal direction on the projection surface (110).

[0056] According to an example, the projection point acquisition module (413) can acquire second position data of pixel projection points (321) by performing interpolation on the first position data based on a predetermined interpolation method, considering that the first position data of the sensing measurement points (311) are scattered data. The projection point acquisition module (413) can determine the coordinate axes (u, v) of the second coordinate plane reflecting the curvature characteristics of the projection surface (110) based on the slope distribution as a preprocessing step for performing interpolation on the first position data (see FIG. 6b).

[0057] Specifically, the projection point acquisition module (413) can acquire the local gradient (gx, gy) for each of the first coordinate values ​​(P1(x, y, z)) of the sensing measurement points (311, 641) in the first coordinate plane (e.g., the coordinate plane (610) of FIG. 6a).

[0058] The following <Mathematical Formula 1> describes a formula for obtaining the local gradient (gx, gy) for each of the first coordinate values ​​(P1(x,y,z)).

[0059]

[0060] The projection point acquisition module (413) can determine the coordinate axes (u, v) of the second coordinate plane for determining the second position data by reflecting the distribution of the acquired slopes (see FIG. 6b). The projection point acquisition module (413) can determine the coordinate axes (u, v) of the second coordinate plane based on, for example, the eigenvectors (e1, e2) regarding the direction and the eigenvalues ​​(L1, L2) regarding the slopes based on the distribution of the acquired slopes. The vectors u and v that determine the coordinate axes (u, v) can always be orthogonal. For example, the ratio of the eigenvalues ​​(L1, L2) regarding the slopes can determine the magnitudes of the vectors u and v (see FIG. 8a, FIG. 8b, FIG. 8c, FIG. 8d, or FIG. 8e).

[0061] The following <Mathematical Formula 2> describes a formula for determining the coordinate axes (u, v) of the second coordinate plane.

[0062]

[0063] Here, L1 or L2 is an eigenvalue for the gradient, and e1 or e2 is an eigenvector for the direction.

[0064] The projection point acquisition module (413) can determine the eigenvector (e1, e2) regarding the direction and the eigenvalue (L1, L2) regarding the slope based on the distribution of the acquired slope. For example, the projection point acquisition module (413) can determine the eigenvector (e1, e2) which is an axis that can most optimally compress the distribution of the slope and the eigenvalue (L1, L2) which is a weight by considering the eigendistribution (eigenanalysis) of the slope.

[0065] The following <Mathematical Formula 3> describes a formula for determining the eigenvalues ​​(L1, L2) of the slope.

[0066]

[0067] Here, T is A(1)-A(4), D is A(1)A(4)-A(2)A(3), and the covariance matrix (A) is am.

[0068] The following <Mathematical Formula 4> describes a formula for determining the eigenvector (e1, e2) regarding the direction.

[0069]

[0070] The projection point acquisition module (413) can acquire second coordinate values ​​(u, v) of the second plane orthogonal coordinate system by projecting the coordinate values ​​(x, y) in the first coordinate plane included in the first coordinate values ​​(P1(x, y, z)) of the sensing measurement points (311, 641) acquired in the first coordinate plane (610) onto the second plane orthogonal coordinate system by a new coordinate axis (u, v).

[0071] The number of pixels of an image projected onto a projection area (120) of a projection surface (110) by an image projection device (100) (e.g., the number of pixel projection points (321, 643) of FIG. 3 or FIG. 6d) may be relatively greater than the number of sensing measurement points (311, 641). Therefore, the projection point acquisition module (413) may acquire position data of pixel projection points (321, 643) based on position data of sensing measurement points (311, 641) by using a specific interpolation technique.

[0072] According to an example, the projection point acquisition module (413) can perform scattered data interpolation using the second position data to acquire third position data corresponding to a plurality of pixel projection points (643) on which the light signal (403) is to be projected onto the projection area (120) of the projection surface (110).

[0073] As an example, the projection point acquisition module (413) can acquire third position data corresponding to a plurality of pixel projection points (643), wherein the optical signal (403) is projected onto a plurality of pixel projection points (643) of a projection area (120) of a projection surface (110) by performing scattering data interpolation of the second position data.

[0074] For example, the scattered data interpolation method may be a natural neighbor interpolation method. The natural neighbor interpolation method may include, for example, an area-weighted interpolation method. The area-weighted interpolation method may be a scattered data interpolation method based on Voronoi tessellation. Voronoi tessellation is a technique for dividing a specific area (e.g., a projection area (120)) by cells that each use a plurality of positional data (e.g., two-dimensional positional information (x, y)) existing on a plane as reference positional data. In this case, the internal points (P2 (x2, y2)) included in the cell may be closer to the internal point (P1 (x1, y1)) corresponding to the reference positional data of the cell than to other cells, which may be explained by the following <Mathematical Formula 5>.

[0075]

[0076] For example, the projection point acquisition module (413) can determine the Euclidean distance as a preprocessing step, as the area-weighted interpolation method is based on Voronoi fragmentation (see Fig. 7). The Euclidean distance can be generally applied in cluster analysis, which divides groups based on the distance between data.

[0077] The following <Mathematical Equation 6> defines the Euclidean distance.

[0078]

[0079] Here, u1 is and u2 is and v1 is and v2 is am.

[0080] For example, the projection point acquisition module (413) can perform area-weighted interpolation using the second coordinate values ​​(u, v) on the second coordinate plane, which is the second location data, to acquire the coordinate values ​​(u', v') of the new location (x). For example, the projection point acquisition module (413) can acquire coordinate values ​​(f(x)) located around the new location (x) to be acquired. i )) with weight (w) i (x)) can be multiplied and added to obtain the value (G(x)) at the new location (x). The following <Mathematical Formula 7> defines the value (G(x))) at the new location (x).

[0081]

[0082] Here, w i (x) is And, is the size of the intersection area between the new cell and the existing cell, and A(x) is the size of the new cell.

[0083] As described above, the value (G(x)) at the new location (x) can be calculated based on the size of the area that the newly created cell at the new location (x) takes from the surrounding cells.

[0084] By repeatedly performing the above-described operation, the projection point acquisition module (413) can acquire third position data, which is a coordinate value in the coordinate space corresponding to the pixel projection points (321, 641).

[0085] The image correction module (415) can correct the input image (401) based on the third position data acquired by the projection point acquisition module (413) so that the image to be projected on the curved projection area (120) can be seen as a flat image to the viewer (e.g., the viewer (130) of FIG. 2). The image correction module (415) can provide the corrected image to the image projector (440).

[0086] In one example, the image projection device (100) may include additional components such as a user interface (I / F). For example, the user interface (I / F) may be configured to receive information from a user. The user interface (I / F) may receive commands or data to be used in a component (e.g., processor (410)) of the image projection device (100) from an external source (e.g., a user) of the image projection device (100). The user interface (I / F) may include, for example, a microphone, a mouse, a keyboard, keys (e.g., buttons), a remote control, or a digital pen (e.g., a stylus pen). In one example, the user interface (I / F) may be configured to transmit information to the user. The user interface (I / F) may output an audio signal to the external source of the image projection device (100) through a component such as a speaker. For example, the speaker may be used for general purposes such as multimedia playback or recording playback.

[0087] In one example, the image projection device (100) may include additional components such as a transceiver. The transceiver may be configured to exchange information with at least one electronic device. The transceiver may transmit and receive data or signals with a remote control device or external sensors under the control of the processor (410).

[0088] According to an example, the transmitter and receiver may include, but are not limited to, a Bluetooth communication unit, a BLE (Bluetooth low energy) communication unit, a near field communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an infrared (IrDA, infrared data association) communication unit, a WFD (Wi-Fi Direct) communication unit, a UWB (ultra-wideband) communication unit, an Ant+ communication unit, or a microwave (uWave) communication unit, depending on the performance and structure of the image projection device (100).

[0089] According to one example, the transceiver may support establishing a direct (e.g., wired) communication channel or a wireless communication channel with a remote control device and performing communication through the established communication channel. The transceiver may include one or more CPs that support direct (e.g., wired) communication or wireless communication. One or more CPs may operate independently with respect to the processor (410). The transceiver may include, for example, a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules may communicate with at least one remote control device, which is an external electronic device, via a network (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or IrDA, or a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a WAN)). These different types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips).

[0090] According to one example, the image projection device (100) may include an external sensor as an additional component. Sensing data acquired through the external sensor may include information to be used to acquire the position of the image projection device (100). The processor (410) may use the sensing data to identify the position of the image projection device (100). The processor (410) may perform image processing on the input image (401) based on the position of the image projection device (100) to generate an output image (403) to be projected onto the projection area (120) of the projection surface (110) through the image projector (440). The output image (403) to be projected through the image projector (440) may be a corrected image so that it can be displayed as a flat surface without distortion due to a curved surface of the projection area (120) at the position of the image projection device (100).

[0091] FIG. 5 is a control flowchart (500) for obtaining position data of an area (e.g., projection area (120) of FIG. 1 or FIG. 2) on which image data is to be projected in an image projection device (e.g., image projection device (100) of FIG. 2) according to one embodiment.

[0092] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0093] Referring to FIG. 5, the image projection device (100) can, in operation 510, obtain position data (hereinafter, referred to as “first position data” or “first coordinate value”) corresponding to a plurality of measurement points (e.g., sensing measurement points (311) of FIG. 3) (hereinafter, referred to as “sensing measurement points (311)”) included in a projection surface (e.g., projection surface (110) of FIG. 1 or 2). The sensing measurement points (311) may be distributed on the projection surface (110). In one example, when the projection surface (110) is a curved surface, the sensing measurement points (311) may not be uniformly distributed on the projection surface (110), but may be arranged in an irregularly distributed manner. For example, assuming at least one sensor (420) that transmits beams so that the measurement points are evenly distributed on a projection surface of a plane, the beams transmitted by the sensor (420) can provide a distribution of measurement points whose spacing becomes narrower in proportion to the inclination of the curved projection surface (110). That is, measurement points existing in an area with a steep incline (hereinafter referred to as the “first inclined surface”) on the curved projection surface (110) can be distributed at relatively wider intervals than measurement points existing in an area with a relatively shallower incline (hereinafter referred to as the “second inclined surface”). Accordingly, the density of measurement points on the first inclined surface can be relatively lower than the density of measurement points on the second inclined surface. For example, the first inclined surface can be distinguished based on the difference in the degree of inclination relative to the second inclined surface due to the curvature of the projection surface (110), which can be an exemplary assumption. An example of sensing measurement points (331) being distributed on the projection surface (110) can be seen in FIG. 3.

[0094] According to an example, the reason why the density of the measurement points on the first and second slopes is different is because the distance (z value) from the image projection device (100) (or the viewer (e.g., the viewer (130) of FIG. 2)) to the sensing measurement points (311) distributed on the projection surface (110) may be different due to the curvature of the projection surface (110). The curvature of the projection surface (110) may be a cause of the distance at which the signal (e.g., an infrared (IR) beam) transmitted using at least one sensor (e.g., a distance sensor (420) of FIG. 4 (e.g., a time of flight (ToF) sensor)) reaches the corresponding measurement point in order for the image projection device (100) to obtain the first coordinate value corresponding to the sensing measurement points (311) is different. The number of sensing measurement points (311) on the projection surface (110) may be determined by the resolution of at least one sensor (420). There are. For example, the number of sensing measurement points (311) determined by the resolution of at least one sensor (420) may be '240 × 180'. For example, the number of sensing measurement points (311) determined by the resolution of at least one sensor (420) may be '640 × 480'. The number of pixels of the image projected onto the projection area (120) of the projection surface (110) by the image projection device (100) (e.g., the number of pixel projection points (321) of FIG. 3) may be relatively greater than the number of sensing measurement points (311). According to an example, the image projection device (100) may obtain the position data of the pixel projection points (321) based on the position data of the sensing measurement points (311) by using a specific interpolation technique. A detailed description thereof will be described later.

[0095] According to an example, the image projection device (100) can obtain first coordinate values ​​(P1(x,y,z)) as first position data corresponding to a plurality of measurement points (641) included in a curved projection surface (110). The first coordinate values ​​(P1(x,y,z)) may be a spatial orthogonal coordinate system obtained targeting a coordinate space. For example, the first coordinate values ​​(P1(x,y,z)) may be defined as position data (x value, y value) corresponding to a planar orthogonal coordinate system (or a two-dimensional (2D) orthogonal coordinate system) corresponding to a coordinate plane and position data (z value) corresponding to a depth or distance.

[0096] According to the above, the image projection device (100) can detect first position data (or first coordinate values ​​(P1(x,y,z))) in a coordinate space corresponding to sensing measurement points (311) distributed on the projection surface (110) by at least one sensor (420).

[0097] The image projection device (100) can determine position data (hereinafter, referred to as 'second position data' or 'second coordinate values') reflecting the curvature characteristic of the projection surface (110) in operation 520. The image projection device (100) can, for example, determine second coordinate values ​​in a planar rectangular coordinate system reflecting the inclination distribution of the sensing measurement points (311) as the second position data based on the first position data. According to an example, the curvature characteristic of the projection surface (110) can include information about the directionality of a wave that propagates in a single direction on the projection surface (110). For example, the direction in which the wave propagates on the projection surface (110) can be a horizontal direction from left to right or from right to left. For example, the direction in which the wave propagates on the projection surface (110) can be a vertical direction from top to bottom or from bottom to top. For example, the direction in which the wave propagates on the projection surface (110) may be a first diagonal direction from the lower left corner to the upper right corner or from the upper right corner to the lower left corner. For example, the direction in which the wave propagates on the projection surface (110) may be a second diagonal direction from the upper left corner to the lower right corner or from the lower right corner to the upper left corner. As an example, in FIG. 3, the wave is horizontal on the projection surface (110).

[0098] Specifically, the image projection device (100) can obtain the inclination (gx, gy) of the sensing measurement points (331) on the first coordinate plane by the coordinate axes (x, y) based on the first position data (see FIG. 6a). The image projection device (100) can determine the coordinate axes (e.g., v (625), u (627) of FIG. 6b) of the second coordinate plane for determining the second position data by reflecting the distribution of the obtained inclination (see FIG. 6b). For example, the image projection device (100) can confirm the distribution of the inclination obtained for the sensing measurement points (331) on the coordinate plane (e.g., the coordinate plane (620) of FIG. 6b) that has the inclination (gx, gy) of the sensing measurement points (331) as the coordinate axes (e.g., gy (621), gx (623) of FIG. 6b). For example, in a coordinate plane (e.g., coordinate plane (620) in FIG. 6b) with the slope (gx, gy) as the coordinate axis (e.g., gy(621), gx(623) in FIG. 6b), the distribution of the slope may have a distribution that slopes from the upper left to the lower right (see FIG. 6b). For example, the distribution of the technique may be influenced by the curvature characteristics of the projection surface (110), i.e., the direction (or slope) in which the wave propagates on the projection surface (110).

[0099] The image projection device (100) can determine an eigenvector for direction and an eigenvalue for slope based on the distribution of the acquired slope. The image projection device (100) can obtain the coordinate axes (u, v) of the second coordinate plane based on the eigenvector and the eigenvalue. For example, the image projection device (100) can obtain the coordinate axes (u, v) of the second coordinate plane by the <Mathematical Formula 1> defined above. The image projection device (100) can determine the size of the coordinate axes (u, v) in the second coordinate plane by the ratio of the eigenvalues. The image projection device (100) can determine the coordinate values ​​(u value, v value)) which are the planar orthogonal coordinates corresponding to the first coordinate value (P1(x, y, z)), which is the first position data, in the second coordinate plane as the second position data.

[0100] According to the above, the image projection device (100) can determine the second position data on a coordinate plane that reflects the curvature characteristics of the projection surface (110) based on the first position data. According to an example, the image projection device (100) can obtain the inclination of the sensing measurement points (331) on the first coordinate plane using the first position data. The image projection device (100) can determine the coordinate axes (u, v) of the second coordinate plane for determining the second position data by reflecting the distribution of the obtained inclination. For example, the image projection device (100) can determine the eigenvector for the direction and the eigenvalue for the slope based on the distribution of the obtained inclination. The image projection device (100) can obtain the coordinate axes (u, v) of the second coordinate plane based on the eigenvector and the eigenvalue. The image projection device (100) can determine the size of the coordinate axes (u, v) in the second coordinate plane, for example, by the ratio of the eigenvalues.

[0101] The image projection device (100), in operation 530, may perform a predetermined data interpolation (e.g., scattered data interpolation) using the second position data to obtain third position data corresponding to pixel projection points (643) on which the light signal (403) is to be projected in the projection area (120) of the projection surface (110). The third position data may include a spatial orthogonal coordinate system corresponding to the position of each of the pixel projection points (321) in the coordinate space. For example, the scattered data interpolation may be a natural proximity interpolation method. The natural proximity interpolation method may include, for example, an area-weighted interpolation method. The area-weighted interpolation method may be a scattered data interpolation method based on Voronoi fragmentation. Voronoi fragmentation is a technique for dividing a specific area (e.g., the projection area (120)) by cells that use each of a plurality of position data (e.g., two-dimensional position information (x, y)) existing on a plane as reference position data. In this case, the internal points contained in the cell may be closer to the internal points corresponding to the reference position data of that cell than to other cells.

[0102] According to the above, the image projection device (100) can identify the planar orthogonal coordinate system (u, v) of the sensing measurement points (311) on the second coordinate plane based on the second position data, and determine the distance between the sensing measurement points (311) on the second coordinate plane using the identified planar orthogonal coordinate system (u, v). The image projection device (100) can perform Voronoi fragmentation based on the distance between the determined sensing measurement points (641), and perform scattered data interpolation on the first cells obtained as a result of the Voronoi fragmentation to obtain second cells corresponding to the pixel projection points (321). The shape of the first cells and / or the second cells can be narrowed in the direction in which the slope exists on the projection surface (110) depending on the curvature characteristic (see FIG. 8a, FIG. 8b, FIG. 8c, FIG. 8d, or FIG. 8e).

[0103] FIG. 6a is a drawing for explaining the gradient distribution of sensing measurement points (e.g., measurement point (311) of FIG. 3) in the first coordinate plane, FIG. 6b is a drawing for explaining obtaining a new coordinate axis (u, v) based on the gradient distribution in the second coordinate plane, FIG. 6c is a drawing for explaining the result of performing Voronoi fragmentation, and FIG. 6d is a drawing for explaining obtaining pixel projection points by performing data interpolation.

[0104] The first graph (610) of FIG. 6a illustrates the distribution of each sensing measurement point (611) and the distribution of the inclination (gx, gy) based on the first position data corresponding to the sensing measurement points (331) in the first coordinate plane by the predetermined coordinate axes (x, y) (617, 615). Each sensing measurement point (611) may be irregularly distributed in the first coordinate plane. It can be confirmed that the inclination at each sensing measurement point (611) has a predetermined directionality (613) by reflecting the curvature characteristic of the projection surface (110).

[0105] The second graph (620) of Fig. 6b illustrates the distribution of the slope (gx, gy) values ​​using the slope (gx, gy) of the sensing measurement points (611) as the coordinate axes. The distribution of the slope (gx, gy) values ​​illustrated in the second graph (620) reflects the curvature characteristic of the projection surface (110). The two vectors (u, v) (627, 625) to be defined as the coordinate axes for the second coordinate plane (629) can be determined based on the distribution of the slope (gx, gy) values ​​that reflect the curvature characteristic that can be confirmed by the second graph (620).

[0106] The third graph (630) of Fig. 6c illustrates a cell structure (631) corresponding to the result of performing Voronoi fragmentation using the second coordinate values ​​(u, v) corresponding to each sensing measurement point (611) on the second coordinate plane (629) by the new coordinate axes (u, v) (627, 625). The illustrated cell structure (631) may narrow in the direction in which the slope exists on the projection surface (110) depending on the curvature characteristics.

[0107] The fourth graph (640) of FIG. 6d illustrates pixel projection points (643) obtained by performing scattered data interpolation on the second coordinate values ​​(u, v) corresponding to each sensing measurement point (611) in the second coordinate plane (629) by the new coordinate axes (u, v) (627, 625).

[0108] Figure 7 is a drawing for explaining an example of measuring the distance between second coordinates for Voronoi fragmentation.

[0109] Referring to FIG. 7, the first a coordinate value (x1, y1) in the first coordinate plane (e.g., the coordinate plane (610) of FIG. 6a) corresponding to the first sensing measurement point (710) can be projected to the first b coordinate value (u1, v1) in the second coordinate plane (e.g., the coordinate plane (620) of FIG. 6b). The second a coordinate value (x2, y2) in the first coordinate plane corresponding to the second sensing measurement point (720) can be projected to the second b coordinate value (u2, v2) in the second coordinate plane. For example, by substituting the first a coordinate value (x1, y1) and the second a coordinate value (x2, y2) in the first coordinate plane into the second coordinate plane, the first b coordinate value (u1, v1) and the second b coordinate value (u2, v2) can be obtained as two vector values ​​(u, v).

[0110] For example, the distance between the first coordinates (d[(x1, y1), (x2, y2)]) can be calculated by the distance between the firstb coordinate values ​​(u1, v1) and the secondb coordinate values ​​(u2, v2) corresponding to the second coordinates. For example, the distance between the firstb coordinate values ​​(u1, v1) and the secondb coordinate values ​​(u2, v2) can be calculated by the definition in <Mathematical Formula 6> above.

[0111] FIG. 8a, FIG. 8b, FIG. 8c, FIG. 8d, and FIG. 8e are drawings for explaining changes in cell shape according to the curvature characteristics of a projection surface (e.g., the projection surface (110) of FIG. 1 or FIG. 2).

[0112] Referring to FIGS. 8a, 8b, 8c, 8d, and 8e, the eigenvalues ​​for the slope reflecting the curvature characteristic can affect determining the size of the coordinate axes (u, v) of the corresponding coordinate plane. For example, when the ratio of the eigenvalues ​​on the coordinate axis u increases, the length of the coordinate axis u becomes shorter, and when the ratio of the eigenvalues ​​on the coordinate axis u decreases, the length of the coordinate axis u becomes longer. For example, when the ratio of the eigenvalues ​​on the coordinate axis v increases, the length of the coordinate axis v becomes shorter, and when the ratio of the eigenvalues ​​on the coordinate axis v decreases, the length of the coordinate axis v becomes longer. In one example, the ratio of the eigenvalues ​​increases as the slope of the slope increases, and the ratio decreases as the slope of the slope decreases. Therefore, the cell according to the result of Voronoi fragmentation may have a shape that narrows in the direction in which the slope exists on the projection surface (110) due to the change in the ratio of the eigenvalues ​​according to the curvature characteristic.

[0113] For example, if the projection surface (110) is a plane without a slope, the sizes of the coordinate axes (u, v) may be the same (815) and may be uniform in the shape of the cells (813) corresponding to the sensing measurement points (811) (see FIG. 8a). For example, it can be confirmed that the ratio of the eigenvalues ​​on the coordinate axis u and / or the ratio of the eigenvalues ​​on the coordinate axis v changes in response to a change in the inclination and / or direction of the slope on the projection surface (110). Accordingly, the image output by the image projection device (100) can be corrected to reflect the curvature characteristics of the projection surface (110), so that the viewer can view an undistorted image even on a curved projection surface (110).

[0114] FIG. 9a is a drawing showing an undistorted input image (910) input to an image projection device (image projection device (100) of FIG. 2), FIG. 9b is a drawing showing an image (920) displayed when the image projection device (100) projects the input image (910) onto a curved projection surface (110) without correction, FIG. 9c is a drawing showing an image (930) displayed when the image projection device (100) projects the input image (910) onto the projection surface (110) after correcting it without considering the curvature characteristics of the projection surface (110), and FIG. 9d is a drawing showing an image (940) displayed when the image projection device (100) projects the input image (910) onto the projection surface (110) after correcting it in consideration of the curvature characteristics of the projection surface (110).

[0115] In the display images (920, 930, 940) illustrated in FIGS. 9b to 9c, when no correction is performed, the vertical axis (921, 931, 941) and / or the horizontal axis (923, 933, 943) show the most distortion. In addition, when correction is performed in consideration of the curvature characteristics of the projection surface (110) in the display images (920, 930, 940), the vertical axis (921, 931, 941) and / or the horizontal axis (923, 933, 943) may have relatively less distortion.

[0116] FIG. 10 is a block diagram of an electronic device (1001) (e.g., an image projection device (100) of FIG. 2) within a network environment (1000) according to one or more embodiments.

[0117] Referring to FIG. 10, in a network environment (1000), an electronic device (1001) may communicate with an electronic device (1003) via a first network (1098) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (1005) or a server (1007) via a second network (1096) (e.g., a long-range wireless communication network). In one example, the electronic device (1001) may communicate with the electronic device (1005) via the server (1007). In one example, the electronic device (1001) may include a processor (1010), a memory (1020), an audio module (1040), an image module (1050), a sensor module (1060), a power management module (1070), an input module (1082), an interface (1084), a connection terminal (1086), or a communication module (1090). In some examples, the electronic device (1001) may omit at least one of these components (e.g., the input module (1082)), or may have one or more other components added. In some examples, some of these components may be integrated into a single component.

[0118] The processor (1010) may control at least one other component (e.g., a hardware or software component) of the electronic device (1001) connected to the processor (1010) by executing, for example, software (e.g., a program (1030)), and may perform various data processing or calculations. In one example, as at least a part of the data processing or calculation, the processor (1010) may store a command or data received from another component (e.g., a sensor module (1060) or a communication module (1090)) in a volatile memory (1022), process the command or data stored in the volatile memory (1022), and store the resulting data in a non-volatile memory (1024). In one example, the processor (1010) may include a main processor (1012) (e.g., a CPU or an AP) or a secondary processor (1014) (e.g., a GPU, an NPU, an ISP, a sensor hub processor, or a CP) that can operate independently or together therewith. For example, if the electronic device (1001) includes a main processor (1012) and a secondary processor (1014), the secondary processor (1014) may be configured to use less power than the main processor (1012) or to be specialized for a given function. The secondary processor (1014) may be implemented separately from the main processor (1012) or as a part thereof.

[0119] The auxiliary processor (1014) may control at least a portion of functions or states associated with at least one component (e.g., a sensor module (1060) or a communication module (1090)) of the electronic device (1001), for example, on behalf of the main processor (1012) while the main processor (1012) is in an inactive (e.g., sleep) state, or together with the main processor (1012) while the main processor (1012) is in an active (e.g., application execution) state. In one example, the auxiliary processor (1014) (e.g., an ISP or a CP) may be implemented as a part of another functionally related component (e.g., a communication module (1090)). In one example, the auxiliary processor (1014) (e.g., an NPU) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (1001) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (1007)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0120] The memory (1020) can store various data used by at least one component (e.g., the processor (1010) or the sensor module (1060)) of the electronic device (1001). The data can include, for example, software (e.g., the program (1030)) and input data or output data for commands related thereto. The memory (1020) can include volatile memory (1022) or non-volatile memory (1024).

[0121] The program (1030) may be stored as software in memory (1020) and may include, for example, an operating system (1036), middleware (1034), or an application (1032).

[0122] The input module (1082) can receive commands or data to be used in a component of the electronic device (1001) (e.g., a processor (1010)) from an external source (e.g., a user) of the electronic device (1001). The input module (1082) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0123] The audio module (1040) may include an audio processing module (1042) or an audio output module (1044). The audio output module (1044) may output an audio signal to the outside of the electronic device (1001). The audio output module (1044) may include, for example, a speaker. The speaker may be used for general purposes such as multimedia playback or recording playback. The audio processing module (1042) may convert sound into an electrical signal, or vice versa. In one example, the audio module (1040) may acquire sound through the input module (1082), output sound through the audio output module (1044), or an external electronic device (e.g., electronic device (1003)) (e.g., a speaker or headphones) directly or wirelessly connected to the electronic device (1001).

[0124] The image module (1050) may include an image processing module (1052) or an image output module (1054). The image processing module (1052) may output a video signal to the outside of the electronic device (1001). The image output module (1054) may include, for example, a display and / or an optical projector. The optical projector may convert an electrical video signal into an optical signal and output it. The image processing module (1052) may convert an image into an electrical signal or, conversely, convert an electrical signal into an image. According to an example, the image module (1050) may acquire an image through the input module (1082), or output an image through the aspect output module (1054), or an external electronic device (e.g., the electronic device (1003)) directly or wirelessly connected to the electronic device (1001). The image module (1050) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device.

[0125] The sensor module (1060) can detect the operating status (e.g., power or temperature) of the electronic device (1001) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to an example, the sensor module (1060) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0126] The interface (1084) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (1001) to an external electronic device (e.g., the electronic device (1003)). The interface (1084) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface (e.g., Bixby).

[0127] The connection terminal (1086) may include a connector through which the electronic device (1001) may be physically connected to an external electronic device (e.g., the electronic device (1003)). In one example, the connection terminal (1086) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0128] The power management module (1070) can manage power supplied to the electronic device (1001). According to one embodiment, the power management module (1070) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0129] The communication module (1090) may support the establishment of a direct (e.g., wired or wireless) communication channel or a wireless communication channel between the electronic device (1001) and an external electronic device (e.g., electronic device (1003), electronic device (1005), or server (1007)), and the performance of communication through the established communication channel. The communication module (1090) may operate independently from the processor (1010) (e.g., AP) and may include one or more CPs that support direct (e.g., wired) communication or wireless communication. In one example, the communication module (1090) may include a wireless communication module (1092) (e.g., a cellular communication module, a short-range wireless communication module, or a GNSS communication module) or a wired communication module (1094) (e.g., a LAN communication module or a power line communication module). Any of these communication modules may communicate with an external electronic device (1005) via a first network (1098) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (1096) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1092) may use subscriber information (e.g., an international mobile subscriber identity (IMSI)) to verify or authenticate the electronic device (1001) within a communication network such as the first network (1098) or the second network (1096).

[0130] The wireless communication module (1092) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimizing terminal power and connecting multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (1092) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (1092) may support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (1092) may support various requirements specified in the electronic device (1001), an external electronic device (e.g., the electronic device (1005)), or a network system (e.g., the second network (1096)). According to one embodiment, the wireless communication module (1092) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0131] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0132] In one example, commands or data may be transmitted or received between the electronic device (1001) and an external electronic device (1005) via a server (1007) connected to a second network (1096). Each of the external electronic devices (1003 or 1005) may be the same or a different type of device as the electronic device (1001). In one example, all or part of the operations executed in the electronic device (1001) may be executed in one or more of the external electronic devices (1003, 1005, or 1007). For example, when the electronic device (1001) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (1001) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (1001). The electronic device (1001) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (1001) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one example, the external electronic device (1005) may include an Internet of Things (IoT) device. The server (1007) may be an intelligent server utilizing machine learning and / or a neural network.In one example, an external electronic device (1005) or server (1007) may be included within the second network (1096). The electronic device (1001) may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0133] According to an example, an image projection apparatus (100) may include at least one sensor (ToF) (420). The image projection apparatus (100) may include at least one memory (430) including a non-volatile recording medium that stores instructions. The image projection apparatus (100) may include an image projector (440) configured to project an optical signal (403) corresponding to an output image onto a projection surface (110). The image projection apparatus (100) may include at least one processor (410) including a processing circuit. When the instructions are individually or collectively executed by the at least one processor (410), they may cause the image projection apparatus (100) to perform at least one operation. The at least one operation may include an operation of detecting first position data (619) in a coordinate space corresponding to a plurality of sensing measurement points (641) distributed on the projection surface (110) by the at least one sensor (420). The at least one operation may include an operation of determining second position data in a coordinate plane reflecting a curvature characteristic of the projection surface (110) based on the first position data (619). The at least one operation may include an operation of performing area-weighted interpolation using the second position data to obtain third position data corresponding to a plurality of pixel projection points (643) on which the light signal (403) is to be projected in a projection area (120) of the projection surface (110).

[0134] According to an example, when the instructions are individually or collectively executed by at least one processor, the image projection device (100) may be caused to perform an operation of obtaining a local gradient of the plurality of sensing measurement points (641) in a first coordinate plane by the first position data (619).

[0135] For example, when the instructions are individually or collectively executed by at least one processor, the image projection device (100) may be caused to perform an operation of determining a coordinate axis (u, v) of a second coordinate plane for determining the second position data by reflecting the distribution of the acquired inclination.

[0136] For example, when the instructions are individually or collectively executed by at least one processor, they may cause the image projection device (100) to perform an operation of determining an eigenvector regarding direction and an eigenvalue regarding slope based on the distribution of the acquired slope.

[0137] For example, when the instructions are individually or collectively executed by at least one processor, they may cause the image projection device (100) to perform an operation of determining the coordinate axes (u, v) of the second coordinate plane based on the eigenvector and the eigenvalue.

[0138] For example, when the instructions are individually or collectively executed by at least one processor, they may cause the image projection device (100) to perform an operation of determining the size of the coordinate axes (u, v) in the second coordinate plane by the ratio of the eigenvalues.

[0139] According to an example, when the instructions are individually or collectively executed by at least one processor, the image projection device (100) may be caused to perform an operation of identifying a planar rectangular coordinate system (u, v) of the plurality of sensing measurement points (641) in the second coordinate plane based on the second position data.

[0140] For example, when the instructions are individually or collectively executed by at least one processor, they may cause the image projection device (100) to perform an operation of determining a distance between the plurality of sensing measurement points (641) in the second coordinate plane using the identified planar orthogonal coordinate system (u, v).

[0141] For example, when the instructions are individually or collectively executed by at least one processor, they may cause the image projection device (100) to perform an operation of performing Voronoi tessellation based on the distances between the determined plurality of sensing measurement points (641).

[0142] For example, when the instructions are individually or collectively executed by at least one processor, the image projection device (100) may be caused to perform an operation of performing scattered data interpolation on the first cells obtained as a result of the Voronoi fragmentation to obtain second cells corresponding to the plurality of pixel projection points (643).

[0143] In one example, the shape of the first cells and / or the second cells may be narrowed in the direction in which the slope exists on the projection surface (110) according to the curvature characteristic.

[0144] As an example, the curvature characteristics of the projection surface (110) may include information about the directionality of a wave propagating in a single direction on the projection surface (110).

[0145] According to an example, an operating method of an image projection apparatus (100) may include an operation of detecting first position data (619) in a coordinate space corresponding to a plurality of sensing measurement points (641) distributed on a projection surface (110) on which an optical signal (403) corresponding to an output image is to be projected by at least one sensor (420). The operating method may include an operation of determining second position data in a coordinate plane reflecting a curvature characteristic of the projection surface (110) based on the first position data (619). The operating method may include an operation of performing area-weighted interpolation using the second position data to obtain third position data corresponding to a plurality of pixel projection points (643) on which the optical signal (403) is to be projected in a projection area (120) of the projection surface (110).

[0146] According to an example, the operation of determining the second position data may include an operation of obtaining a local gradient of the plurality of sensing measurement points (641) in a first coordinate plane by the first position data (619).

[0147] According to an example, the operation of determining the second position data may include an operation of determining the coordinate axes (u, v) of a second coordinate plane for determining the second position data by reflecting the distribution of the acquired slope.

[0148] According to an example, the operation of determining the coordinate axes (u, v) of the second coordinate plane may include an operation of determining an eigenvector regarding direction and an eigenvalue regarding slope based on the distribution of the acquired slope.

[0149] According to an example, the operation of determining the coordinate axis (u,v) of the second coordinate plane may include an operation of obtaining the coordinate axis (u,v) of the second coordinate plane based on the eigenvector and the eigenvalue.

[0150] According to an example, the operation of obtaining the coordinate axis (u,v) of the second coordinate plane may include an operation of determining the size of the coordinate axis (u,v) in the second coordinate plane by the ratio of the eigenvalues.

[0151] According to an example, the operation of obtaining the third position data may include an operation of identifying a planar orthogonal coordinate system (u, v) of the plurality of sensing measurement points (641) in the second coordinate plane based on the second position data.

[0152] According to an example, the operation of obtaining the third position data may include an operation of determining a distance between the plurality of sensing measurement points (641) in the second coordinate plane using the identified planar orthogonal coordinate system (u, v).

[0153] According to an example, the operation of obtaining the third position data may include an operation of performing Voronoi tessellation based on the distance between the determined plurality of sensing measurement points (641).

[0154] According to an example, the operation of obtaining the third position data may include an operation of performing scattered data interpolation on the first cells obtained as a result of the Voronoi fragmentation to obtain second cells corresponding to the plurality of pixel projection points (643).

[0155] In one example, the shape of the first cells and / or the second cells may be narrowed in the direction in which the slope exists on the projection surface (110) according to the curvature characteristic.

[0156] As an example, the curvature characteristics of the projection surface (110) may include information about the directionality of a wave propagating in a single direction on the projection surface (110).

[0157] According to one example, a recording medium storing computer-readable instructions may be provided. The instructions, when executed by at least a part of at least one processor included in an image projection apparatus (100), may cause the image projection apparatus (100) to perform at least one operation. The at least one operation may include an operation of detecting first position data (619) in a coordinate space corresponding to a plurality of sensing measurement points (641) distributed on a projection surface (110) on which an optical signal (403) corresponding to an output image is to be projected by at least one sensor (420). The at least one operation may include an operation of determining second position data in a coordinate plane reflecting a curvature characteristic of the projection surface (110) based on the first position data (619). The at least one operation may include performing area-weighted interpolation using the second position data to obtain third position data corresponding to a plurality of pixel projection points (643) on which the light signal (403) is to be projected in the projection area (120) of the projection surface (110).

[0158] According to an example, the operation of determining the second position data may include an operation of obtaining a local gradient of the plurality of sensing measurement points (641) in a first coordinate plane by the first position data (619).

[0159] According to an example, the operation of determining the second position data may include an operation of determining an eigenvector regarding direction and an eigenvalue regarding slope based on the distribution of the acquired slope.

[0160] According to an example, the operation of determining the second position data may include an operation of obtaining the coordinate axes (u, v) of a second coordinate plane for determining the second position data based on the eigenvector and the eigenvalue.

[0161] For example, the size of the coordinate axes (u,v) can be determined by the ratio of the eigenvalues.

[0162] According to an example, the operation of obtaining the third position data may include an operation of identifying a planar orthogonal coordinate system (u, v) of the plurality of sensing measurement points (641) in the second coordinate plane based on the second position data.

[0163] According to an example, the operation of obtaining the third position data may include an operation of determining a distance between the plurality of sensing measurement points (641) in the second coordinate plane using the identified planar orthogonal coordinate system (u, v).

[0164] According to an example, the operation of obtaining the third position data may include an operation of performing Voronoi tessellation based on the distance between the determined plurality of sensing measurement points (641).

[0165] According to an example, the operation of obtaining the third position data may include an operation of performing scattered data interpolation on the first cells obtained as a result of the Voronoi fragmentation to obtain second cells corresponding to the plurality of pixel projection points (643).

[0166] In one example, the shape of the first cells and / or the second cells may be narrowed in the direction in which the slope exists in the projection surface (110) according to the curvature characteristic.

[0167] As an example, the curvature characteristics of the projection surface (110) may include information about the directionality of a wave propagating in a single direction on the projection surface (110).

[0168] Electronic devices according to various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, display devices (e.g., TVs, monitors, optical projectors), portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the aforementioned devices.

[0169] The various embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" can each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish the corresponding components from other corresponding components and do not limit the corresponding components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as being "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., via wired or wirelessly), wirelessly, or through a third component.

[0170] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0171] Various embodiments of the present document may be implemented as software (e.g., a program) including one or more commands stored in a storage medium (e.g., a memory (430)) readable by a machine (e.g., an image projection device (100)). For example, a processor (e.g., a processor (410)) of the machine (e.g., an image projection device (100)) may call at least one command among the one or more commands stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one command called. The one or more commands may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0172] According to one embodiment, a method according to one or more embodiments disclosed in the present document may be provided as a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0173] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In the image projection apparatus (100), At least one sensor (ToF) (420); At least one memory (430) comprising a non-volatile recording medium for storing instructions; An image projector (440) configured to project an optical signal (403) corresponding to an output image onto a projection surface (110); and At least one processor (410) operatively connected to at least one sensor (420), at least one memory (430) and an image projector (440), and including a processing circuit; When the above instructions are individually or collectively executed by at least one processor (410), they cause the image projection device (100) to perform at least one operation, At least one of the above actions: An operation of detecting first position data (619) in a coordinate space corresponding to a plurality of sensing measurement points (641) on the projection surface (110) using at least one sensor (420); An operation of determining second position data on a coordinate plane reflecting the curvature characteristics of the projection surface (110) based on the first position data (619); and An operation of acquiring third position data corresponding to multiple pixel projection points (643) Includes, Here, the optical signal (403) is projected onto a plurality of pixel projection points of the projection area (120) of the projection surface (110) by performing area-weighted interpolation of the second position data, in an image projection device (100).

2. In paragraph 1, When the above instructions are individually or collectively executed by at least one processor, the image projection device (100) causes: An operation of obtaining the local gradient of the plurality of sensing measurement points (641) in the first coordinate plane by the first position data (619); and An operation of determining the coordinate axes (u, v) of the second coordinate plane for determining the second position data by reflecting the distribution of the acquired slope. An image projection device (100) that causes the image to be projected.

3. In paragraph 2, When the above instructions are individually or collectively executed by at least one processor, the image projection device (100) causes: An operation of determining an eigenvector for direction and an eigenvalue for slope based on the distribution of the obtained slope; and An operation of determining the coordinate axis (u, v) of the second coordinate plane based on the eigenvector and the eigenvalue. An image projection device (100) that causes the image to be projected.

4. In paragraph 3, When the above instructions are individually or collectively executed by at least one processor, the image projection device (100) causes: An operation of determining the size of the coordinate axis (u, v) in the second coordinate plane by the ratio of the above eigenvalues. An image projection device (100) that causes the image to be projected.

5. In any one of paragraphs 1 to 4, When the above instructions are individually or collectively executed by at least one processor, the image projection device (100) causes: An operation of identifying a planar orthogonal coordinate system (u, v) of the plurality of sensing measurement points (641) in the second coordinate plane based on the second position data; and An operation of determining the distance between the plurality of sensing measurement points (641) in the second coordinate plane using the above-identified planar orthogonal coordinate system (u, v). An image projection device (100) that causes the image to be projected.

6. In paragraph 5, When the above instructions are individually or collectively executed by at least one processor, the image projection device (100) causes: An operation of performing Voronoi tessellation based on the distance between the above-determined plurality of sensing measurement points (641); and An operation of performing scattered data interpolation on the first cells obtained as a result of the above Voronoi fragmentation to obtain second cells corresponding to the plurality of pixel projection points (643). An image projection device (100) that causes the image to be projected.

7. In the operating method of the image projection apparatus (100), An operation of detecting first position data (619) in a coordinate space corresponding to a plurality of sensing measurement points (641) distributed on a projection surface (110) on which an optical signal (403) corresponding to an output image is projected by at least one sensor (420) of the image projection device (100); An operation of determining second position data on a coordinate plane reflecting the curvature characteristics of the projection surface (110) based on the first position data (619); and An operation of acquiring third position data corresponding to multiple pixel projection points (643) Includes, Here, the optical signal (403) is projected onto a plurality of pixel projection points of the projection area (120) of the projection surface (110) by performing area-weighted interpolation of the second position data.

8. In paragraph 7, The operation of determining the above second location data is: An operation of obtaining the local gradient of the plurality of sensing measurement points (641) in the first coordinate plane by the first position data (619); and An operation to determine the coordinate axes (u, v) of the second coordinate plane by reflecting the distribution of the obtained slopes. A method of operation, comprising:

9. In paragraph 8, The operation of determining the coordinate axes (u, v) of the second coordinate plane is as follows: An operation of determining an eigenvector for direction and an eigenvalue for slope based on the distribution of the obtained slope; and An operation of obtaining the coordinate axes (u, v) of the second coordinate plane based on the eigenvector and the eigenvalue. A method of operation, comprising:

10. In paragraph 9, The operation of obtaining the coordinate axis (u, v) of the second coordinate plane is as follows: An operation of determining the size of the coordinate axis (u, v) in the second coordinate plane by the ratio of the above eigenvalues. A method of operation, comprising:

11. In any one of paragraphs 7 to 10, The operation of acquiring the above third location data is as follows: An operation of identifying a planar orthogonal coordinate system (u, v) of the plurality of sensing measurement points (641) in the second coordinate plane based on the second position data; An operation of determining the distance between the plurality of sensing measurement points (641) in the second coordinate plane using the identified planar orthogonal coordinate system (u, v); An operation of performing Voronoi tessellation based on the distance between the above-determined plurality of sensing measurement points (641); and An operation of performing scattered data interpolation on the first cells obtained as a result of the above Voronoi fragmentation to obtain second cells corresponding to the plurality of pixel projection points (643). A method of operation, comprising:

12. In a non-transitory storage medium storing instructions readable by at least one computer, When the above instructions are executed by at least a part of at least one processor (410) of an image projection apparatus (100), the image projection apparatus (100) causes the image projection apparatus (100) to perform at least one operation, At least one of the above actions: An operation of detecting first position data (619) in a coordinate space corresponding to a plurality of sensing measurement points (641) distributed on a projection surface (110) on which an optical signal (403) corresponding to an output image is projected by at least one sensor (420) of an image projection sensor; An operation of determining second position data on a coordinate plane reflecting the curvature characteristics of the projection surface (110) based on the first position data (619); and An operation of acquiring third position data corresponding to multiple pixel projection points (643) Includes, Here, the optical signal (403) is projected onto a plurality of pixel projection points of the projection area (120) of the projection surface (110) by performing area-weighted interpolation of the second position data, in a recording medium.

13. In paragraph 12, The operation of determining the above second location data is: An operation of obtaining a local gradient of the plurality of sensing measurement points (641) on a first coordinate plane based on the first position data (619); An operation of determining an eigenvector for direction and an eigenvalue for slope based on the distribution of the obtained slope; and It includes an operation of obtaining the coordinate axis (u, v) of the second coordinate plane based on the above eigenvector and the above eigenvalue, A recording medium wherein the size of the coordinate axes (u, v) is determined by the ratio of the eigenvalues.

14. In paragraph 12 or 13, The operation of acquiring the above third location data is as follows: An operation of identifying a planar orthogonal coordinate system (u, v) of the plurality of sensing measurement points (641) in the second coordinate plane based on the second position data; An operation of determining the distance between the plurality of sensing measurement points (641) in the second coordinate plane using the identified planar orthogonal coordinate system (u, v); An operation of performing Voronoi tessellation based on the distance between the above-determined plurality of sensing measurement points (641); and An operation of performing scattered data interpolation on the first cells obtained as a result of the above Voronoi fragmentation to obtain second cells corresponding to the plurality of pixel projection points (643). A recording medium including:

15. In any one of paragraphs 6, 11 or 14, The shape of the first cells or the second cells is narrowed in the direction in which the slope exists on the projection surface (110) based on the curvature characteristic, The curvature characteristics of the projection surface (110) include information about the directionality of a wave traveling in a single direction on the projection surface (110), and a recording medium for an image projection device (100) and an operating method.

Images