Electronic device for compensating for keystone correction generated external region and control method thereof

Abstract

An electronic device is disclosed. The electronic device includes: a projection unit for projecting an image of a specified resolution; a camera; and at least one processor including processing circuitry, connected to the projection unit and the camera, and controlling the electronic device, where the at least one processor may: perform keystone correction on the first image individually and / or collectively; controlling the projection unit such that an image having a specified resolution in which the keystone-corrected first image is included in one region and the second image is included in the remaining region is projected into a projection region; capturing the projection area and an area outside the projection area through the camera to obtain a captured image; adjusting a pixel value of the second image based on first pixel information corresponding to other areas in the captured image and second pixel information corresponding to an area outside the projection area; and controlling the projection unit such that an image having a specified resolution in which the keystone-corrected first image is included in one region and the second image having the adjusted pixel value is included in the remaining region is projected into the projection region.

Description

Technical Field

[0001] This disclosure relates to an electronic device and its control method, and for example to an electronic device and its control method for compensating for external regions caused by trapezoidal correction. Background Technology

[0002] With the development of electronic technology, electronic devices offering various functions are being developed. Specifically, in recent years, various types of projectors have become widespread.

[0003] A projector is a device that projects images and creates images by projecting light onto a projection area. Compared to other types of display devices, it has the advantage of being able to easily achieve large screens.

[0004] In recent years, projectors that offer keystone correction have become increasingly popular, improving user convenience. Summary of the Invention

[0005] Technical solution

[0006] An electronic device according to an example embodiment includes: a projector configured to project an image having a specified resolution; a camera; and at least one processor, including processing circuitry, connected to the projector and the camera, and individually and / or collectively configured to control the electronic device, wherein the at least one processor is individually and / or collectively configured to: perform keystone correction on a first image; control the projector to project an image of the specified resolution onto a projection area, wherein the first image with keystone correction is included in one area and a second image is included in the remaining area; acquire a captured image by the camera, the captured image including a photograph of the projection area and an outer area of ​​the projection area; adjust the pixel values ​​of a second image based on first pixel information corresponding to the remaining area of ​​the captured image and second pixel information corresponding to the outer area of ​​the projection area; and control the projector to project an image of the specified resolution onto the projection area, wherein the first image with keystone correction is included in one area and a second image having adjusted pixel values ​​is included in the remaining area.

[0007] The image can be an image of a specified color, and at least one processor can be configured individually and / or collectively to adjust the pixel values ​​of the second image such that the difference between the first pixel information and the second pixel information is less than a threshold.

[0008] At least one processor may be configured individually and / or collectively to: acquire captured images of a projected area and an outer area via a camera at specified time intervals; adjust pixel values ​​of a second image based on first pixel information corresponding to the remaining areas and second pixel information corresponding to the outer areas in each of a plurality of captured images acquired at specified time intervals; and stop acquiring captured images and adjusting pixel values ​​based on the difference between the first pixel information and the second pixel information being less than a threshold.

[0009] At least one processor may be configured individually or collectively to: identify a pattern corresponding to an outer region of a projected area in a captured image; and adjust pixel values ​​of a second image based on the identified pattern.

[0010] At least one processor may be configured individually or collectively to adjust the pixel values ​​of the second image based on at least one of the brightness or color of the keystone-corrected image.

[0011] The device may also include a sensor, and at least one processor may be configured individually and / or collectively to: acquire sensing information of ambient illuminance, including that of the electronic device, via the sensor; and adjust pixel values ​​of a second image based on the sensing information.

[0012] The device may also include sensors, and at least one processor may be configured individually and / or collectively to: acquire sensing information via the sensors, the sensing information including the angle of rotation of the electronic device relative to the direction of gravity and the angle of rotation of the electronic device relative to the projection area; and perform trapezoidal correction on the first image by changing the shape of the first image based on the sensing information.

[0013] The device may also include a user interface, and at least one processor may be configured individually and / or collectively to adjust the pixel values ​​of the second image based on the color corresponding to a user command received through the user interface.

[0014] The projection area can be the area onto which the light emitted from the projector is projected.

[0015] A method for controlling an electronic device according to an example embodiment includes: performing keystone correction on a first image; projecting an image having a specified resolution onto a projection area, wherein the keystone-corrected first image is included in one area and a second image is included in the remaining areas; obtaining a captured image including the projection area and an outer area of ​​the projection area; adjusting pixel values ​​of the second image based on first pixel information corresponding to the remaining areas of the captured image and second pixel information corresponding to the outer areas of the projection area; and projecting an image of the specified resolution onto the projection area, wherein the keystone-corrected first image is included in one area and a second image having adjusted pixel values ​​is included in the remaining areas.

[0016] The image can be an image of a specified color, and the adjustment can include: adjusting the pixel values ​​of the second image such that the difference between the first pixel information and the second pixel information is less than a threshold.

[0017] The method may include acquiring a captured image comprising a projected region and an outer region at specified time intervals, and adjusting the pixel values ​​of a second image based on first pixel information corresponding to the remaining regions and second pixel information corresponding to the outer regions in each of a plurality of captured images acquired at specified time intervals, and the method may further include stopping the acquisition of the captured image and adjusting the pixel values ​​based on the difference between the first pixel information and the second pixel information being less than a threshold.

[0018] The method may further include identifying a pattern corresponding to the outer region of the projected area in the captured image, and the adjustment may include adjusting the pixel values ​​of the second image based on the identified pattern.

[0019] The adjustment may include adjusting the pixel values ​​of the second image based on at least one of the brightness or color of the keystone-corrected image.

[0020] The method may further include obtaining sensing information including the ambient illuminance of the electronic device, and the adjustment may include adjusting the pixel values ​​of the second image based on the sensing information.

[0021] The method may further include: obtaining sensing information, including the angle of rotation of the electronic device relative to the direction of gravity and the angle of rotation of the electronic device relative to the projection area; and performing trapezoidal correction on the first image by changing the shape of the first image based on the sensing information.

[0022] Adjustments may include adjusting the pixel values ​​of the second image based on the color corresponding to the user command.

[0023] The projection area can be the area onto which the light emitted from the projector is projected.

[0024] According to an example embodiment, a non-transitory computer-readable recording medium may have a program stored thereon that, when executed individually and / or jointly by at least one processor of an electronic device, causes the electronic device to perform operations including: performing keystone correction on a first image; projecting an image having a specified resolution onto a projection area, wherein the keystone-corrected first image is included in one area and a second image is included in the remaining area; obtaining a captured image including the projection area and an outer area of ​​the projection area; adjusting pixel values ​​of the second image based on first pixel information corresponding to the remaining area of ​​the captured image and second pixel information corresponding to the outer area of ​​the projection area; and projecting an image of the specified resolution onto the projection area, wherein the keystone-corrected first image is included in one area and a second image having adjusted pixel values ​​is included in the remaining area. Attached Figure Description

[0025] The above and other aspects, features, and advantages of certain embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0026] Figure 1 This is a diagram illustrating example trapezoidal correction of an image, intended to aid in understanding this disclosure;

[0027] Figure 2 This is a block diagram illustrating an example configuration of an electronic device according to various embodiments;

[0028] Figure 3 This is a block diagram illustrating an example configuration of an electronic device according to various embodiments;

[0029] Figure 4 This is a diagram illustrating example trapezoidal correction according to various embodiments;

[0030] Figure 5 This is a diagram illustrating example external regions produced by trapezoidal correction according to various embodiments;

[0031] Figure 6 This is a flowchart illustrating example methods for trapezoidal correction and external region compensation according to various embodiments;

[0032] Figure 7 The diagram illustrates example methods for performing trapezoidal correction and external region compensation according to various embodiments; and

[0033] Figure 8 This is a flowchart illustrating an example method for controlling an electronic device according to various embodiments. Detailed Implementation

[0034] Various exemplary embodiments of this disclosure can be modified in different ways. Therefore, various exemplary embodiments are shown in the accompanying drawings and described in detail in the specific implementation. However, it should be understood that this disclosure is not limited to the specific embodiments, but includes all modifications, equivalents, and substitutions without departing from the scope and spirit of this disclosure. Furthermore, if well-known functions or structures would obscure this disclosure with unnecessarily detailed descriptions, they may not be described in detail.

[0035] This disclosure provides an electronic device and its control method for minimizing and / or reducing the user's sense of heterogeneity by compensating for the external area generated by trapezoidal correction.

[0036] The present disclosure will be described in more detail below with reference to the accompanying drawings.

[0037] In consideration of the functions described herein, commonly used terms are selected as the terms used in the embodiments of this disclosure. However, these terms may be modified based on the intent of those skilled in the art, judicial precedent, the emergence of new technologies, etc. Furthermore, in certain circumstances, terms may be chosen arbitrarily. In such cases, the meaning of such terms is detailed in the corresponding description of this disclosure. Therefore, the terms used in the embodiments of this disclosure are defined based on their meaning and throughout the entirety of this disclosure, rather than their simple names.

[0038] In this disclosure, the expressions “have,” “may have,” “include,” “may include,” etc., indicate the presence of a corresponding feature (e.g., a value, function, operation, or component such as a part), without excluding the presence of additional features.

[0039] The statement “at least one of A and / or B” should be understood as indicating any one of “A”, “B”, or “both A and B”.

[0040] The expressions “first,” “second,” “first,” “second,” etc., used in this disclosure may refer to various components regardless of their order and / or importance, and these expressions are used only to distinguish one component from another, without limiting the corresponding component.

[0041] Unless the context clearly indicates otherwise, the singular form of a term may include its plural form. It should be understood that the terms "comprising," "formed by," etc., as used in this application specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof mentioned in this disclosure, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0042] In this disclosure, the term "user" may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0043] In the following, various exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings.

[0044] Figure 1 This is a diagram illustrating the trapezoidal correction of an image, used to aid in understanding this disclosure.

[0045] When a projector is placed upright on a flat surface with its orientation facing the screen, a rectangular image can be displayed on the screen. On the other hand, when the projector is not placed upright on a flat surface with its orientation facing the screen, an image that is distorted vertically, horizontally, or rotated may be displayed on the screen. This distortion is called the trapezoidal effect, and the function to eliminate this distortion is called trapezoidal correction.

[0046] For example, when a projector is not placed upright on a flat surface with its orientation towards the screen, a projection area may be formed, such as... Figure 1 Region 101. In other words, the projector can project an image onto region 101. The projection area can be the area onto which light emitted from the projector is projected.

[0047] Projectors can use keystone correction to project images onto a portion of the projection area, for example... Figure 1 Region 102 is shown in the image. However, even in this case, light emitted from the projector can still be projected onto the remaining areas of the projection area other than region 102. For example, the projector can project the remaining areas of the projection area other than region 102 as black. However, since a projector is a device that displays colors on a screen using a light source, even projecting black may provide a sense of incongruity to the user. In other words, the user may perceive a sense of incongruity due to the difference between the remaining areas of the projection area other than region 102 and the outer areas of the projection area.

[0048] Figure 2 This is a block diagram illustrating an example configuration of an electronic device 100 according to various embodiments.

[0049] The electronic device 100 can be a device that projects content by changing the position of the electronic device 100. For example, the electronic device 100 can be a projector that can move on its own.

[0050] The electronic device 100 can be a device that provides an image to a screen. For example, the electronic device 100 can be a projector with keystone correction.

[0051] refer to Figure 1 The electronic device 100 includes a projector 110, a camera 120, and a processor (e.g., including processing circuitry) 130.

[0052] Projector 110 can project images with a preset (e.g., specified) resolution onto a projection surface. For example, projector 110 can use a light source (e.g., a lamp or LED) to project images or videos that include at least one of content received from a source device or pre-stored content onto the projection area.

[0053] Camera 120 is configured to capture still images or moving images. Camera 120 can capture still images at a specific point in time, or it can capture still images continuously.

[0054] Camera 120 can photograph the area in front of electronic device 100 to capture the area with projected content. Processor 130 can perform image processing based on the image captured by camera 120.

[0055] Camera 120 may include, for example, a lens, shutter, aperture, solid-state imaging device, analog front-end (AFE), and timing generator (TG). The shutter controls the time it takes for light reflected from the subject to enter camera 120, and the aperture controls the amount of light entering the lens by mechanically increasing or decreasing the size of the opening. When the light reflected from the subject accumulates into photocharge, the solid-state imaging device outputs an image of the photocharge as an electrical signal. The TG outputs a timing signal for reading out pixel data from the solid-state imaging device, and the AFE samples and digitizes the electrical signal output from the solid-state imaging device.

[0056] The processor 130 may include various processing circuits and control the overall operation of the electronic device 100. For example, the processor 130 may be connected to each component of the electronic device 100 to provide overall operation of the electronic device 100. For example, the processor 130 may be connected to configurations such as a projector 110, a camera 120, a sensor (not shown), a user interface (not shown) to control the operation of the electronic device 100.

[0057] One or more processors 130 may include one or more of a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a majority integrated core (MIC), a neural processing unit (NPU), a hardware accelerator, or a machine learning accelerator. One or more processors 130 may control one or any combination of other components of the electronic device 100 and may perform communication-related operations or data processing. One or more processors 130 may execute one or more programs or instructions stored in memory. For example, one or more processors 130 may perform a method according to an embodiment by executing one or more instructions stored in memory.

[0058] When the method according to the embodiments includes multiple operations, the multiple operations can be executed by one processor or multiple processors. For example, when the first operation, the second operation, and the third operation are performed by the method according to the embodiments, all of the first operation, the second operation, and the third operation can be executed by the first processor, or the first operation and the second operation can be executed by the first processor (e.g., a general-purpose processor), while the third operation can be executed by the second processor (e.g., an artificial intelligence-specific processor).

[0059] One or more processors 130 may be implemented as a single-core processor including a single core, or as one or more multi-core processors including multiple cores (e.g., homogeneous multi-core or heterogeneous multi-core). When one or more processors 130 are implemented as multi-core processors, each of the multiple cores included in the multi-core processor may include processor internal memory, such as cache memory and on-chip memory, and a common cache shared by the multiple cores may be included in the multi-core processor. Furthermore, each (or some of the multiple cores) included in the multi-core processor may independently read and execute program instructions to implement the method according to the embodiment, or all (or some of the multiple cores) may be coupled to read and execute program instructions to implement the method according to the embodiment.

[0060] When the method according to the embodiment includes multiple operations, the multiple operations can be executed by one core of a multi-core processor, or they can be executed by multiple cores. For example, when the first operation, the second operation, and the third operation are performed by the method according to the embodiment, all of the first operation, the second operation, and the third operation can be executed by the first core of the multi-core processor, or the first operation and the second operation can be executed by the first core of the multi-core processor, and the third operation can be executed by the second core of the multi-core processor.

[0061] In various embodiments of this disclosure, one or more processors 130 may refer, for example, to a system-on-a-chip (SoC), a single-core processor, a multi-core processor, or a core included in a single-core or multi-core processor that integrates one or more processors and other electronic components. Here, a core may be implemented as a CPU, GPU, APU, MIC, NPU, hardware accelerator, or machine learning accelerator, etc., but is not limited to the various embodiments of this disclosure. However, for ease of description, the term "processor 130" will be used below to describe the operation of the electronic device 100. In other words, processor 130 may include various processing circuitry and / or multiple processors. For example, as used herein (including the claims), the term "processor" may include various processing circuitry, including at least one processor, wherein one or more of the at least one processor may be configured individually and / or in a distributed manner to perform the various functions described herein. As used herein, when "processor," "at least one processor," and "one or more processors" are described as being configured to perform multiple functions, these terms cover, for example, but not limited to, a situation where one processor performs some of the functions while another processor performs other functions, and also cover a situation where a single processor can perform all of the functions. Additionally, at least one processor may include, for example, a combination of processors that perform various described / disclosed functions in a distributed manner. At least one processor may execute program instructions to implement or perform various functions.

[0062] The processor 130 can perform keystone correction on the first image and control the projector 110 to project an image with a preset resolution onto a projection area, in which the keystone-corrected first image is included in one area and the second image is included in the remaining area. Here, the projection area can be the area to which light emitted from the projector 110 is projected.

[0063] For example, processor 130 can perform keystone correction on a first image with a resolution of 1920×1080. In this case, the resolution of the keystone-corrected first image may be less than 1920×1080. When projector 110 projects an image with a resolution of 1920×1080, since an image with a resolution of 1920×1080 is required, processor 130 can control projector 110 to obtain an image with a resolution of 1920×1080, wherein the keystone-corrected first image is included in one area and the second image is included in the remaining area, and project the obtained image onto the projection area. The second image may be a black image. However, this disclosure is not limited to this, and the second image may not be black, and may be an image with a preset color set during manufacturing. In addition, the first image may be an image scaled to a resolution that projector 110 can output. For example, if the resolution that the projector 110 can output is 1920×1080, and the original image has a resolution of 3840×2160, then the processor 130 can down-convert the original image to obtain a first image with a resolution of 1920×1080. Therefore, when keystone correction is performed on the first image, the resolution of the keystone-corrected first image can become smaller than the resolution that the projector 110 can output. Here, the down-conversion can be performed frame by frame after the original image is decoded.

[0064] When an image in which a first image with trapezoidal correction is included in one region and a second image is included in the remaining region is projected onto a projection region, the first region corresponding to the first image and the second region corresponding to the second image in the projection region can be distinguished. However, even if the projector 110 projects a black image onto the second region, the light of the projector 110 is not blocked, but rather light representing black is projected onto the second region. Therefore, the color of the second region can be different from the color of the outer region of the projection region.

[0065] Because users may perceive a sense of heterogeneity due to color differences, in order to reduce this perception, the processor 130 can capture an image by taking pictures of the projection area and the area outside the projection area using the camera 120. Based on the first pixel information corresponding to the remaining area in the captured image and the second pixel information corresponding to the area outside the projection area, the processor 130 adjusts the pixel values ​​of the second image and controls the projector 110 to project the image onto the projection area. In this image, the first image with keystone correction is included in one area, and the second image with adjusted pixel values ​​is included in the remaining area.

[0066] For example, the preset image may include an image with a preset color, and the processor 130 may adjust the pixel values ​​of the second image so that the difference between the first pixel information and the second pixel information is less than a threshold, and may control the projector 110 to project an image with a preset resolution onto a projection area, in which the first image with keystone correction is included in one area and the second image with adjusted pixel values ​​is included in the remaining area.

[0067] The processor 130 can repeat the above operation to reduce the difference between the second region and the outer region of the projection area.

[0068] For example, processor 130 can acquire captured images of the projected area and the outer area via camera 120 at preset time intervals, and adjust the pixel values ​​of the second image based on the first pixel information corresponding to the projected area and the second pixel information corresponding to the outer area in each of the multiple captured images acquired at the preset time intervals. Processor 130 can repeat this operation, and stop acquiring captured images and adjusting pixel values ​​when the difference between the first pixel information and the second pixel information becomes less than a threshold.

[0069] The processor 130 can identify a pattern corresponding to the outer region of the projected area in the captured image, and can adjust the pixel values ​​of the second image based on the identified pattern. For example, if the screen includes a preset pattern, the outer region of the projected area in the captured image can be represented by the preset pattern. The processor 130 can identify the preset pattern in the captured image, and can adjust the pixel values ​​of the second image to include the preset pattern.

[0070] The processor 130 can adjust the pixel values ​​of the second image by further considering (or further based on) at least one of the brightness or color of the keystone-corrected image.

[0071] The electronic device 100 may also include a sensor, and the processor 130 may obtain sensing information including the ambient light of the electronic device via the sensor, and may adjust the pixel values ​​of the second image by further considering the sensing information.

[0072] The electronic device 100 may also include a sensor, and the processor 130 may obtain sensing information via the sensor, the sensing information including the angle of rotation of the electronic device relative to the direction of gravity and the angle of rotation of the electronic device relative to the projection area, and perform trapezoidal correction on the first image by changing the shape of the first image based on the sensing information.

[0073] The electronic device 100 may also include a user interface, and the processor 130 may adjust the pixel values ​​of the second image based on the color corresponding to a user command received via the user interface.

[0074] However, this disclosure is not limited thereto, and the processor 130 may also perform at least one of keystone correction or adjustment of the pixel values ​​of the second image based on user commands.

[0075] Processor 130 can recommend multiple colors to the user to adjust the pixel values ​​of the second image. For example, processor 130 can obtain pixel values ​​for adjusting the pixel values ​​of the second image based on first pixel information corresponding to the remaining areas in the captured image and second pixel information corresponding to the outer areas of the projected area, and recommend multiple pixel values ​​to the user by changing the obtained pixel values ​​in multiple predetermined ways. Processor 130 can obtain multiple pixel values ​​by changing at least one of the brightness or white balance of the obtained pixel values.

[0076] The processor 130 can control the projector 110 to project multiple colors. In response to a user command to select one of the multiple colors, the processor 130 can adjust the pixel values ​​of a second image to the selected color.

[0077] However, this disclosure is not limited thereto, and the electronic device 100 may also include a display, and the processor 130 may control the display to display multiple colors. The processor 130 may provide multiple colors to a user terminal device, and when a user selects one of the multiple colors using the user terminal device, after receiving information about the selected color from the user terminal device, adjust the pixel values ​​of the second image to the selected color.

[0078] The processor 130 can also obtain color information for adjusting pixel values ​​of the second image by inputting the captured image into a neural network model. The neural network model can refer to, for example, a model that has learned the relationship between sample captured images and sample color information to adjust the pixel values ​​of the second image.

[0079] The artificial intelligence-related functions disclosed herein can be operated via processor 130 and memory.

[0080] Processor 130 may include one or more processors. In this case, the one or more processors may be a general-purpose processor (e.g., CPU, AP, or DSP), a dedicated graphics processor (e.g., GPU or Visual Processing Unit (VPU)), or a dedicated artificial intelligence processor (e.g., NPU).

[0081] One or more processors can control the input data to be processed based on predefined operating rules or artificial intelligence models stored in memory. When one or more processors are dedicated AI processors, these processors can be designed with hardware architectures specifically designed to process particular AI models. The predefined operating rules or AI models are characterized by being created through training.

[0082] "Creating through training" can refer to, for example, training a basic artificial intelligence model using a large amount of training data via a training algorithm, thereby setting predefined operating rules or artificial intelligence to perform desired characteristics (or purposes). This training can be done within the device itself performing artificial intelligence according to this disclosure, or it can be done via a separate server and / or system. Examples of training algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, etc.

[0083] Artificial intelligence models can include multiple neural network layers. Each of these layers can have multiple weight values, and neural network computations are performed by calculating the results of the previous layer and these weights. The weights of multiple neural network layers can be optimized using the training results of the artificial intelligence model. For example, during training, multiple weights can be updated to reduce or minimize the loss or cost values ​​obtained from the artificial intelligence model.

[0084] Artificial neural networks can include, but are not limited to, deep neural networks (DNNs), such as convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), generative adversarial networks (GANs), deep Q-networks, etc.

[0085] Figure 3 This is a block diagram illustrating an example configuration of an electronic device according to various embodiments. Electronic device 100 may include a projector 110, a camera 120, and a processor (e.g., including processing circuitry) 130. Electronic device 100 may also include a sensor 140, a user interface (e.g., including interface circuitry) 150, a communication interface (e.g., including communication circuitry) 160, a memory 170, a display 180, a microphone 185, and a speaker 190. Here, the terms may be omitted. Figure 2 The components shown overlap Figure 3 Description of the components shown.

[0086] Sensor 140 may include a sensor for identifying the distance from electronic device 100 to the projection surface. For example, sensor 140 may include a ToF sensor. However, this disclosure is not limited thereto, and sensor 140 may be any sensor capable of identifying the distance from electronic device 100 to the projection surface. Alternatively, processor 130 may identify the distance from electronic device 100 to the projection surface via camera 120.

[0087] User interface 150 may include various interface circuits and may be implemented as buttons, touchpads, mice, keyboards, etc., or may be implemented as a touch screen capable of performing display and input functions. Here, buttons may be various types of buttons formed in any arbitrary area (e.g., front, side, rear, etc.), such as mechanical buttons, touchpads, scroll wheels, etc.

[0088] The communication interface 160 may include various communication circuits and is configured to communicate with various types of external devices according to various communication methods. For example, the electronic device 100 may communicate with a content server or user terminal device via the communication interface 160.

[0089] The communication interface 160 may include a Wi-Fi module, a Bluetooth module, an infrared communication module, a wireless communication module, etc. Here, each communication module can be implemented as at least one hardware chip.

[0090] The Wi-Fi and Bluetooth modules use Wi-Fi and Bluetooth methods respectively for communication. When using either a Wi-Fi or Bluetooth module, various connection information such as SSID and session key are first sent and received. After establishing a communication connection using this information, various other information can be sent and received. The infrared communication module performs communication based on Infrared Data Association (IrDA) communication technology, which uses infrared light—between visible light and millimeter waves—to wirelessly transmit data over short distances.

[0091] The wireless communication module includes at least one communication chip that performs communication according to various wireless communication standards, such as ZigBee, 3rd generation (3G), 3rd generation Partnership (3GPP), Long Term Evolution (LTE), LTE-A Advanced (LTE-A), 4th generation (4G) and 5th generation (5G).

[0092] The communication interface 160 may include wired communication interfaces such as HDMI, DP, Thunderbolt, USB, RGB, D-SUB, DVI, etc.

[0093] The communication interface 160 may include at least one of a local area network (LAN) module, an Ethernet module, or a wired communication module that uses twisted-pair, coaxial, or fiber optic cables to perform communication.

[0094] Memory 170 can refer to hardware that stores information (e.g., data) in electrical or magnetic form for access by processor 130 and the like. For this purpose, memory 170 can be implemented as at least one of the following hardware: non-volatile memory, volatile memory, flash memory, hard disk drive (HDD) or solid-state drive (SSD), RAM, ROM, etc.

[0095] The memory 170 may store at least one instruction required for the operation of the electronic device 100 or the processor 130. The instruction may include code units instructing the operation of the electronic device 100 or the processor 130, and may be written in machine language (a language that a computer can understand). The memory 170 may store multiple instructions as an instruction set for performing specific tasks of the electronic device 100 or the processor 130.

[0096] The memory 170 can store data, which is information in units of bits or bytes, and can represent characters, numbers, images, etc. For example, the memory 170 can store a trapezoidal correction module, an image analysis module, etc.

[0097] Processor 130 can access memory 170, and processor 130 can execute instructions, instruction sets, or read / write / modify / delete / update data.

[0098] The display 180 is configured to display content and can be implemented as various types of displays, such as, but not limited to, liquid crystal displays (LCDs), organic light-emitting diode (OLED) displays, and plasma display panels (PDPs). The display 180 may also include driving circuitry, backlight units, etc., which can be implemented as amorphous silicon (a-si) TFTs, low-temperature polycrystalline silicon (LTPS) TFTs, organic TFTs (OTFTs), etc. Furthermore, the display 180 can be implemented as a touchscreen combined with a touch sensor, a flexible display, a three-dimensional (3D) display, etc.

[0099] Microphone 185 is configured to receive sound and convert it into an audio signal. Microphone 185 is electrically connected to processor 130 and can receive sound under the control of processor 130.

[0100] For example, the microphone 185 may be integrally formed on the top, front, side, or other parts of the electronic device 100. Alternatively, the microphone 185 may be formed on a remote control or the like, which is separate from the electronic device 100. In this case, the remote control can receive sound through the microphone 185 and provide the received sound to the electronic device 100.

[0101] Microphone 185 may include various components, such as a microphone for collecting analog sound, an amplifier circuit for amplifying the collected sound, an A / D converter circuit for sampling the amplified sound and converting it into a digital signal, and a filter circuit for removing noise components from the converted digital signal.

[0102] The microphone 185 can be implemented as a sound sensor and can be configured in any way that allows it to collect sound.

[0103] The speaker 190 is configured to output not only various audio data processed by the processor 130, but also various notification sounds, voice messages, etc.

[0104] As shown above, the electronic device 100 can minimize and / or reduce the user's sense of heterogeneity by compensating for the external area generated by trapezoidal correction, thereby enhancing the user's sense of immersion.

[0105] In the following text, reference will be made to Figure 4 , Figure 5 , Figure 6 and Figure 7 (It can be called) Figures 4 to 7 The operation of the electronic device 100 will be described in more detail below. For ease of explanation, Figures 4 to 7 Various example embodiments are described herein. However, Figures 4 to 7 Various example implementations can be practiced in any combination.

[0106] Figure 4 This is a diagram illustrating example trapezoidal correction according to various embodiments.

[0107] The processor 130 can obtain sensing information via the sensor 140, which includes the angle of rotation of the electronic device 100 relative to the direction of gravity and the angle of rotation of the electronic device 100 relative to the projection area 410, and can identify whether to perform keystone correction based on the obtained sensing information.

[0108] For example, processor 130 can obtain the angle of rotation of electronic device 100 relative to the direction of gravity via accelerometer 140-1, and the angle of rotation of electronic device 100 relative to the projection area via multiple distance sensors 140-2, 140-3. When it is identified, based on the obtained angles, that the projection area (e.g., 410) is not parallel to the screen and is not rectangular, it can be determined that keystone correction will be performed. When keystone correction is determined, processor 130 can perform keystone correction on the first image based on the obtained angles. For example, processor 130 can obtain the projection matrix and transformation matrix based on the posture of electronic device 100 (represented as roll angle, pitch angle, and yaw angle) and distances d1, d2 from the screen, and perform keystone correction on the first image based on the projection matrix and transformation matrix.

[0109] However, this disclosure is not limited thereto, and the processor 130 can identify whether to perform keystone correction based on a user command. The processor 130 can obtain sensing information, and when it is identified that keystone correction will be performed based on the sensing information, it can notify the user that keystone correction needs to be performed, and perform keystone correction in response to the user's confirmation command.

[0110] When the electronic device 100 is turned on, the processor 130 can acquire sensing information and determine whether to perform keystone correction based on the acquired sensing information. When a change in the posture of the electronic device 100 is detected during image projection, the processor 130 can acquire sensing information and determine whether to perform keystone correction based on the acquired sensing information. In this case, the processor 130 can keep one of the accelerometer or multiple distance sensors always on. For example, the processor 130 can turn on the accelerometer to detect posture changes, and when a posture change is detected, turn on multiple distance sensors to acquire sensing information, and determine whether to perform keystone correction based on the acquired sensing information. The processor 130 can turn on multiple distance sensors to acquire the angle of rotation of the electronic device 100 relative to the screen, and when a posture change is detected, turn on the accelerometer to acquire sensing information, and determine whether to perform keystone correction based on the acquired sensing information.

[0111] The foregoing description illustrates the use of multiple distance sensors, but this disclosure is not limited thereto. For example, processor 130 may capture images of the screen via camera 120 and determine whether keystone correction should be performed based on the captured images.

[0112] Figure 5 This is a diagram showing an example external region produced by trapezoidal correction according to various embodiments.

[0113] Projector 110 can project images with a preset resolution. For example, projector 110 can project images with a resolution of 1920×1080. When projecting an image with a resolution of 1920×1080, processor 130 can control projector 110 to project the image onto the projection area without scaling. Alternatively, when the image does not have a resolution of 1920×1080, processor 130 can control projector 110 to project the image onto the projection area after scaling the image to a resolution of 1920×1080. Hereinafter, it is described that the first image is scaled to a preset resolution that projector 110 can project.

[0114] When the electronic device 100 is not placed parallel to the screen, the projection area may not be rectangular. For example, when the electronic device 100 is not placed parallel to the screen, the first image at 510-1 can be projected onto a projection area of ​​form 510-2 on the screen 520, such as... Figure 5 As shown at the top. In this case, processor 130 can perform keystone correction on the first image.

[0115] For example, processor 130 can distort the first image of 510 to look like 530-1, and include the distorted first image in a portion of the 540-1 region, such as Figure 5As shown in the lower part. Area 540-1 may correspond to a preset resolution that the projector 110 can project, and may be data prior to projection. In other words, the data prior to projection may include a distorted first image in area 530-1 of area 540-1 and a second image in the remaining areas of area 540-1.

[0116] Processor 130 can project data prior to projection onto the screen. Since electronic device 100 is not placed parallel to the screen, area 540-1 can be projected onto area 540-2 on screen 520. However, since the first image is represented using only area 530-1 within area 540-1, it can be projected onto the rectangular area 530-2 on screen 520. The second image can be projected onto area 540-2, excluding area 530-2.

[0117] Since the processor 130 adjusts the second image based on the screen 520 (excluding the 540-2 area), the processor 130 can minimize and / or reduce the differences in color, brightness, etc. between the 540-2 area (excluding the 530-2 area) and the screen 520 area (excluding the 540-2 area) to reduce the user's sense of heterogeneity.

[0118] Figure 6 This is a flowchart illustrating example methods for trapezoidal correction and external region compensation according to various embodiments.

[0119] The processor 130 can identify whether it is in automatic keystone correction mode (S610).

[0120] When in automatic keystone correction mode, processor 130 can obtain sensor information (S620-1) and perform coordinate calculations for automatic keystone correction (S630-1) to perform keystone correction image processing (S640). When not in automatic keystone correction mode, processor 130 can receive user input (S620-2) and perform coordinate calculations for manual keystone correction (S630-2) to perform keystone correction image processing (S640).

[0121] The processor 130 can project the processed image and identify whether it is in automatic external area correction mode (S650).

[0122] When in automatic external area correction mode, processor 130 can acquire the captured image (S660-1) and select the trapezoidal external area correction color (S670). On the other hand, when not in automatic external area correction mode, processor 130 can receive user input (S660-2) and select the trapezoidal external area correction color (S670).

[0123] The processor 130 can correct the outer region of the trapezoid based on the selected color (S680) and output the corrected image (S690).

[0124] Figure 7 This is a diagram illustrating a method for performing trapezoidal correction and external region compensation according to an embodiment.

[0125] Processor 130 can perform trapezoidal correction and external area compensation.

[0126] For example, the processor 130 may use at least one of the accelerometer 710-1, distance sensor 710-2, 3D TOF sensor 710-3 or image sensor 710-4 to obtain sensor information and process the obtained sensor information (720).

[0127] The processor 130 may perform image processing (730) on the original image based on at least one of the processed sensor information or user input. For example, the processor 130 may perform keystone correction on the original image based on at least one of the processed sensor information or user input.

[0128] The processor 130 can perform external region correction based on at least one of sensor information obtained from the image sensor or user input (S740).

[0129] The processor 130 can combine the processed original image and the corrected external region (750) and output an image (760).

[0130] Figure 8 This is a flowchart illustrating an example method for controlling an electronic device according to various embodiments.

[0131] Perform keystone correction on the first image (S810). Project an image with a preset resolution onto a projection area, in which the keystone-corrected first image is included in one area and the second image is included in the remaining area (S820). Obtain a captured image by capturing the projection area and the area outside the projection area (S830). Adjust the pixel values ​​of the second image based on the first pixel information corresponding to the remaining area in the captured image and the second pixel information corresponding to the outer area of ​​the projection area (S840). Project an image with a preset resolution onto the projection area, in which the keystone-corrected first image is included in one area and the second image with adjusted pixel values ​​is included in the remaining area (S850).

[0132] In addition, the preset image is an image with a preset color, and the adjustment (S840) step may include: adjusting the pixel values ​​of the second image so that the difference between the first pixel information and the second pixel information is less than a threshold.

[0133] The acquisition (S830) step may include acquiring a captured image by taking pictures of the projection area and the outer area at a preset time interval. The adjustment (S840) step may include adjusting the pixel value of the second image based on the first pixel information corresponding to the remaining area and the second pixel information corresponding to the outer area in each of the multiple captured images acquired at the preset time interval. The control method may also include stopping the acquisition of the captured image and adjusting the pixel value when the difference between the first pixel information and the second pixel information is less than a threshold.

[0134] The method may further include identifying a pattern corresponding to the outer region of the projected region in the captured image, and the adjustment (S840) step may include adjusting the pixel values ​​of the second image based on the identified pattern.

[0135] The adjustment (S840) step may include adjusting the pixel values ​​of the second image by further considering at least one of the brightness or color of the trapezoidally corrected image.

[0136] The method may also include obtaining sensing information including the ambient illuminance of the electronic device, and the adjustment (S840) step may include adjusting the pixel values ​​of the second image by further taking the sensing information into account.

[0137] The method may further include obtaining sensing information, including the angle of rotation of the electronic device along the direction of gravity and the angle of rotation of the electronic device relative to the projection area, and the step of performing (S810) may include performing trapezoidal correction on the first image by changing the shape of the first image based on the sensing information.

[0138] The adjustment (S840) step may include adjusting the pixel values ​​of the second image based on the color corresponding to the user command.

[0139] In addition, the projection area can be the area onto which light emitted from an electronic device is projected.

[0140] According to various embodiments of this disclosure, electronic devices can minimize and / or reduce the user's sense of heterogeneity by compensating for the external area generated by trapezoidal correction, thereby enhancing the user's sense of immersion.

[0141] Furthermore, computer instructions for performing processing operations of the apparatus according to the various embodiments described above may be stored in a non-transitory computer-readable medium. When executed by a processor of a specific device, the computer instructions stored in such a non-transitory computer-readable medium allow the specific device to perform the processing operations in the apparatus according to the various embodiments described above. A non-transitory computer-readable medium refers to a medium that stores data and can be read by a device. Specific examples of non-transitory computer-readable media may include CDs, DVDs, hard disks, Blu-ray discs, USB drives, memory cards, ROMs, etc.

[0142] Furthermore, components (e.g., modules or programs) according to the various embodiments described above may include a single entity or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be included in the various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into one entity and perform the same or similar functions performed by each corresponding component prior to integration. The operations performed by modules, programs, or other components according to the various embodiments may be performed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be performed in a different order or omitted, or other operations may be added.

[0143] While various exemplary embodiments of this disclosure have been shown and described above, this disclosure is not limited to the various exemplary embodiments described above, and various modifications can be made by those skilled in the art without departing from the spirit of this disclosure (including the appended claims), and such modifications should not be understood as departing from the technical concept or prospect of this disclosure. It will also be understood that any embodiment described herein can be used in conjunction with any other embodiment described herein.

Claims

1. An electronic device comprising: A projector, configured to project an image with a specified resolution; camera; as well as At least one processor, including processing circuitry, is connected to the projector and the camera, and is configured individually and / or collectively to control the electronic device. Wherein, the at least one processor is configured individually and / or collectively as follows: Perform trapezoidal correction on the first image; The projector is controlled to project an image of a specified resolution onto a projection area, wherein a first image with keystone correction is included in one area and a second image is included in the remaining areas; The camera captures images of the projection area and the area outside the projection area to obtain captured images. Based on the first pixel information corresponding to the remaining regions in the captured image and the second pixel information corresponding to the outer regions of the projected region, the pixel values ​​of the second image are adjusted; and The projector is controlled to project an image of a specified resolution onto the projection area, wherein the first image with keystone correction is included in the one area and the second image with adjusted pixel values ​​is included in the remaining area.

2. The electronic device according to claim 1, wherein, The specified image includes an image with specified colors; and The at least one processor is individually and / or collectively configured to adjust the pixel values ​​of the second image such that the difference between the first pixel information and the second pixel information is less than a threshold.

3. The electronic device according to claim 2, wherein, The at least one processor is configured individually and / or collectively to: The captured image is obtained by taking pictures of the projection area and the external area at specified time intervals using the camera; The pixel values ​​of the second image are adjusted based on the first pixel information corresponding to the remaining regions in each of the plurality of captured images obtained at the specified time interval and the second pixel information corresponding to the outer regions; as well as Based on the fact that the difference between the first pixel information and the second pixel information is less than a threshold, the acquisition of the captured image and the adjustment of the pixel values ​​are stopped.

4. The electronic device according to claim 1, wherein, The at least one processor is configured individually and / or collectively to: Identify the pattern corresponding to the outer region of the projection area in the captured image; as well as The pixel values ​​of the second image are adjusted based on the identified pattern.

5. The electronic device according to claim 1, wherein, The at least one processor is individually and / or collectively configured to further adjust the pixel values ​​of the second image based on at least one of the brightness or color of the keystone-corrected image.

6. The electronic device according to claim 1, further comprising: sensor, Wherein, the at least one processor is configured individually and / or collectively as follows: The sensor obtains sensing information including the ambient illuminance of the electronic device; and The pixel values ​​of the second image are further adjusted based on the sensed information.

7. The electronic device according to claim 1, further comprising: sensor, Wherein, the at least one processor is configured individually and / or collectively as follows: Sensing information is obtained through the sensor, including the angle of rotation of the electronic device relative to the direction of gravity and the angle of rotation of the electronic device relative to the projection area; and The trapezoidal correction is performed on the first image by changing its shape based on the sensing information.

8. The electronic device according to claim 1, further comprising: User interface, The at least one processor is individually and / or collectively configured to adjust the pixel values ​​of the second image based on the color corresponding to a user command received through the user interface.

9. The electronic device according to claim 1, wherein, The projection area includes the area onto which light emitted from the projector is projected.

10. A method for controlling an electronic device, comprising: Perform trapezoidal correction on the first image; An image with a specified resolution is projected onto a projection area, in which a first image, which has been keystone corrected, is included in one area and a second image is included in the remaining area; A captured image is obtained by photographing the projection area and the area outside the projection area; Based on the first pixel information corresponding to the remaining areas in the captured image and the second pixel information corresponding to the outer area of ​​the projected area, the pixel values ​​of the second image are adjusted; as well as The image of the specified resolution is projected onto the projection area, wherein the first image with trapezoidal correction is included in the one area and the second image with adjusted pixel values ​​is included in the remaining area.

11. The method according to claim 10, wherein, The specified image includes an image with the specified color; and The adjustment includes: adjusting the pixel values ​​of the second image so that the difference between the first pixel information and the second pixel information is less than a threshold.

12. The method according to claim 11, wherein, The acquisition includes: obtaining the captured image by taking pictures of the projection area and the external area at specified time intervals; The adjustment includes: adjusting the pixel values ​​of the second image based on the first pixel information corresponding to the remaining regions and the second pixel information corresponding to the outer regions in each of a plurality of captured images obtained at the specified time interval; and The method further includes: stopping the acquisition of the captured image and adjusting the pixel value based on the fact that the difference between the first pixel information and the second pixel information is less than a threshold.

13. The method of claim 10, further comprising: Identify the pattern corresponding to the outer region of the projected area in the captured image. The adjustment includes adjusting the pixel values ​​of the second image based on the identified pattern.

14. The method of claim 10, wherein, The adjustment includes further adjusting the pixel values ​​of the second image based on at least one of the brightness or color of the keystone-corrected image.

15. The method of claim 10, further comprising: Obtain sensing information including the ambient illuminance of the electronic device; and The adjustment includes further adjusting the pixel values ​​of the second image based on the sensing information.

Images