Projector and control method therefor

Abstract

A projector is disclosed. The projector comprises: a memory; a projection unit; a distance sensor for sensing a distance to a projection surface; an image sensor for imaging the projection surface; and a processor. The processor identifies the state of the projection surface on the basis of the results of sensing by the distance sensor and the image sensor, sets a correction weight for each region according to the identified state and stores the correction weight in the memory, applies the correction weight for each region to an image to be projected onto the projection surface to correct the image, and controls the projection unit to project the corrected image onto the projection surface.

Description

Projector and its control method

[0001] The present disclosure relates to a projector and a control method thereof, and more particularly, to a projector and a control method thereof that correct an image by taking into account an uneven projection surface and project the corrected image onto a projection surface.

[0002] Advances in electronic technology have led to the development and widespread adoption of various types of electronic devices. In particular, projectors, which can project images onto screens or walls, are used in a variety of settings, including homes, offices, and public spaces, and have continued to evolve in recent years.

[0003] A projector can project light generated from a light source onto a screen or wall through a projection lens. Projectors can also perform corrections, such as keystone correction, to provide the user with the optimal image.

[0004] Projectors project images by projecting light onto a projection surface. Uneven projection surfaces can cause the image itself to appear distorted. Conventional projectors attempt to address this issue by correcting the image before projecting. However, if a portion of the projection surface is uneven, unintended screen distortion can occur even when a corrected image is projected.

[0005] According to the present disclosure, a projector according to at least one embodiment includes a memory, a projection unit, a distance sensor for sensing a distance from a projection surface, an image sensor for capturing an image of the projection surface, and a processor. The processor identifies a state of the projection surface based on sensing results of the distance sensor and the image sensor, sets correction weights for each area according to the identified state, and stores the set correction weights in the memory. The processor applies the correction weights for each area to an image to be projected onto the projection surface to correct it, and controls the projection unit to project the corrected image onto the projection surface.

[0006] Meanwhile, a method for controlling a projector according to one or more embodiments of the present disclosure includes the steps of projecting a reference pattern image onto a projection surface, sensing a distance for each area of ​​the projection surface onto which the reference pattern image is projected using a distance sensor, capturing an image of the projection surface onto which the reference pattern image is projected using an image sensor, identifying a state of the projection surface based on a sensing value of the distance sensor and captured data of the image sensor, setting and storing a correction weight for each area of ​​the projection surface according to the identified state, applying the correction weight for each area to an image to be projected onto the projection surface to correct it, and projecting the corrected image onto the projection surface.

[0007] Meanwhile, a computer-readable recording medium including a program for executing a method for controlling a projector according to one or more embodiments of the present disclosure includes the steps of: projecting a reference pattern image onto a projection surface; sensing a distance for each area of ​​the projection surface onto which the reference pattern image is projected using a distance sensor; capturing an image of the projection surface onto which the reference pattern image is projected using an image sensor; identifying a state of the projection surface based on sensing values ​​of the distance sensor and the image sensor; setting and storing a correction weight for each area of ​​the projection surface according to the identified state; applying the correction weight for each area to an image to be projected onto the projection surface to correct it; and projecting the corrected image onto the projection surface.

[0008] FIG. 1 is a block diagram showing the configuration of a projector according to various embodiments of the present disclosure.

[0009] FIG. 2 is a block diagram showing an example of a detailed configuration of a projector according to various embodiments.

[0010] FIGS. 3 and 4 are drawings for explaining identification of an uneven area using a reference pattern image according to various embodiments.

[0011] FIGS. 5 to 8 are diagrams for explaining the setting of correction weights for non-uniform areas according to various embodiments.

[0012] Figures 9 to 11 are drawings for explaining a method of locally correcting the surrounding area of ​​an uneven area.

[0013] Figures 12 to 20 are flowcharts for explaining a method of controlling a projector according to various embodiments.

[0014] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail.

[0015] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0016] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0017] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

[0018] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0019] When it is said that a component (e.g., a first component) is “operatively or communicatively coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0020] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0021] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented as at least one processor (not shown), excluding any "modules" or "parts" that need to be implemented as specific hardware.

[0022] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0023] FIG. 1 is a block diagram showing the configuration of a projector according to various embodiments of the present disclosure.

[0024] A projector (100) refers to an electronic device that projects light onto various projection surfaces such as a wall, screen, floor, or ceiling to display an image. The projector (100) can be implemented in various forms, such as a mobile projector equipped with wheels for movement, a fixed projector that is fixed to a ceiling or wall for use, a basic projector that is placed on the floor for use, and an aerial projector that can be used by floating in the air like a drone.

[0025] According to FIG. 1, the projector (100) includes a memory (110), a projection unit (120), a distance sensor (130), an image sensor (140), and a processor (150).

[0026] The memory (110) can store at least one command, data, program, etc. required for the operation of the projector (100). For example, the memory (110) can store data on a reference pattern image. The reference pattern image includes an image used to inspect the uniformity of the projection surface. The reference pattern image can be prepared in various forms. For example, it can be implemented in various forms such as a checkered pattern composed of multiple grids, a matrix pattern in which multiple dots, lines, symbols, characters, etc. are arranged in a matrix shape, etc. The memory (110) can store various data on the size, shape, color, display position, etc. of the reference pattern image. The memory (110) can be implemented in the form of a memory embedded in the projector (100) or in the form of a memory that can be attached or detached to the projector (100) depending on the purpose of data storage. For example, data for driving the projector (100) may be stored in a memory embedded in the projector (100), and data for the expansion function of the projector (100) may be stored in a memory that is detachable from the projector (100).

[0027] In the case of memory embedded in the projector (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD)).

[0028] The memory (110) may be implemented as a single memory that stores data generated from various operations according to the present disclosure, but is not limited thereto, and the memory (110) may be implemented to include multiple memories that each store different types of data or each store data generated at different stages.

[0029] The projection unit (120) is configured to project light to the outside to express an image.

[0030] The projection unit (120) can be implemented in various projection methods (e.g., CRT (cathode-ray tube) method, LCD (Liquid Crystal Display) method, DLP (Digital Light Processing) method, laser method, etc.).

[0031] When implemented in a CRT manner, the projection unit (120) may include a cathode ray tube (CRT) and a lens. When an image is displayed on the cathode ray tube of the projection unit (120), the light emitted from the cathode ray tube is magnified through the lens and projected to the outside. Depending on the number of cathode ray tubes, it is divided into a single-tube type and a three-tube type, and in the case of a three-tube type, the red, green, and blue cathode ray tubes may be implemented separately.

[0032] When implemented in an LCD manner, the projection unit (120) may include a light source, a liquid crystal display (LCD), other lenses, etc. The LCD method is a method of displaying an image by transmitting light from a light source through a liquid crystal display. The LCD method is divided into a single-panel type and a three-panel type. In the case of the three-panel type, the light from the light source is separated into red, green, and blue by a dichroic mirror (a mirror that reflects only light of a specific color and transmits the rest), and then passes through the liquid crystal display and then the light is gathered into one place again.

[0033] The DLP method is a method of displaying images using a DMD (Digital Micromirror Device) chip. The DLP projection unit (120) may include a light source, a color wheel, a DMD chip, a projection lens, etc. Light output from the light source may be colored as it passes through a rotating color wheel. The light passing through the color wheel is input to the DMD chip. The DMD chip is configured to include numerous micromirrors. The DMD chip reflects the input light. The projection lens may play a role in magnifying the light reflected from the DMD chip to an image size.

[0034] As another example, the laser-based projection unit (120) includes a Diode Pumped Solid State (DPSS) laser and a galvanometer. To output various colors, DPSS lasers may be provided for each RGB color. The galvanometer uses a motor to rapidly rotate a mirror to reflect the laser. For example, the galvanometer can rotate the mirror at a maximum speed of 40 KHz / sec.

[0035] The projection unit (120) may include various types of light sources. For example, the projection unit (120) may include at least one light source among a lamp, an LED, and a laser.

[0036] The projection unit (120) can output images in a 4:3 screen ratio, a 5:4 screen ratio, or a 16:9 wide screen ratio depending on the purpose of the projector (100) or the user's settings, and can output images in various resolutions such as WVGA (854*480), SVGA (800*600), XGA (1024*768), WXGA (1280*720), WXGA (1280*800), SXGA (1280*1024), UXGA (1600*1200), Full HD (1920*1080), UHD (3840*2160), etc. depending on the screen ratio.

[0037] The projection unit (120) can perform various functions for adjusting the output image under the control of the processor (150). For example, the projection unit (120) can perform functions such as zoom, keystone, quick corner (4 corners), and lens shift.

[0038] Specifically, the projection unit (120) can perform a zoom function to enlarge or reduce an image depending on the distance from the screen (projection distance).

[0039] The method of performing a zoom function can be divided into a hardware method that adjusts the screen size by moving the lens and a software method that adjusts the screen size by cropping the image. When the zoom function is performed, the focus of the image needs to be adjusted. Focus adjustment can be performed using a manual focus method or an electric focus method. The manual focus method refers to a method of focusing manually. The electric focus method refers to a method of automatically focusing using a motor. The projector (100) can selectively provide a digital zoom function or an optical zoom function.

[0040] The projection unit (120) can also perform a keystone correction function. If the height is not right for frontal projection, the screen may be distorted upwards or downwards. The keystone correction function refers to a function that corrects a distorted screen. For example, if distortion occurs in the left and right directions of the screen, it can be corrected using horizontal keystone, and if distortion occurs in the up and down directions, it can be corrected using vertical keystone. The quick corner (4 corner) correction function is a function that corrects the screen when the center area of ​​the screen is normal but the corner areas are not balanced. The lens shift function is a function that moves the screen as it is when the screen is off the screen.

[0041] The projection unit (120) can automatically analyze the surrounding environment and projection environment and provide zoom / keystone / focus functions even without user operation. Specifically, the projection unit (120) can automatically provide zoom / keystone / focus functions based on the distance between the projector (100) and the screen detected by a sensor (depth camera, distance sensor, infrared sensor, light sensor, etc.), information about the space where the projector (100) is currently located, information about the amount of ambient light, etc.

[0042] Additionally, the projection unit (120) can provide a lighting function using a light source. In particular, the projection unit (120) can output light using a light source such as one or more LEDs.

[0043] The projection unit (120) may output a light source using a surface-emitting LED, depending on the implementation example. The surface-emitting LED is an LED having a structure in which an optical sheet is arranged on the upper side of the LED so that light is evenly distributed and output.

[0044] The projection unit (120) may also provide the user with a dimming function for adjusting the intensity of the light source. Specifically, when the user inputs a command for adjusting the intensity of the light source through various operation interfaces such as a touch screen, button, dial, etc., the projection unit (120) may control the LED to output light with an intensity corresponding to the user command. Alternatively, the projection unit (120) may provide a dimming function based on content analyzed by the processor (150) without user input. Specifically, the projection unit (120) may control the LED to output the intensity of the light source based on information about the currently provided content (e.g., content type, content brightness, etc.).

[0045] The projection unit (120) can also control the color temperature under the control of the processor (150).

[0046] The distance sensor (130) is a configuration for sensing the distance to an external object. The processor (150) can identify the distance to a projection surface or other external objects based on the sensing value of the distance sensor (130). The distance sensor (130) may include at least one of an ultrasonic sensor, an infrared sensor, a laser sensor, an optical distance sensor, a radar (RADAR) sensor, a lidar (LIDAR) sensor, a photodiode sensor, and a time of flight (TOF) sensor.

[0047] For example, when implemented as an ultrasonic sensor, the distance sensor (130) includes an output unit and a receiver unit. When the output unit outputs sound waves, the receiver unit can receive the sound waves being echoed by the target object. The processor (150) can calculate the distance to the target object using the time difference between the output time and the reception time of the sound waves based on the speed of sound (340 m / s).

[0048] When implemented as an infrared sensor, the distance sensor (130) includes a light emitter and a light receiver. When the light emitter emits infrared light, the infrared light may be reflected upon striking a target object. The light receiver detects the reflected signal. Since ultrasonic sensors use light rather than sound waves, they can measure distances by calculating the angle between the focus formed on the target object and the sensor based on optical triangulation.

[0049] In addition, the TOF sensor is a sensor that measures the distance to an object using signals such as near-infrared, ultrasonic, and laser. The TOF sensor may include a transmitter that outputs various signals and a receiver that receives a corresponding reflected signal. The TOF sensor can measure the distance between a device equipped with the sensor, i.e., a projector (100), and the object by using the time (time of flight) it takes for a signal emitted from the transmitter to be reflected by the object and return.

[0050] The image sensor (140) is a component for capturing images. The image sensor (140) can capture images of various external objects, including projection surfaces, under the control of the processor (150). For example, the image sensor (140) provided in a camera can convert an image captured through a camera lens into a digital signal and generate image data based on the converted signal. The image sensor (140) can be classified into a CMOS (Complementary Metal Oxide Semiconductor) image sensor and a CCD (Charge-Coupled Device) image sensor, depending on its structure.

[0051] The processor (150) is a component connected to each component of the projector (100) to control the overall operation of the projector (100). The processor (150) may be implemented as a digital signal processor (DSP), a microprocessor, a GPU (Graphics Processing Unit), an AI (Artificial Intelligence) processor, an NPU (Neural Processing Unit), or a TCON (Time Controller) for processing a digital image signal. However, the processor (150) is not limited thereto, and may include one or more of a central processing unit (CPU), a MCU (Micro Controller Unit), an MPU (micro processing unit), a controller, an application processor (AP), a communication processor (CP), or an ARM processor, or may be defined by the relevant terminology. In addition, the processor (150) may be implemented as a SoC (System on Chip), an LSI (Large Scale Integration) having a built-in processing algorithm, or may be implemented in the form of an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array).

[0052] In addition, the processor (150) for executing the artificial intelligence model according to one embodiment may be implemented through a combination of a general-purpose processor such as a CPU, an AP, a DSP (Digital Signal Processor), a graphics-only processor such as a GPU, a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU, and software. The processor (150) may be controlled to process input data according to a predefined operation rule or artificial intelligence model stored in a memory. Alternatively, if the processor (150) is a dedicated processor (or an artificial intelligence-only processor), it may be designed with a hardware structure specialized for processing a specific artificial intelligence model. For example, the hardware specialized for processing a specific artificial intelligence model may be designed with a hardware chip such as an ASIC or an FPGA.

[0053] When the processor (150) is implemented as a dedicated processor, it may be implemented to include memory for implementing embodiments of the present disclosure, or may be implemented to include a memory processing function for utilizing external memory. The processor (150) may be implemented as one or more processors.

[0054] When projecting an image onto a projection surface using a projector (100), if an uneven area exists within the projection surface, the distortion of the screen may not be resolved by keystone correction. In particular, in the case of an ultra-short throw (UST) projector, since the image is projected from a close distance, such screen distortion has a significant impact on the display of the projected screen.

[0055] The processor (150) identifies the state of the projection surface based on the sensing results of the distance sensor (130) and the image sensor (140). The processor (150) can activate the distance sensor (130) to identify the distance to the projection surface. In addition, the processor (150) can activate the image sensor (140) to obtain a photographed image of the projection surface. In the present disclosure, activation may mean an operation of turning on the sensor or controlling the sensor to perform its function while turned on.

[0056] Specifically, the processor (150) can compare the sensing results of the distance sensor (130) with the sensing results of the image sensor (140), and identify an uneven area of ​​the projection surface based on the comparison results. In the present disclosure, an uneven area means an area in which the uniformity of the surface condition is below a certain range. For example, an area that includes at least one protruding or sunken area, making it difficult to display a flat image, can be identified as an uneven area.

[0057] For example, the processor (150) may control an image sensor (140) capable of wide-area sensing to first capture an image of the entire projection surface, identify an uneven area of ​​the projection surface based on the capture result, and control a distance sensor (130) to sense the distance from the projection surface to the identified uneven area.

[0058] The processor (150) sets correction weights for each area according to the status of the identified projection surface and stores them in the memory (110). The correction weights for each area refer to weights set to individually apply correction levels to each area when the entire area of ​​the image to be projected onto the projection surface is divided into multiple areas. The method for setting the correction weights for each area of ​​the projection surface will be described again in the following section.

[0059] The processor (150) can set different correction weights for each area of ​​an image to be projected onto a projection surface and store them in the memory (110). When the processor (150) projects an image onto the projection surface, it applies the correction weights for each area stored in the memory (110) to correct the image and then controls the projection unit (120) to project the corrected image. Accordingly, correction at the same level is not performed on the entire image, but correction can be performed locally on an image area to be displayed in an uneven area within the projection surface and its surrounding area.

[0060] In FIG. 1, the projector (100) is illustrated and described as including basic configurations for sensing the state of the projection surface and correcting the projection image based on the sensing result, but the projector (100) may further include additional configurations for correcting the projection image by identifying the state of the projection surface in various ways.

[0061] FIG. 2 is a block diagram showing an example of a detailed configuration of a projector according to various embodiments.

[0062] According to FIG. 2, the projector (100) may further include a memory (110), a projection unit (120), a distance sensor (130), an image sensor (140), a processor (150), and a direction detection sensor (160), a communication interface (171), an operation interface (172), an input / output interface (173), a display (174), a speaker (175), a microphone (176), a power supply (177), etc. However, the present invention is not limited thereto, and the projector (100) may further include other components, or some of the components may be omitted or changed. Since the memory (110), the projection unit (120), the distance sensor (130), the image sensor (140), and the processor (150) of the configuration of FIG. 2 have been described in FIG. 1, a redundant description thereof will be omitted.

[0063] The direction detection sensor (160) is configured to sense the projection direction of the projection unit (120). The direction detection sensor (160) may include at least one of a gyro sensor, a geomagnetic sensor, a tilt sensor, and a compass.

[0064] The processor (150) can identify the projection direction of the image and the tilt state of the projector (100) based on the sensing result of the direction detection sensor (160), and can adjust at least one of the image projection angle, focal length, and projection distance of the projection unit (120) based on the identified state.

[0065] The communication interface (171) is a configuration for performing communication with at least one external device. The communication interface (171) may include at least one wireless communication module, at least one wired communication module, etc. Each communication module may be implemented in the form of at least one hardware chip. For example, the wireless communication module may include at least one module among a Wi-Fi module, a Bluetooth module, an infrared communication module, or other communication modules. In addition, the communication interface (171) may include at least one communication chip for performing communication according to various wireless communication standards such as Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.

[0066] The wired communication module may include, for example, at least one of a Local Area Network (LAN) module, an Ethernet module, a pair of cables, a coaxial cable, a fiber optic cable, or an Ultra Wide-Band (UWB) module. The projector (100) may perform various operations, such as demultiplexing, deinterleaving, decoding, scaling, and filtering, on content data provided from an external source (e.g., a web server, a broadcasting station, a multimedia playback device, etc.) connected via a communication interface (171) to configure the content data into an image, and project the configured image onto a projection surface via a projection unit (120).

[0067] The operation interface (172) is configured to receive user operation input. The operation interface (172) may include various buttons, a touch screen, etc. provided on the main body of the projector (100).

[0068] The input / output interface (173) is a configuration for inputting and outputting various external signals. The input / output interface (173) can input at least one of audio and image signals from an external device, and can output a control command to the external device. The input / output interface (173) can be implemented as at least one wired input / output interface among HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), USB C-type, DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (Dsubminiature), and DVI (Digital Visual Interface). In Fig. 2, the input / output interface (173) and the communication interface (171) are illustrated separately, but when communication with an external device is performed through the input / output interface (173), the input / output interface (173) can also be viewed as a configuration included in the communication interface (171).

[0069] The display (174) is configured to display the operating status of the projector (100), notification messages, UI screens, etc. The display (174) can be implemented as various types of displays such as an LCD (Liquid Crystal Display) and an OLED (Organic Light Emitting Diodes) display. Alternatively, the display (174) can be implemented with only one or more light-emitting elements. The processor (150) can change the display status of the display (174) according to various states such as when the projector (100) is turned on, in a normal operating state, insufficient power, or in an error state, so that the user can intuitively understand the status of the projector (100).

[0070] The speaker (175) is a component that outputs an audio signal. In Fig. 2, only the speaker (175) is illustrated, but various components such as an audio decoder, an audio output mixer, an audio signal processor, and an amplifier circuit for processing an audio signal may be further included in the projector (100). The speaker (175) may be configured as one or more, and when implemented as multiple speakers, they may be arranged symmetrically on the exterior of the main body to output audio signals in all directions, that is, in all 360 degrees.

[0071] The microphone (176) is a component for receiving various audio signals. The microphone (176) can receive a user's voice or other sounds and convert them into audio data. The microphone (176) can be formed integrally on the upper side, front side, side side, etc. of the projector (100). The projector (100) can include various components such as an amplifier circuit for amplifying the audio signal received through the microphone (176), an A / D conversion circuit for sampling the amplified audio signal and converting it into a digital signal, and a filter circuit for removing noise components from the converted digital signal.

[0072] The power supply unit (177) can supply power to various components of the projector (100) by receiving power from an external source. The power supply unit (177) can receive power through various methods. For example, if a charging station (not shown) for charging is included in the environment in which the projector (100) operates, the power supply unit (177) can receive power through the charging station. Alternatively, the power supply unit (177) can receive power from an external device, power source, etc. through various wired or wireless charging methods. Alternatively, the power supply unit (177) may include a battery. When an external power source is connected, the power supply unit (177) can charge and use the power of the battery using an electric signal applied from the external power source. Alternatively, the power supply unit (177) may be implemented with a structure in which a replaceable battery is mounted.

[0073] Although not shown in FIG. 2, when the projector (100) is implemented as a mobile type, it may further include at least one motor, a plurality of wheels, at least one shaft and gear for transmitting the power of the motor to the plurality of wheels, etc.

[0074] As described above, the projector (100) may include various configurations depending on its type. Accordingly, the projector (100) may project images onto various types of projection surfaces, including projection surfaces such as walls, ceilings, and floors that partially contain uneven areas. If the processor (150) determines that the projection surface contains an uneven area, it identifies the area and locally corrects the image as described above. Below, an example of a method for identifying an uneven area will be described in detail.

[0075] Figures 3 and 4 are diagrams for explaining the identification of an uneven area using a reference pattern image according to various embodiments. Figure 3 shows a case where a reference pattern image is projected onto a projection surface while the entire projection surface is uniform. The reference pattern image in Figure 3 represents a grid-like pattern in which lines or points are regularly arranged.

[0076] The processor (150) can obtain data on a reference pattern image stored in the memory (110) and control the projection unit (120) to project an image corresponding to the data onto a projection surface. In addition, the processor (150) can control the image sensor (140) to capture a projection surface onto which the reference pattern image is projected.

[0077] Figure 3 illustrates a case where a reference pattern image of a grid pattern is projected on a projection surface without any uneven areas. In this case, the reference pattern image projected on the projection surface does not show grid curvature, and the distance between each line is maintained at a constant interval.

[0078] Fig. 4 illustrates a case where a reference pattern image is projected when the outer area of ​​the projection surface is uniform but an uneven area exists in the interior of the projection surface. As shown in Fig. 4, the pattern of the edge areas of the reference pattern image projected on the projection surface maintains a certain interval between lines, but in the interior area, it can be seen that the grid is curved and the interval between each line is also deformed. If the surface of the projection surface is uneven or if there are decorations or insects attached to the projection surface, only a part of the projection surface, not the entire surface, may be uneven. In such a case, if the reference pattern image is projected, some uneven areas can be identified, as shown in Fig. 4.

[0079] The processor (150) can identify an uneven area within the projection surface based on the captured image of the image sensor (140) and the sensing value of the distance sensor (130). Specifically, the processor (150) can identify an uneven area within the projection surface based on the sensing result of the image sensor (140) that captured a reference pattern image projected onto the projection surface. The processor (150) can compare the reference pattern image stored in the memory (110) with the reference pattern image of the projection surface captured through the image sensor (140), and identify an uneven area within the projection surface based on the comparison result.

[0080] Next, the processor (150) can sense the distance between the projector (100) and the projection surface using the distance sensor (130), and identify an uneven area within the projection surface based on the sensing result. In this case, the processor (150) can generate coordinate information based on the reference pattern image projected on the projection surface, and control the distance sensor (130) according to the generated coordinate information to sense the distance from the projection surface for each area or coordinate point of the projection surface. For example, if the projector (100) is placed on the floor and the projection surface is vertically erected at a certain distance away, the distance to the upper edge of the reference pattern image is measured to be longer than the distance to the lower edge depending on the arrangement position of the distance sensor (130) within the projector (100). That is, the distance increases at a constant rate as it goes upward depending on the height of the projector (100) and the height of the reference pattern image.

[0081] However, if there is some uneven area, the distance to each point within the uneven area is measured non-constantly. That is, the distance may be measured to be further for a sunken area than for an adjacent point, and may be measured to be closer for a protruding area. Therefore, if the distances between points of the same height within the reference pattern image are measured differently, the processor (150) may identify the area as an uneven area. In addition, the processor (150) may identify the uneven area by comparing the uneven state of the projection surface sensed through the distance sensor (130) with the uneven state of each area within the captured image captured by the image sensor (140). For example, the processor (150) may compare the first uneven area identified through the distance sensor (130) with the second uneven area identified through the image sensor (140), and determine the common area where the first uneven area and the second uneven area overlap as the uneven area to be corrected on the projection surface based on the comparison result.

[0082] Alternatively, the processor (150) may combine the first and second uneven areas based on the comparison results, and determine the combined uneven area of ​​the two areas as the uneven area to be corrected on the projection surface. In the above-described section, it was described that the shooting data of the image sensor is first checked and then the sensing value of the distance sensor is checked, but the order is not necessarily limited, and the two tasks may be reversed, or the two tasks may be performed in parallel.

[0083] When a non-uniform region is identified, the processor (150) performs processing to locally correct the image. Specifically, the processor (150) can set and apply different correction weights for each region.

[0084] FIGS. 5 to 8 are diagrams for explaining the setting of correction weights for an uneven area according to various embodiments. FIG. 5 shows original image data stored in a memory (110), and FIG. 6 shows a state in which an image is projected onto a projection surface in a state in which the projection surface is uneven. For the convenience of explanation, objects (510, 520, 530) in the form of circular figures are illustrated in FIGS. 5 and 6, but this is merely an example, and actual images may include various objects such as people, buildings, and cars.

[0085] According to FIG. 5, the first object (510) is an ellipse that is elongated in the vertical direction, but is almost circular. However, due to the area non-uniformity, FIG. 6 shows a case where the width of the upper and lower sides is reduced and it is expressed as if it were an almost circular object (610).

[0086] The second object (520) of Fig. 5, on the other hand, is almost circular, but in Fig. 6, the width of the upper and lower sides is increased, and it is expressed as an oval object (620) that is elongated in the vertical direction. The third object (530) of Fig. 5 is expressed as an object (630) of the same size as in Fig. 6 without any particular distortion.

[0087] Comparing FIGS. 5 and 6, it can be seen that the area where the third object (530, 630) is located on the projection surface is uniform, while the areas where the first object (510, 610) and the second object (520, 620) are located are uneven. In addition, it can be seen that the area where the first object (510, 610) is located and the area where the second object (520, 620) is located in FIG. 6 have different projection surface heights.

[0088] Fig. 7 illustrates a case where image correction is performed in batches based on a specific uneven area on the projection surface. Fig. 7 illustrates a state in which the same correction weight is applied to the first object (610) and the second object (620) located in the uneven area in Fig. 6 for correction. According to Fig. 7, the second object (620) of Fig. 6 is corrected with a correction value with the correction weight applied, and is shown in a state (720) in which it is corrected to have almost the same shape as the first object (520) of the original image. However, it can be confirmed that the correction weight is excessively applied to the first object (610) before correction in Fig. 6, and the vertical width is displayed in a state (710) in which it is greatly increased after correction. Therefore, when the uneven state of the projection surface is different for each area, it is necessary to set the correction weight differently for each identified area.

[0089] The processor (150) can set different correction weights for the identified non-uniform area and the surrounding area and store them in the memory (110), and correct the image projected onto the projection surface based on the different set correction weights.

[0090] The processor (150) can compare the captured data of the image sensor (140) with the data for the reference pattern image stored in the memory (110), and measure the degree to which the reference pattern image is deformed in the non-uniform region based on the comparison result. Specifically, the processor (150) can calculate the amount of deformation for each coordinate position of the reference pattern image projected on the projection surface based on the reference pattern image stored in the memory (110). The processor (150) can set a correction weight for each region for the non-uniform region according to the calculated amount of deformation for each coordinate position.

[0091] Fig. 8 shows a drawing that has been corrected by setting different correction weights for each uneven area of ​​the projection surface. For example, Fig. 8 shows an image that has been corrected by applying a correction weight of 80% to the uneven area where the object (620) of Fig. 6 is located, and by applying a correction weight of 20% to the uneven area where the object (610) is located.

[0092] As described above, the processor (150) can set different correction weights depending on the amount of deformation for each coordinate position of the entire non-uniform area.

[0093] Alternatively, if there are continuous non-uniform regions among the entire non-uniform regions, the processor (150) identifies the region with the most severe non-uniformity among the non-uniform regions. Specifically, the processor (150) can identify the region with the greatest difference in line spacing of the reference pattern image and the region with the greatest difference in distance from surrounding points within the photographed data as the region with the most severe non-uniformity. The processor (150) can set the correction weight for the region with the most severe non-uniformity to a reference value (e.g., 100% or 1) and gradually reduce the correction weight in a direction away from the reference value.

[0094] Alternatively, the processor (150) may set the correction weight of the non-uniform region located at the center among the consecutive non-uniform regions as a reference value (e.g., 100% or 1) and gradually reduce the correction weight in a direction away from it.

[0095] Alternatively, when the non-uniform areas are separated from each other as in the examples described in FIGS. 5 to 8, the correction weights may be set differently for each non-uniform area based on the size difference between the original image and the distorted image. In FIGS. 5 to 8, since the second object (520) is more distorted than the first object (510), the processor (150) may set the correction weight of the second object (520) to be greater than the correction weight of the first object (510).

[0096] The method of setting the correction weights may vary depending on various embodiments.

[0097] For example, the processor (150) may use machine learning to set correction weights for non-uniform areas of an image. In this case, the processor (150) may use a supervised learning method or an unsupervised learning method to set the correction weights.

[0098] Alternatively, the processor (150) may measure the difference value for each coordinate point between the original image data stored in the memory (110) and the captured data based on the coordinate points of the reference pattern image. The processor (150) may measure the similarity between the original image data and the captured data through a histogram comparison obtained based on the measured difference value, and may repeatedly perform a process of correcting the similarity using deep learning to determine the final correction weight.

[0099] Alternatively, the processor (150) may identify the state of the projection surface based on the sensing results of the distance sensor (130) and the image sensor (140), and set a correction weight for each area according to the identified state. The processor (150) may identify the maximum value among the correction weights set for each area, set the correction weight based on the uneven area identified as the maximum correction weight in the image, and then sequentially decrease the correction weights of the surrounding areas. In this case, the processor (150) may capture a projection surface on which a corrected image is projected using the image sensor (140), and compare the captured image with a reference pattern image stored in the memory (110) to measure the similarity for the corrected image. For example, the processor (150) may compare the uneven state for the corrected image for each area, and if the similarity is 90% or more through the comparison result, the correction weight may be set to the final value.

[0100] FIGS. 9 to 11 are drawings for explaining a method of locally correcting the surrounding area of ​​an uneven area. FIG. 9 is a drawing showing a pattern image projected in a uniform projection surface, FIG. 10 shows a pattern image projected on an uneven area, and FIG. 11 shows a pattern image corrected by applying a correction weight to the uneven area of ​​FIG. 10. In addition, for convenience of explanation, only horizontal lines of the reference pattern image are shown in FIGS. 9 to 11, but in actual correction, correction can be performed in the same manner in the vertical line direction.

[0101] According to Fig. 10, a non-uniform region (1010) is identified within the projection surface, and the upper region (1020) and lower region (1030) of the identified non-uniform region exhibit a uniform state. In this case, if correction is performed by applying a correction weight only to the identified non-uniform region (1010), unintended image distortion may occur in the surrounding region of the non-uniform region.

[0102] According to FIG. 11, when correction is performed only for the non-uniform region (1010) without considering correction for the surrounding regions (1020, 1030) of the non-uniform region (1010) in FIG. 10, it can be seen that distortion occurs in the images of the upper region (1120) and the lower region (1130) due to the influence of the correction of the region (1110).

[0103] For example, when performing correction for the non-uniform region (1010) of FIG. 10, the processor (150) may apply a correction value in the downward direction to the two upper lines distorted in the upward direction, and may apply a correction value in the upward direction to the two lower lines distorted in the downward direction. In this case, if correction for the peripheral regions (1020, 1030) is not considered, the peripheral region (1110) in FIG. 11 experiences image distortion in the downward direction due to the correction effect for the upper part of the non-uniform region (1110), and the peripheral region (1130) experiences image distortion in the upward direction due to the correction effect for the lower part of the non-uniform region (1110).

[0104] The processor (150) can set a correction weight different from that of the non-uniform region for the surrounding area of ​​the identified non-uniform region, and can locally warp the image based on the differently set correction weight to correct it. The processor (150) can set a correction weight relatively smaller than the correction weight set for the non-uniform region for the surrounding area.

[0105] For example, the processor (150) may subdivide the identified non-uniform region into unit regions of a preset size and set different correction weights for each subdivided unit region. In addition, the processor (150) may also subdivide the surrounding region of the non-uniform region into unit regions of a preset size and sequentially decrease the correction weights for each unit region of the subdivided surrounding region based on the difference in correction weights between the most adjacent unit regions in the surrounding region.

[0106] The processor (150) controls the image sensor (140) to capture a projection surface on which a corrected image is projected, and may also determine whether to re-correct based on the captured image of the image sensor (140).

[0107] For example, the processor (150) may control the projection unit (120) to project the corrected frame after correcting at least one frame of the actual content image using a correction weight. The processor (150) may control the image sensor (140) to capture the corrected frame projected on the projection surface, and then compare the captured image with the original frame to determine whether re-correction is required.

[0108] Alternatively, the processor (150) may correct the reference pattern image by applying a correction weight. The processor (150) may control the projection unit (120) to project the corrected reference pattern image, and then control the image sensor (140) to capture the corrected reference pattern image projected onto the projection surface. The processor (150) may compare the captured image with the reference pattern image stored in the memory (110) to determine whether the corrected image needs to be re-corrected.

[0109] That is, the processor (150) compares the non-uniformity state of each region for the non-uniform region based on the correction frame and the original frame or the corrected reference pattern image and the reference pattern image before correction, and can re-perform correction by changing the correction weight for each region based on the comparison result.

[0110] The processor (150) may periodically determine whether an uneven area exists and whether it needs to be corrected. Specifically, the processor (150) may control the image sensor (140) to capture an image every certain period (e.g., 10 minutes) while projecting an image corrected by applying a correction weight, and then compare the image with the original image to determine whether an uneven area exists. If an uneven area is found in the corrected image as a result of the confirmation, the correction weight may be readjusted.

[0111] Alternatively, the processor (150) may periodically project a corrected reference pattern image, compare it with the reference pattern image stored in the memory (110), and then determine whether to re-correct the corrected image based on the comparison result. For example, if the projection surface is a screen, some areas of the screen may be curved, such as raised or lowered, due to wind or external force. Accordingly, the screen state at the time of correction and the subsequent screen state may change. Alternatively, the state of the projection surface may change if an insect or other foreign substance adheres to it and then falls off. Accordingly, the processor (150) may periodically perform the above-described re-correction operation to adaptively respond to changes in the state of the projection surface.

[0112] Meanwhile, a plurality of reference pattern images may be prepared and stored in the memory (110). In this case, the processor (150) may perform correction using a different reference pattern image for each cycle. For example, the memory (110) may store data on a plurality of reference pattern images implemented in various forms, such as a checkered pattern composed of a plurality of grids, a matrix pattern in which a plurality of dots, lines, symbols, characters, etc. are arranged in a matrix-like manner, etc.

[0113] The processor (150) can correct an image by applying a correction weight set for each of a plurality of reference pattern images, and control the projection unit (120) so that the corrected plurality of reference pattern images are continuously projected onto the projection surface. The processor (150) can control the image sensor (140) so that the projection surface on which the corrected plurality of reference pattern images are projected is photographed according to a preset cycle, and can also periodically compare the photographed images of the image sensor (140) with the reference pattern images to periodically determine whether to re-correct the corrected images.

[0114] Accordingly, the processor (150) periodically determines whether to re-correct the corrected image based on a plurality of reference pattern images implemented at various angles, sizes, and shapes, and re-corrects the correction weights according to the determination result, thereby preventing image distortion from occurring even when images of various shapes are projected on an uneven area of ​​the projection surface. The processor (150) controls the image sensor (140) to capture the entire projection surface on which the corrected image is projected, and compares the captured image of the image sensor (140) with the reference pattern image stored in the memory (140) to identify the uniformity of the entire projection surface. In this case, when the processor (150) performs image correction according to the correction weights of the uneven area, it can identify that unintended image distortion occurs in the surrounding area of ​​the uneven area or that image distortion occurs at a specific location of the image.

[0115] The processor (150) can determine whether the identified uniformity is within a preset reference range, and can change the correction weight for each area based on the determination result to re-perform the correction. The processor (150) can repeat the above-described operation multiple times to set the correction weight at the state where the degree of unevenness is the lowest as the final value, and can continuously apply the correction weight to correct the image.

[0116] When the correction weight is determined for the projection surface, the processor (150) can match information about the projection surface and information about the correction weight for each area and store them in the memory (110). Accordingly, when a situation arises in which an image must be projected again on the same projection surface later, the processor (150) can immediately obtain the correction weight for each area stored in the memory (110) and correct the image. Information about the projection surface may include the position of the projection surface within the entire space where the projector (100) is located, the position of the projector (100), the size of the image, etc. When the position and projection direction of the projector (100) are identified using a lidar sensor or the like provided in the projector (100), the processor (150) can determine whether the projection surface is a projection surface on which an image has been previously projected based on the information stored in the memory (110). If the processor (150) has the projection history, it can obtain information on the correction weight for each area corresponding to the projection surface or the location of the non-uniform area and use it for correction.

[0117] Meanwhile, if the processor (150) identifies a state in which the left and right heights of the image are different based on the sensing value of the image sensor (140), it may control the projection unit (120) to perform keystone correction as described above, or control the projection unit (120) to project the screen in an area with high uniformity.

[0118] The processor (150) can identify the state of the projection surface based on the sensing result of at least one of the distance sensor (130), the image sensor (140), and the direction detection sensor (160), and can identify the number of planes included in the projection surface. For example, the processor (150) can generate coordinate information based on a reference pattern image projected on the projection surface, and control the distance sensor (130) according to the generated coordinate information to sense the distance to the projection surface for each area of ​​the projection surface or each coordinate point.

[0119] The processor (150) can identify a corner axis where distance values ​​for each coordinate point are symmetrical based on the sensing result of the distance sensor (140), and can identify the number of planes included in the projection surface according to the number of corner axes. For example, if there is one corner axis, the processor (150) can identify the number of planes included in the projection surface as two, and if there are three corner axes, the processor (150) can identify the number of planes included in the projection surface as three.

[0120] Alternatively, the processor (150) may identify the number of planes included in the projection surface based on the sensing result of the image sensor (140). The processor (150) may generate coordinate information based on a reference pattern image projected on the projection surface, and identify the number of planes by detecting corner points in an image of the projection surface captured according to the generated coordinate information. For example, if the number of corner points in the captured image is 4, the processor (150) may identify the number of planes included in the projection surface as 1, and if the number of corner points is 6, the processor (150) may identify the number of planes included in the projection surface as 2. In addition, if the number of corner points is 7, the processor (150) may identify the number of planes included in the projection surface as 3.

[0121] When the number of planes included in the projection surface is identified as plural, the processor (150) can control the projection unit (120) to apply a correction weight to each area of ​​the projection surface including a connection area where the plurality of planes are connected to each other to correct and project the corrected image onto the projection surface. For example, the processor (150) can set a correction weight based on an area including a corner axis where planes are connected to each other on the projection surface, and then set the correction weight of the surrounding area to sequentially decrease. In this case, when an uneven area exists in at least one plane among the plurality of planes, the image can be corrected by applying the correction weight set for the uneven area.

[0122] When the number of planes included in the projection surface is identified as multiple, the processor (150) can control the projection unit (120) to perform keystone correction on an image to be projected onto the projection surface according to the identified state of the projection surface. In addition, the processor (150) can control the projection unit (120) to correct an image on which keystone correction has been performed by applying a correction weight for each area, and to project the corrected image onto the projection surface.

[0123] In the above-described section, it has been described that the processor (150) controls the projection unit (120) to first perform keystone correction on the image, and then corrects the image on which keystone correction has been performed by applying correction weights for each area. However, the order is not necessarily limited, and the two operations may be reversed, or the two operations may be performed in parallel.

[0124] As described above, in the prior art, since only keystone correction is performed without considering the number of planes included in the projection surface or the presence of uneven areas within the projection surface, the screen distortion phenomenon may not be resolved. According to the present disclosure, since at least one of keystone correction and uneven area correction is performed on an image to be projected onto the projection surface according to the identified state of the projection surface, such screen distortion phenomenon can be resolved.

[0125] The processor (150) can control the projection unit (120) to project an image on one of the plurality of planes when the number of planes included in the projection surface is identified as plural and a correction weight set for a connection area where the plurality of planes are connected to each other exceeds a settable threshold. The threshold represents a limit value of the correction weight that allows the processor (150) to control the projection unit (120) to correct an uneven area of ​​the image.

[0126] When the correction weight set for the connection area where multiple planes are connected to each other exceeds a settable threshold, the processor (150) can adjust at least one of the image projection angle, focal length, and projection distance of the projection unit (120) to reduce the size of the image projected on the projection surface or change the projection angle.

[0127] When the number of planes included in the projection surface is identified as plural and the correction weight for the non-uniform area in at least one of the plurality of planes is identified as being greater than a threshold, the processor (150) may control the projection unit (120) so that an image is projected on an area excluding the area greater than the threshold.

[0128] Specifically, when an uneven area exceeding a threshold value is identified among a plurality of planes included in the projection surface, the processor (150) may control the projection unit (120) to project an image on at least one plane among the remaining planes, excluding the plane including the area exceeding the threshold value. The processor (150) may reduce the size of the image projected on the projection surface, or control the projection unit (120) to project an image by selecting any one of the remaining planes, excluding the plane identified as exceeding the threshold value. For example, the processor (150) may control the projection unit (120) to project an image by selecting the largest plane among the remaining planes, excluding the plane identified as exceeding the threshold value.

[0129] Alternatively, the processor (150) may adjust at least one of the image projection angle, focal length, and projection distance of the projection unit (150) so that an image is projected on at least one plane except for a plane identified as being greater than a threshold value.

[0130] Figures 12 to 17 are flowcharts for explaining a method of controlling a projector according to various embodiments.

[0131] According to Fig. 12, the projector projects a reference pattern image onto the projection surface (S1210). The projector can store data regarding the reference pattern image in memory. The projector can sense the distance from the projection surface and the size of the projection surface using a distance sensor, and can also adjust the size of the reference pattern image projected onto the projection surface based on the sensing results.

[0132] The projector senses the distance to each area of ​​the projection surface on which the reference pattern image is projected using a distance sensor (S1220). In this case, the projector generates coordinate information based on the reference pattern image projected on the projection surface, and controls the distance sensor according to the generated coordinate information to sense the distance to the projection surface for each area of ​​the projection surface or each coordinate point. Alternatively, if the distance sensor has a function to generate and provide coordinate information on its own, the projector can sense the distance to the projection surface for each area of ​​the projection surface or each coordinate point based on the coordinate information provided by the distance sensor.

[0133] The projector captures an image of the projection surface onto which the reference pattern image is projected using an image sensor (S1230). For example, the projector can set a wide-angle shooting mode that can capture a wide area by adjusting the angle of the camera lens, and capture the entire projection surface onto which the reference pattern image is projected at once. Alternatively, the projector can divide the entire projection surface into screen areas of a preset size and capture images for each divided screen area.

[0134] The projector identifies the state of the projection surface based on the sensing values ​​of the distance sensor and the captured data of the image sensor (S1240). The projector can identify uneven areas based on the sensing values ​​of the distance sensor and the captured data of the image sensor, and compare the identified results to identify uneven areas to be corrected.

[0135] The projector sets and stores correction weights for each area of ​​the projection surface according to the identified state of the projection surface (S1250). For example, the projector can measure the degree to which the reference pattern image is deformed in an uneven area based on the captured data of the image sensor, and set correction weights for each area according to the degree of deformation of the uneven area. Alternatively, the projector can set correction weights for each area according to the difference in distance for an uneven area based on the distance from the projection surface to a uniform area on the projection surface based on the sensed value of the distance sensor. The projector can also combine correction weights extracted based on the sensed value of the distance sensor and correction weights extracted based on the captured data of the image sensor, and set correction weights for each area of ​​the projection surface using the combined correction weights.

[0136] The projector applies correction weights for each area to the image to be projected onto the projection surface to correct (S1260) and projects the corrected image onto the projection surface (S1270). Specifically, the projector applies correction weights for each area of ​​the projection surface set based on the reference pattern image to the projection image to be projected onto the projection surface to correct and projects the corrected image onto the projection surface.

[0137] According to FIG. 13, the projector can identify the state of the projection surface by comparing the unevenness of the projection surface sensed by the distance sensor with the unevenness of each area within the captured image captured by the image sensor (S1310). For example, the projector can identify a first unevenness area of ​​the projection surface based on the sensing value of the distance sensor, and can identify a second unevenness area based on the captured image data of the image sensor. The projector can identify the state of the projection surface by comparing the first unevenness area and the second unevenness area.

[0138] The projector may also identify an uneven area within the projection surface based on the comparison results (S1320). For example, based on the comparison results between the first uneven area and the second uneven area, the common area of ​​the first uneven area and the second uneven area, or both areas, may be determined as an uneven area to be corrected on the projection surface.

[0139] According to FIG. 14, the projector can set and store different correction weights for the identified non-uniform region and the surrounding region (S1410). For example, the projector can set a correction weight for the surrounding region of the non-uniform region that is relatively smaller than the correction weight set for the identified non-uniform region.

[0140] This allows the projector to prevent unintended image distortion in surrounding areas when performing image correction for non-uniform areas.

[0141] According to FIG. 15, the projector can capture a projection surface onto which a corrected image is projected (S1510). For example, the projector can project a reference pattern image to which correction weights are applied for each area of ​​the projection surface onto the projection surface, and capture the reference pattern image projected onto the projection surface using an image sensor.

[0142] The projector can compare the captured image with a reference pattern image to determine whether the image requires recalibration (S1520). The projector can compare the captured image with the reference pattern image stored in memory against the corrected reference pattern image, and perform recalibration for uneven areas based on the comparison results.

[0143] According to Fig. 16, the projector can capture the entire projection surface onto which the corrected image is projected (S1610). In this case, the projector uses an image sensor to capture the entire projection surface, including not only the non-uniform area but also the surrounding area of ​​the non-uniform area, on which the corrected reference pattern image is projected.

[0144] The projector can identify the uniformity of the entire projection surface by comparing the captured image with the reference pattern image (S1620). The projector can identify the uniformity of the entire projection surface by comparing the captured image of the entire projection surface with the reference pattern image stored in the memory. The projector can determine whether the identified uniformity is within a preset reference range (S1630).

[0145] If the identified uniformity for the entire projection surface is within a preset reference range, the projector determines and stores the correction weights set for each area of ​​the projection surface (S1640). However, if the identified uniformity for the entire projection surface is above a preset reference range, the projector can change the correction weights to recalibrate the image (S1650).

[0146] According to FIG. 17, the projector can select one of a plurality of reference pattern images at preset intervals and correct the selected reference pattern image using a correction weight (S1710). In this case, the projector can store data for the plurality of reference pattern images in memory. The plurality of reference pattern images can include reference pattern images of various shapes so that uneven areas of the projection surface can be inspected according to various angles, shapes, sizes, etc.

[0147] The projector can project a corrected reference pattern image onto a projection surface and capture the projection surface onto which the corrected reference pattern image is projected (S1720). For example, the projector can project multiple reference pattern images, to which correction weights are applied for each area of ​​the projection surface, onto the projection surface according to a preset cycle and capture the reference pattern image projected onto the projection surface using an image sensor.

[0148] The projector can determine whether to recalibrate the image by comparing the captured image with the selected reference pattern image (S1730). The projector can periodically compare the captured image with the reference pattern image stored in the memory for the corrected reference pattern image, and perform recalibration for the non-uniform area based on the comparison result. For example, the projector can set the final correction weight as the maximum value or the average value of the correction weights obtained through comparison of multiple corrected reference pattern images with the reference pattern image stored in the memory, and apply the set final correction weight to the image to be projected on the projection surface to correct it.

[0149] According to FIG. 18, the projector can identify the projection direction and tilt status of the projector using a direction detection sensor (S1810). In this case, the direction detection sensor is configured to sense the projection direction of an image projected from the projector, and may include at least one of a gyro sensor, a geomagnetic sensor, a tilt sensor, and a compass.

[0150] The projector can adjust at least one of the projection angle, focal length, and projection distance of the projector based on the identified projection direction and tilt status (S1820). The projector can perform these operations based on the sensing value of the direction detection sensor, but if it performs these operations in conjunction with the sensing value of the distance sensor, the projection angle, focal length, and projection distance of the image with respect to the projection surface can be adjusted more accurately.

[0151] Additionally, the projector can increase the uniformity of the corrected image by adjusting the projector image based on the projection direction and tilt status of the projector before performing correction for each area of ​​the projection surface.

[0152] According to FIG. 19, the projector can identify the state of the projection surface based on at least one of the sensing value of the distance sensor, the image capture data of the image sensor, the projection direction and the tilt state of the projector (S1910).

[0153] The projector can identify the number of planes included in the projection surface based on the identified state of the projection surface (S1920). For example, the projector can identify a reference line along which the direction of increase / decrease of distance values ​​changes based on the sensing values ​​of the distance sensor as a corner axis, and can identify the number of planes included in the projection surface based on the number of corner axes. If there is one corner axis, the projector can identify the number of planes included in the projection surface as two, and if there are three corner axes, the projector can identify the number of planes included in the projection surface as three.

[0154] The projector determines whether the number of planes included in the projection surface is plural (S1930), and if the number of planes included in the projection surface is identified as one plane rather than plural, the projector can correct the image to be projected onto the projection surface by applying a correction weight for each area (S1940).

[0155] When the number of planes included in the projection surface is identified as plural, the projector can correct by applying a correction weight to each area of ​​the projection surface including a connection area where the plurality of planes are connected to each other based on the identified state of the projection surface, and project the corrected image onto the projection surface (S1950). For example, the projector can correct by applying a correction weight to a connection area including a corner axis where walls are connected to each other, and can correct the image by setting a correction weight for a surrounding area of ​​the connection area to be different from the connection area. In this case, if an uneven area exists within at least one of the planes included in the projection surface, the projector can correct by applying a correction weight to the uneven area based on the identified state of the projection surface.

[0156] A projector can also adjust the left and right heights of images projected onto multiple planes by performing keystone correction on the image to be projected onto the projection surface. The projector can correct the image by applying correction weights for each area to the image to which keystone correction has been performed, and project the corrected image onto the projection surface.

[0157] According to FIG. 20, the projector identifies whether the number of planes included in the projection surface is plural (S2010), and if the number of planes included in the projection surface is identified as plural, the projector can set a correction weight for each area of ​​the projection surface including a connection area where the plurality of planes are connected to each other based on the identified state of the projection surface (S2020).

[0158] The projector can set and store in memory correction weights for each area for a connection area where a plurality of planes are connected to each other and for plane areas around the connection area based on the identified state of the projection surface.

[0159] When the number of planes included in the projection surface is identified as one plane, the projector applies a correction weight for each area to correct the image to be projected onto the projection surface (S2050).

[0160] The projector can identify whether a correction weight set for a connection area where multiple planes are connected to each other exceeds a settable threshold (S2030). If the correction weight set for the connection area exceeds the threshold based on the identification result, the projector can adjust at least one of the projection angle, focal length, and projection distance of the projector so that an image is projected on one of the multiple planes (S2040).

[0161] If the correction weight set for the connection area based on the identification result is below a threshold value, the projector can correct the image to be projected onto the projection surface by applying the correction weight for each area (S2050).

[0162] The projector may identify whether there is an area where a correction weight set on at least one of a plurality of planes is greater than or equal to a threshold value. If an area where a correction weight is greater than or equal to the threshold value is identified on at least one plane, the projector may adjust at least one of an image projection angle, a focal length, and a projection distance of the projector so that an image is projected on at least one plane excluding the plane identified as being greater than or equal to the threshold value. For example, the projector may reduce the size of an image projected on a projection surface so that an image is projected on the remaining planes excluding the planes identified as being greater than or equal to the threshold value. Alternatively, the projector may adjust an image projection angle so that an image is projected on any one of the remaining planes excluding the planes identified as being greater than or equal to the threshold value.

[0163] If the projector does not identify an area on the projection surface that is greater than a threshold value, it can perform correction by applying a correction weight for each area to the image to be projected onto the projection surface (S2050).

[0164] Meanwhile, according to an embodiment of the present disclosure, the various embodiments described above may be implemented as software including commands stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The machine, which is a device capable of calling commands stored from the storage medium and operating according to the called commands, may include a projector according to the disclosed embodiments. When the commands are executed by the processor, the processor may directly or under the control of the processor perform a function corresponding to the commands using other components. The commands may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain signals and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0165] Furthermore, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0166] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the corresponding components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0167] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In the projector, memory; Projection section; A distance sensor for sensing the distance to the projection surface; An image sensor for capturing the projection surface; and Processor; including; The above processor, Based on the sensing results of the distance sensor and the image sensor, the state of the projection surface is identified, and correction weights for each area are set according to the identified state and stored in the memory. A projector that applies correction weights for each area to an image to be projected onto the projection surface, corrects the image, and controls the projection unit to project the corrected image onto the projection surface.

2. In paragraph 1, The above memory is, Stores data for the reference pattern image, The above processor, Controlling the projection unit to project the above reference pattern image onto the projection surface, Controlling the image sensor to capture the projection surface onto which the above reference pattern image is projected, A projector that identifies an uneven area within the projection surface based on a captured image of the image sensor and a sensed value of the distance sensor, and sets different correction weights for the uneven area and the surrounding area and stores them in the memory.

3. In paragraph 2, The above processor, Correcting the reference pattern image using the set correction weight, and controlling the projection unit to project the corrected reference pattern image onto the projection surface, A projector that controls the image sensor to capture the projection surface onto which the corrected reference pattern image is projected, and compares the captured image of the image sensor with the reference pattern image to determine whether the image needs to be re-corrected.

4. In paragraph 3, The above processor, A projector that controls the image sensor to capture the entire projection surface onto which the corrected image is projected, compares the captured image of the image sensor with the reference pattern image to identify the uniformity of the entire projection surface, and determines whether the identified uniformity is within a preset reference range.

5. In paragraph 2, The above memory is, Stores data for multiple reference pattern images, The above processor, A projector for selecting one of the plurality of reference pattern images at preset intervals, correcting the selected reference pattern image using the correction weight, controlling the projection unit to project the corrected reference pattern image onto the projection surface, controlling the image sensor to capture the projection surface onto which the corrected reference pattern image is projected, and comparing the captured image of the image sensor with the selected reference pattern image to determine whether or not to re-correct the image.

6. In paragraph 1, Further comprising a direction detection sensor for sensing the projection direction of the above projection unit; The above processor, A projector that identifies the projection direction and the tilt state of the projector based on the sensing result of the direction detection sensor, and adjusts at least one of the image projection angle, focal length, and projection distance of the projection unit based on the identified state.

7. In paragraph 6, The above processor, A projector, wherein, based on the sensing results of at least one of the distance sensor, the image sensor, and the direction detection sensor, if the number of planes included in the projection surface is identified as plural, a correction weight is applied to each area including a connecting area where the plurality of planes are connected to each other to perform correction, and the projection unit is controlled to project the corrected image onto the projection surface.

8. In paragraph 7, The above processor, A projector that controls the projection unit to project the image on one of the plurality of planes when the correction weight set for the above connection area exceeds a settable threshold.

9. In the method of controlling the projector, A step of projecting a reference pattern image onto a projection surface; A step of sensing the distance of each area of ​​the projection surface onto which the reference pattern image is projected using a distance sensor; A step of capturing an image of a projection surface onto which the reference pattern image is projected using an image sensor; A step of identifying the state of the projection surface based on the sensing value of the distance sensor and the imaging data of the image sensor; A step of setting and storing correction weights for each area of ​​the projection surface according to the identified status; A step of correcting by applying the correction weight for each area to the image to be projected onto the projection surface; and A control method comprising the step of projecting the corrected image onto the projection surface.

10. In paragraph 9, The step of identifying the state of the above projection surface is: A step of comparing the uneven state of the projection surface sensed by the distance sensor with the uneven state of each area within the photographed image captured by the image sensor; and A control method, comprising: a step of identifying an uneven area within the projection surface based on a comparison result; 11. In paragraph 10, The step of setting and saving the correction weights for each area above is: A control method, comprising: a step of differently setting and storing correction weights for the identified non-uniform area and the surrounding area; 12. In paragraph 9, A step of photographing the entire projection surface onto which the corrected image is projected; A step of comparing the captured image with the reference pattern image to identify the uniformity of the entire projection surface; and A control method further comprising a step of determining whether the identified uniformity is within a preset standard range.

13. In paragraph 9, A step of selecting one of the plurality of reference pattern images at preset intervals and correcting the selected reference pattern image using the correction weight; A step of projecting a corrected reference pattern image onto the projection surface and photographing the projection surface onto which the corrected reference pattern image is projected; and A control method further comprising: a step of comparing a captured image with the selected reference pattern image to determine whether to recalibrate the image.

14. In paragraph 10, A step of identifying the projection direction and tilt status of the above projector; A control method further comprising: a step of adjusting at least one of an image projection angle, a focal length, and a projection distance of the projector based on the identified state.

15. In paragraph 14, A step of identifying the state of the projection surface based on at least one of the sensing value of the distance sensor, the imaging data of the image sensor, and the projection direction and inclination state of the projector; A step of identifying the number of planes included in the projection surface based on the identified state of the projection surface; and A control method further comprising: a step of applying a correction weight to each area of ​​the projection surface including a connection area where a plurality of planes are connected to each other, and projecting the corrected image onto the projection surface, when the number of planes included in the projection surface is identified as plural.

Images