Projector and method of controlling the same

Abstract

A projector is disclosed. The projector includes a memory, a projection unit, a distance sensor for sensing a distance to a projection surface, an image sensor for imaging the projection surface, and a processor. The processor identifies a state of the projection surface based on results sensed by the distance sensor and the image sensor, sets a correction weight for each region according to the identified state, stores the correction weight in the memory, applies the correction weight for each region to an image to be projected onto the projection surface to correct the image, and controls the projection unit to project the corrected image onto the projection surface.

Description

Technical Field

[0001] This disclosure relates to a projector and its control method, and more specifically, to a projector and its control method that takes into account a non-uniform projection surface to correct an image and project the corrected image onto the projection surface. Background Technology

[0002] With the development of electronic technology, various types of electronic devices have been developed and popularized. In particular, projectors, which can project images onto screens or walls, are used in various places such as homes, offices, and public spaces, and have continued to develop in recent years.

[0003] Projectors project light from a light source onto a screen or wall using projection lenses. Projectors can also perform corrections such as keystone correction to project the best possible image to the user.

[0004] Projectors project light onto a projection surface to represent an image, and therefore, when the projection surface is non-uniform, the image itself may appear distorted. Traditional projectors have attempted to correct the image itself and project a corrected image. However, when only a portion of the entire projection surface is non-uniform, undesirable screen distortion can occur even when projecting a corrected image. Summary of the Invention

[0005] [Technical Solution]

[0006] According to at least one embodiment of the present disclosure, a projector is provided, the projector comprising: a memory; a projection unit; a distance sensor configured to sense a distance to a projection surface; an image sensor configured to capture an image of the projection surface; and a processor configured to: identify a projection surface state based on sensing results obtained from the distance sensor and the image sensor; set a correction weight for each region based on the identified state and store the correction weight in the memory; correct the image by applying the correction weight for each region to an image to be projected onto the projection surface; and control the projection unit to project the corrected image onto the projection surface.

[0007] According to at least one embodiment of this disclosure, a control method for a projector is provided. The method includes: projecting a reference pattern image onto a projection surface; sensing distances to each region of the projection surface on which the reference pattern image is projected using a distance sensor; capturing an image of the projection surface on which the reference pattern image is projected using an image sensor; identifying a projection surface state based on sensing values ​​obtained from the distance sensor and image capture data obtained from the image sensor; setting and storing correction weights for each region of the projection surface based on the identified state; correcting the image by applying the correction weights for each region to an image to be projected onto the projection surface; and projecting the corrected image onto the projection surface.

[0008] According to at least one embodiment of the present disclosure, a computer-readable recording medium is provided, the computer-readable recording medium including a program for performing a control method for a projector, the control method including: projecting a reference pattern image onto a projection surface; sensing a distance from each region of the projection surface on which the reference pattern image is projected using a distance sensor; capturing an image of the projection surface on which the reference pattern image is projected using an image sensor; identifying a projection surface state based on sensing values ​​obtained from the distance sensor and image capture data obtained from the image sensor; setting and storing correction weights for each region of the projection surface based on the identified state; correcting the image by applying the correction weights for each region to an image to be projected onto the projection surface; and projecting the corrected image onto the projection surface. Attached Figure Description

[0009] Figure 1 This is a block diagram illustrating the configuration of a projector according to various embodiments of the present disclosure.

[0010] Figure 2 This is a block diagram illustrating examples of detailed configurations of a projector according to various embodiments.

[0011] Figure 3 and Figure 4 The accompanying drawings illustrate the identification of non-uniform regions based on reference pattern images according to various embodiments.

[0012] Figures 5 to 8 The accompanying drawings illustrate the setting of correction weights for non-uniform regions according to various embodiments.

[0013] Figures 9 to 11 The accompanying figure illustrates a method for local correction of the peripheral region of a non-uniform area.

[0014] Figures 12 to 20 This is a flowchart illustrating a control method for a projector according to various embodiments. Detailed Implementation

[0015] The terminology used in this specification will be briefly described before this disclosure is described in detail.

[0016] Currently widely used general terms have been selected as terms used in the embodiments of this disclosure, taking into account their function in this disclosure, and these terms may be changed based on the intent of those skilled in the art, judicial precedent, the emergence of new technologies, etc. Furthermore, in certain circumstances, terms may be arbitrarily chosen by the applicant. The meanings of these terms are detailed in the corresponding descriptions of this disclosure. Therefore, the terms used in this disclosure need to be defined based on their meanings and the content throughout this disclosure, rather than merely on their names.

[0017] In the specification, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of the corresponding feature (e.g., a number, function, operation, or component such as a part) and do not exclude the presence of additional features.

[0018] In this disclosure, expressions such as “A or B”, “at least one of A and / or B”, or “one or more of A and / or B” can include all possible combinations of the items listed together. For example, “A or B”, “at least one of A and B”, or “at least one of A or B” can mean all of the following: 1) including at least one A, 2) including at least one B, or 3) including both at least one A and at least one B.

[0019] The terms "first" and "second" used in this disclosure may refer to various components, regardless of their order and / or importance. These terms are used only to distinguish one component from another and do not limit the corresponding components.

[0020] When it is mentioned that any component (e.g., the first component) is “(operably or communicatively) coupled to” or “connected to” another component (e.g., the second component), it should be understood that any component may be directly coupled to that other component, or may be coupled to that other component through yet another component (e.g., the third component).

[0021] Unless the context clearly indicates otherwise, singular terms may include their plural forms. It should be understood that the terms "comprising" or "formed by" as used in this application indicate the presence of features, numbers, steps, operations, components, parts or combinations thereof mentioned in the specification, and do not exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0022] In this disclosure, a "module" or "device" can perform at least one function or operation and can be implemented by hardware, software, or a combination of hardware and software. Furthermore, in addition to "modules" or "devices" that require specific hardware implementation, multiple "modules" or multiple "devices" can be integrated into at least one module and implemented by at least one processor (not shown).

[0023] In the following, embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings.

[0024] Figure 1 This is a block diagram illustrating the configuration of a projector according to various embodiments of the present disclosure.

[0025] Projector 100 refers to an electronic device that displays images by projecting light onto various projection surfaces (e.g., walls, screens, floors, or ceilings). Projector 100 can be implemented in various forms, such as a mobile projector that includes wheels and is movable, a fixed projector that is fixed to a ceiling or wall, a basic projector placed on the floor, or an aerial projector that can be used while floating in the air like a drone.

[0026] refer to Figure 1 The projector 100 may include a memory 110, a projection unit 120, a distance sensor 130, an image sensor 140, and a processor 150.

[0027] The memory 110 can store at least one instruction, data, program, etc., required for the operation of the projector 100. For example, the memory 110 can store data related to a reference pattern image. The reference pattern image may include an image used to check the uniformity of the projection surface. The reference pattern image can be included in various forms. For example, the reference pattern image can be implemented in various forms, such as a grid pattern including multiple grids, or a matrix pattern in which multiple points, lines, symbols, characters, etc., are arranged like a matrix. The memory 110 can store various data related to the size, shape, color, display position, etc., of the reference pattern image. The memory 110 can be implemented as a memory embedded in the projector 100, or as a memory detachably attached to the projector 100 for data storage purposes. For example, data for driving the projector 100 can be stored in the embedded memory of the projector 100, and data for the extended functions of the projector 100 can be stored in a memory detachably attached to the projector 100.

[0028] The embedded memory of the projector 100 can be implemented as at least one of the following: volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM) or synchronous dynamic random access memory (SDRAM)) or non-volatile memory (e.g., one-time programmable read-only memory (OTPROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard disk drive or solid-state drive (SSD)).

[0029] The memory 110 may be implemented as a single memory storing data generated in various operations according to the present disclosure, but is not limited thereto. The memory 110 may be implemented as including multiple memories, each storing different types of data or data generated at different stages.

[0030] The projection unit 120 is a component used to represent an image by projecting light outward.

[0031] The projection unit 120 can be implemented using various projection methods, such as cathode ray tube (CRT), liquid crystal display (LCD), digital light processing (DLP), or laser methods.

[0032] When implemented using the CRT method, the projection unit 120 may include a CRT and a lens. When an image is displayed on the CRT of the projection unit 120, light emitted from the CRT can be amplified and projected outward through the lens. Based on the number of CRTs, the CRT method can be divided into single-tube and three-tube types, and in the three-tube type, a red CRT, a green CRT, and a blue CRT can be implemented separately.

[0033] When implemented using the LCD method, the projection unit 120 may include a light source, a liquid crystal display (LCD), other lenses, etc. The LCD method refers to a method of displaying an image by transmitting light emitted from the light source through the LCD. LCD methods can be divided into single-panel and three-panel types. In the three-panel type, light emitted from the light source is separated into red, green, and blue by a dichroic mirror (a mirror that reflects only a specific color of light and transmits the rest), transmitted through the LCD, and then focused back into a single point.

[0034] The DLP method refers to a method of displaying images using a digital micromirror device (DMD) chip. A projection unit 120 using the DLP method may include a light source, a color wheel, a DMD chip, a projection lens, etc. Light output from the light source can be colored as it passes through a rotating color wheel. The light passing through the color wheel can be input into the DMD chip. The DMD chip may include a number of micromirrors. The DMD chip can reflect the input light. The projection lens can perform the function of magnifying the light reflected from the DMD chip to the image size.

[0035] In another example, the projection unit 120 using a laser method may include a diode-pumped solid-state (DPSS) laser and a galvanometer. To output various colors, a DPSS laser can be provided for each RGB color. The galvanometer can reflect the laser by using a motor to rotate a mirror at high speed. For example, the galvanometer can cause the mirror to rotate at a maximum of 40 kHz / second.

[0036] The projection unit 120 may include various types of light sources. For example, the projection unit 120 may include at least one of a lamp, a light-emitting diode (LED), or a laser.

[0037] The projection unit 120 can output images in a 4:3 aspect ratio, a 5:4 aspect ratio, or a wider aspect ratio of 16:9, depending on the purpose of the projector 100 or user settings. It can also output images with any resolution based on the aspect ratio, such as Wide Video Graphics Array (WVGA, 854×480 pixels), Super Video Graphics Array (SVGA, 800×600 pixels), Extended Graphics Array (XGA, 1024×768 pixels), Wide Extended Graphics Array (WXGA, 1280×720 pixels or 1280×800 pixels), Super Extended Graphics Array (SXGA, 1280×1024 pixels), Extreme Extended Graphics Array (UXGA, 1600×1200 pixels), and Full HD (1920×1080 pixels).

[0038] The projection unit 120 can perform various functions to adjust the output image under the control of the processor 150. For example, the projection unit 120 can perform zoom function, keystone correction function, fast corner (four corners) correction function, lens shift function, etc.

[0039] Specifically, the projection unit 120 can perform zoom functions to enlarge or reduce the image based on the distance to the screen (projection distance).

[0040] Methods for performing zoom functions can be divided into hardware methods that adjust the screen size by moving the lens and software methods that adjust the screen size by cropping the image. When performing zoom, the image needs to be focused. Focus adjustment can be performed through manual focusing methods, electronic methods, etc. Manual focusing refers to manually adjusting the focus. Electronic focusing refers to automatically adjusting the focus using a motor. The projector 100 can selectively provide digital zoom or optical zoom functions.

[0041] Projection unit 120 can perform keystone correction. During front projection, if the projection height is mismatched, the image may be distorted upwards or downwards. Keystone correction corrects this distortion. For example, horizontal keystone correction corrects horizontal distortion, and vertical keystone correction corrects vertical distortion. Quick corner correction corrects the image when the center area is normal but the corner areas are unbalanced. Lens shift corrects the image position without distortion when it extends beyond the projection area.

[0042] Even without user intervention, the projection unit 120 can automatically provide zoom, keystone correction, and focus functions by analyzing the surrounding environment and the projection environment. Specifically, the projection unit 120 can automatically provide zoom, keystone correction, and focus functions based on information sensed by sensors (e.g., depth camera, distance sensor, infrared sensor, or illuminance sensor) (e.g., the distance between the projector 100 and the screen, information about the space where the projector 100 is currently located, information about the amount of ambient light, etc.).

[0043] Additionally, the projection unit 120 can provide illumination by using a light source. Specifically, the projection unit 120 can output light by using a light source (e.g., one or more LEDs).

[0044] In this embodiment, the projection unit 120 can output a light source using a surface-emitting LED. A surface-emitting LED is an LED having an optical plate disposed above it to uniformly distribute and output light.

[0045] The projection unit 120 can provide a user with a dimming function to adjust the intensity of light emitted from the light source. Specifically, when a user inputs a command to adjust the intensity of light emitted from the light source via various control interfaces (e.g., touchscreen, buttons, or dial pads), the projection unit 120 can control the LED to output light at an intensity corresponding to the user command. Alternatively, the projection unit 120 can provide the dimming function based on content analyzed by the processor 150 without user input. Specifically, the projection unit 120 can control the intensity of light emitted from the light source by the LED output based on information about the currently provided content (e.g., content type or content brightness).

[0046] The color temperature of the projection unit 120 can be controlled by the processor 150.

[0047] Distance sensor 130 is a component for sensing the distance to an external object. Processor 150 can identify the distance to a projection surface or another external object based on the sensing values ​​obtained from distance sensor 130. Distance sensor 130 may include at least one of an ultrasonic sensor, an infrared sensor, a laser sensor, an optical distance sensor, a radar (RADAR) sensor, a laser detection and ranging (LIDAR) sensor, a photodiode sensor, or a time-of-flight (ToF) sensor.

[0048] For example, when the distance sensor 130 is implemented as an ultrasonic sensor, the distance sensor 130 may include a transmitter and a receiver. When sound waves are output from the transmitter, the receiver may receive the echo of the output sound waves reflected from the object. The processor 150 may calculate the distance to the object based on the speed of sound (340 m / s) and the time difference between the output time and the reception time of the sound waves.

[0049] When the distance sensor 130 is implemented as an infrared sensor, it may include a light emitter and a light receiver. When infrared light is emitted from the light emitter, the emitted infrared light may collide with an object and be reflected. The light receiver can detect the reflected incident signal. Infrared sensors can use light instead of sound waves, and therefore, the distance to an object can be measured based on optical triangulation by calculating the angle relative to a focal point formed on the object.

[0050] Furthermore, a TOF sensor refers to a sensor used to measure the distance to an object using signals (e.g., near-infrared light, ultrasound, or laser). A TOF sensor may include a transmitter for outputting various signals and a receiver for receiving the corresponding reflected signals. A TOF sensor can measure the distance between a projector 100 (i.e., the device including the sensor) and an object based on the time (time of flight) it takes for a signal emitted from the transmitter to be reflected by the object and return.

[0051] Image sensor 140 is a component used to capture images. Image sensor 140 can capture images of various external objects, including projection surfaces, under the control of processor 150. For example, image sensor 140 included in a camera can convert images captured through the camera lens into digital signals and generate image data based on the converted signals. Image sensor 140 can be classified based on its structure as a complementary metal-oxide-semiconductor (CMOS) image sensor or a charge-coupled device (CCD) image sensor.

[0052] Processor 150 is a component connected to each component of projector 100 to control the overall operation of projector 100. Processor 150 can be implemented as a digital signal processor (DSP), microprocessor, graphics processing unit (GPU), artificial intelligence (AI) processor, neural processing unit (NPU), or time controller (TCON) for processing digital image signals. However, processor 150 is not limited to these and can include or be defined as one or more of a central processing unit (CPU), microcontroller unit (MCU), microprocessor unit (MPU), controller, application processor (AP), communication processor (CP), or advanced reduced instruction set computer (RISC) machine (ARM) processor. Furthermore, processor 150 can be implemented as a system-on-a-chip (SoC) or a large-scale integration (LSI) in which processing algorithms are embedded, or it can be implemented as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0053] Furthermore, the processor 150 for executing the artificial intelligence model according to the embodiments can be implemented based on software and a combination of the following: a general-purpose processor such as a CPU, AP, or digital signal processor (DSP); a graphics-specific processor such as a GPU or visual processing unit (VPU); or an artificial intelligence-specific processor such as an NPU. The processor 150 can control the processing of input data according to predefined operating rules or an artificial intelligence model stored in memory. Alternatively, when the processor 150 is a dedicated processor (or an artificial intelligence-specific processor), the processor 150 can be designed to have a hardware structure dedicated to processing a specific artificial intelligence model. For example, the hardware dedicated to processing a specific artificial intelligence model can be designed as a hardware chip (e.g., an ASIC or an FPGA).

[0054] When processor 150 is implemented as a dedicated processor, processor 150 may be implemented to include memory for implementing embodiments of the present disclosure, or to include memory processing functions for using external memory. Processor 150 may be implemented as one or more processors.

[0055] When an image is projected onto a projection surface using projector 100 and non-uniform areas exist within the projection surface, keystone correction may not be able to resolve image distortion. Specifically, ultra-short throw (UST) projectors can project images over short distances, therefore, this image distortion can significantly affect the display of the image on the projection screen.

[0056] The processor 150 can identify the state of the projection surface based on sensing results obtained from the distance sensor 130 and the image sensor 140. The processor 150 can activate the distance sensor 130 to identify the distance to the projection surface. Additionally, the processor 150 can activate the image sensor 140 to obtain a captured image of the projection surface. In this disclosure, activation can refer to turning on the sensor or controlling the sensor to perform its function in the on state.

[0057] Specifically, the processor 150 can compare the sensing results obtained from the distance sensor 130 with the sensing results obtained from the image sensor 140, and can identify non-uniform regions included in the projection surface based on the comparison results. In this disclosure, a non-uniform region refers to a region in which the uniformity of the surface state is equal to or less than a predetermined range. For example, a region including at least one protruding region or recessed region, in which it is difficult to display a flat image, can be identified as a non-uniform region.

[0058] For example, processor 150 can control image sensor 140, which is capable of wide-area sensing, to first capture an image of the entire projection surface, identify non-uniform regions included in the projection surface based on the image capture results, and control distance sensor 130 to sense the distance from the projection surface corresponding to the identified non-uniform regions.

[0059] The processor 150 can set correction weights for each region based on the identified projection surface state and store these correction weights in the memory 110. The correction weight for each region refers to the set of weights used to apply a correction level individually to each region when the entire area of ​​the image to be projected onto the projection surface is divided into multiple regions. The method for setting correction weights for each region of the projection surface is further described below.

[0060] The processor 150 can set different correction weights for each region corresponding to the image to be projected onto the projection surface and store the correction weights in the memory 110. When the image is projected onto the projection surface, the processor 150 can control the projection unit 120 to apply the correction weights stored in the memory 110 to correct the image and project the corrected image. Therefore, the same level of correction is not performed on the entire image, and local correction can be performed on image regions displayed in non-uniform areas in and around the projection surface.

[0061] exist Figure 1The illustrations and description are provided based on the case where the projector 100 includes basic components for sensing the state of the projection surface and correcting the projected image based on the sensing results. However, the projector 100 may also include additional components for identifying the state of the projection surface and correcting the projected image in various ways.

[0062] Figure 2 This is a block diagram illustrating examples of detailed configurations of a projector according to various embodiments.

[0063] refer to Figure 2 In addition to the memory 110, projection unit 120, distance sensor 130, image sensor 140, and processor 150, the projector 100 may also include an orientation detection sensor 160, a communication interface 171, a control interface 172, an input / output interface 173, a display 174, a speaker 175, a microphone 176, a power unit 177, etc. However, this disclosure is not limited thereto; the projector 100 may also include other components, or some components may be omitted or modified. Figure 2 Of the components shown, memory 110, projection unit 120, distance sensor 130, image sensor 140, and processor 150 are references. Figure 1 The description is omitted because it is redundant.

[0064] The orientation detection sensor 160 is a component used to sense the projection orientation of the projection unit 120. The orientation detection sensor 160 may include at least one of a gyroscope sensor, a geomagnetic sensor, a tilt sensor, or a compass.

[0065] The processor 150 can identify the image projection direction and tilt state of the projector 100 based on the sensing results obtained from the orientation detection sensor 160, and can adjust at least one of the image projection angle, focal length, and projection distance of the projection unit 120 based on the identified state.

[0066] Communication interface 171 is a component for performing communication with at least one external device. Communication interface 171 may include at least one wireless communication module or at least one wired communication module. Each communication module may be implemented as at least one hardware chip. For example, the wireless communication module may include at least one of a Wi-Fi module, a Bluetooth module, an infrared communication module, or other communication modules. Furthermore, communication interface 171 may include at least one communication chip for performing communication according to various wireless communication standards, such as Zigbee, 3G, 3GPP, LTE, LTE-A, 4G, and 5G.

[0067] For example, the wired communication module may include at least one of a local area network (LAN) module, an Ethernet module, a pairing cable, a coaxial cable, a fiber optic cable, or an ultra-wideband (UWB) module. The projector 100 can perform various operations on content data provided by an external source (e.g., a web server, broadcasting station, or multimedia playback device) connected to it via the communication interface 171, such as demultiplexing, deinterleaving, decoding, scaling, and filtering, to configure an image, and can project the configured image onto a projection surface via the projection unit 120.

[0068] The control interface 172 is a component used to receive user commands. The control interface 172 may include various buttons, touch screens, etc., included in the main body of the projector 100.

[0069] Input / output interface 173 is a component for inputting and outputting various external signals. Input / output interface 173 can receive at least one of audio or image signals from an external device and can output control commands to an external device. Input / output interface 173 can be implemented as at least one of the following wired input / output interfaces: High Definition Multimedia Interface (HDMI), Mobile High Definition Link (MHL), Universal Serial Bus (USB), Universal Serial Bus Type-C interface (USB Type-C), DisplayPort (DP), Thunderbolt, Video Graphics Array (VGA) port, Red Green Blue (RGB) port, D-SUB, and Digital Vision Interface (DVI). Although... Figure 2 The input / output interface 173 and the communication interface 171 are shown as separate components, but when communicating with an external device via the input / output interface 173, the input / output interface 173 can be considered as a component included in the communication interface 171.

[0070] Display 174 is a component used to display the operating status of projector 100, notification messages, user interface (UI) screens, etc. Display 174 can be implemented as various types of displays, such as liquid crystal displays (LCDs) and organic light-emitting diode (OLED) displays. Alternatively, display 174 can be implemented using only one light-emitting element or multiple light-emitting elements. Processor 150 can change the display state of display 174 based on various states, including the power-on state of projector 100, the normal operating state of projector 100, or a low power state or error occurrence state, thereby allowing the user to intuitively identify the state of projector 100.

[0071] Speaker 175 is a component used for outputting audio signals. Although in Figure 2Only speaker 175 is shown, but projector 100 may also include various components for processing audio signals, such as audio decoders, audio output mixers, audio signal processors, and amplification circuitry. Speaker 175 may include one or more speakers, and when implemented as multiple speakers, these speakers may be symmetrically arranged on the exterior of the body to output audio signals in all directions (i.e., 360 degrees).

[0072] Microphone 176 is a component used to receive various audio signals. Microphone 176 can receive user voice or other sounds and convert the received sound into audio data. Microphone 176 can be integrally formed on the upper, front, or side of projector 100. Projector 100 may include various components, such as amplifier circuitry for amplifying audio signals received through microphone 176, analog-to-digital (A / D) converter circuitry for sampling the amplified audio signals and converting the sampled audio signals into digital signals, and filter circuitry for removing noise components from the converted digital signals.

[0073] The power unit 177 can receive power from an external source and supply power to the various components of the projector 100. The power unit 177 can receive power in various ways. For example, when a charging station (not shown) is set up in the environment where the projector 100 operates, the power unit 177 can receive power through the charging station. Alternatively, the power unit 177 can receive power from an external device or power source through various wired or wireless charging methods. Alternatively, the power unit 177 can include a battery. When connected to external power, the power unit 177 can charge the battery and use the battery's power by using an electrical signal applied from the external power source. Alternatively, the power unit 177 can be implemented as including a replaceable battery installed therein.

[0074] Despite Figure 2 Not shown, but when the projector 100 is implemented as a mobile device, the projector 100 may also include at least one motor, multiple wheels, and at least one shaft and gear for transmitting the force of the motor to the multiple wheels.

[0075] As described above, the projector 100 can include various components depending on its type. Therefore, the projector 100 can project images onto various types of projection surfaces, and these projection surfaces can include those containing partially non-uniform regions, such as walls, ceilings, or floors. When the processor 150 determines that the projection surface includes a non-uniform region, the processor 150 can identify the region and perform local correction on the image as described above. Hereinafter, an example of a method for identifying non-uniform regions is described in detail.

[0076] Figure 3 and Figure 4 The accompanying drawings illustrate the identification of non-uniform regions based on reference pattern images according to various embodiments. Figure 3 This illustrates the case where a reference pattern image is projected onto the projection surface in a uniform manner across the entire projection surface. Figure 3 The reference pattern image in the image indicates a grid pattern in which lines or dots are arranged regularly.

[0077] The processor 150 can acquire data related to the reference pattern image stored in the memory 110, and can control the projection unit 120 to project the image corresponding to the data onto the projection surface. Additionally, the processor 150 can control the image sensor 140 to capture an image of the projection surface on which the reference pattern image is projected.

[0078] Figure 3 This illustrates the case where a reference pattern image with a grid pattern is projected onto a projection surface without any non-uniform regions. In this case, no grid curvature appears in the reference pattern image projected onto the projection surface, and the distance between each line remains at a predetermined interval.

[0079] Figure 4 This illustrates the case where the outer region of the projection surface is uniform, while non-uniform regions exist within the projection surface. For example... Figure 4 As shown, it can be confirmed that in the reference pattern image projected onto the projection surface, the pattern in the edge region maintains the predetermined spacing between lines, while the grid in the inner region exhibits curvature, and the spacing between the lines is also distorted. When the projection surface includes a non-uniform surface, or when decorations, insects, etc., are attached to the projection surface, only a portion of the projection surface (rather than the entire projection surface) may be non-uniform. In this case, when projecting the reference pattern image, some non-uniform areas can be identified, such as... Figure 4 As shown.

[0080] Processor 150 can identify non-uniform regions within the projection surface based on captured images obtained from image sensor 140 and sensed values ​​obtained from distance sensor 130. Specifically, processor 150 can identify non-uniform regions within the projection surface by capturing a reference pattern image projected onto the projection surface and based on the sensing results obtained from image sensor 140. Processor 150 can compare the reference pattern image stored in memory 110 with the reference pattern image of the projection surface captured by image sensor 140, and can identify non-uniform regions within the projection surface based on the comparison result.

[0081] Next, the processor 150 can sense the distance between the projector 100 and the projection surface using the distance sensor 130, and can identify non-uniform regions within the projection surface based on the sensing results. In this case, the processor 150 can generate coordinate information based on a reference pattern image projected onto the projection surface, and can control the distance sensor 130 based on the generated coordinate information to sense the distance to the projection surface for each region or coordinate point. For example, when the projector 100 is placed on the floor and the projection surface is vertically erected at a predetermined distance from it, the distance to the upper edge of the reference pattern image may be measured as greater than the distance to its lower edge, based on the arrangement of the distance sensor 130 included in the projector 100. That is, the distance may increase upwards at a predetermined ratio based on the height of the projector 100 and the height of the reference pattern image.

[0082] However, when a partially non-uniform region exists, the distance to each point within that region can be measured as non-uniform. That is, for recessed areas, the distance can be measured as greater than the distance to adjacent points, while for protruding areas, the distance can be measured as less than the distance to adjacent points. Therefore, when the distances between points of the same height in the reference pattern image are measured as different, the processor 150 can identify the corresponding region as a non-uniform region. Furthermore, the processor 150 can identify non-uniform regions by comparing the non-uniformity of the projection surface sensed by the distance sensor 130 with the non-uniformity of each region in the captured image obtained from the image sensor 140. For example, the processor 150 can compare a first non-uniform region identified using the distance sensor 130 with a second non-uniform region identified using the image sensor 140, and based on the comparison result, determine the common area where the first and second non-uniform regions overlap as the target non-uniform region for correction of the projection surface.

[0083] Alternatively, the processor 150 can aggregate the first non-uniform region and the second non-uniform region based on the comparison result, and can determine the non-uniform region obtained by aggregating the two regions as the correction target non-uniform region of the projection surface. In the above description, it is described that image capture data obtained from the image sensor is examined first, and then the sensed values ​​obtained from the distance sensor are examined. However, the order is not limited; the order of the two operations can be reversed, or the two operations can be performed in parallel.

[0084] When a non-uniform region is identified, the processor 150 can perform processing for local correction of the image. Specifically, the processor 150 can set and apply correction weights differently for each region.

[0085] Figures 5 to 8The accompanying drawings illustrate the setting of correction weights for non-uniform regions according to various embodiments. Figure 5 The original image data stored in memory 110 is shown. Figure 6 This illustrates the state of projecting an image onto a projection surface that is in a non-uniform state. For ease of description, in Figure 5 and Figure 6 The circular objects 510, 520, and 530 shown are merely examples, and various objects (e.g., people, buildings, or vehicles) can be included in the actual images.

[0086] Figure 5 The diagram shows the first object 510 having an elongated elliptical shape and nearly a circular shape in the vertical direction. Figure 6 The first object 610 is shown as an object whose upper and lower widths are reduced due to the non-uniformity of the corresponding regions and is therefore represented as a substantially circular object.

[0087] On the contrary, Figure 5 The second object 520 is shown to be nearly circular in shape, while Figure 6 The second object 620 is shown to have increased upper and lower widths and is therefore represented as an elongated elliptical object in the vertical direction. Figure 5 The third object 530 shown is in Figure 6 The middle part is represented as an object 630 with the same size and without any distortion.

[0088] When Figure 5 and Figure 6 During the comparison, it can be confirmed that the area where the third object 530 or 630 is located on the projection plane is in a uniform state, while the areas where the first object 510 or 610 and the second object 520 or 620 are located are in a non-uniform state. Furthermore, it can be understood that... Figure 5 and Figure 6 In the projection, the area where the first object 510 or 610 is located and the area where the second object 520 or 620 is located are at different heights on the projection plane.

[0089] Figure 7 This illustrates a scenario where image correction is performed jointly based on specific non-uniform regions on the projection plane. Figure 7 This demonstrates how the same correction weights are applied to locations Figure 6 The first object 610 and the second object 620 in the non-uniform region are used to perform the correction state. (Reference) Figure 7 , Figure 6 The second object 620 shown is corrected to have a corrected value by applying correction weights to it, and is displayed in a state 720 where the second object 620 is corrected to have a shape substantially the same as the second object 520 in the original image. However, relative to Figure 6 As shown in the diagram of the first object 610 before correction, it can be confirmed that the correction weight was over-applied to the first object 610, and after correction, the first object 610 is displayed in a state 710 where the vertical width of the first object 610 is significantly increased. Therefore, when the non-uniformity of the projection surface varies for each region, it is necessary to set different correction weights for each identified region.

[0090] The processor 150 can set different correction weights for the identified non-uniform regions and their surrounding regions, store the correction weights in the memory 110, and can correct the image projected onto the projection surface based on the different correction weights set.

[0091] Processor 150 can compare image capture data obtained from image sensor 140 with data related to a reference pattern image stored in memory 110, and can measure the degree of distortion of the reference pattern image in non-uniform regions based on the comparison results. Specifically, processor 150 can calculate the amount of distortion at each coordinate position of the reference pattern image projected onto the projection plane based on the reference pattern image stored in memory 110. Processor 150 can set correction weights for each region corresponding to the non-uniform region based on the amount of distortion calculated for each coordinate position.

[0092] Figure 8 The accompanying figure illustrates how correction is performed by setting different correction weights for each non-uniform region of the projection surface. For example, Figure 8 This illustrates how 80% of the correction weight is applied. Figure 6 The image is a non-uniform region where object 620 is located, and the correction is performed by applying a correction weight of 20% to the non-uniform region where object 610 is located.

[0093] As described above, the processor 150 can set different correction weights based on the amount of distortion at each coordinate position in all non-uniform regions.

[0094] Alternatively, when consecutive non-uniform regions exist among all non-uniform regions, the processor 150 can identify the region with the most severe non-uniformity. Specifically, the processor 150 can identify regions with the greatest difference in line spacing and the greatest difference in distance from surrounding points in the reference pattern image of the image capture data as regions with the most severe non-uniformity. The processor 150 can set a correction weight for the region with the most severe non-uniformity to a reference value (e.g., 100% or 1), and can set the correction weight to gradually decrease in the direction away from the region.

[0095] Alternatively, the processor 150 may set the correction weight of the non-uniform region located at the center of a continuous non-uniform region to a reference value (e.g., 100% or 1), and may set the correction weight to gradually decrease in the direction away from the region.

[0096] Alternative sites, when non-uniform regions are as referenced Figures 5 to 8 When separated as in the described example, the processor 150 can set different correction weights for each non-uniform region based on the size difference between the original image and the distorted image. Figures 5 to 8 As shown, the second object 520 may have greater distortion than the first object 510. Therefore, the processor 150 can set the correction weight of the second object 520 to be greater than the correction weight of the first object 510.

[0097] The method for setting the correction weights can vary depending on the specific implementation.

[0098] For example, processor 150 can use machine learning to set correction weights for non-uniform regions included in an image. In this case, processor 150 can apply supervised learning methods or unsupervised learning methods to set the correction weights.

[0099] Alternatively, the processor 150 can measure the difference between the raw image data and the image capture data stored in the memory 110 for each coordinate point based on the coordinate points of the reference pattern image. The processor 150 can measure the similarity between the raw image data and the image capture data by comparing histograms obtained based on the measured differences, and can repeatedly perform the process of measuring similarity by using deep learning to determine the final correction weights.

[0100] Alternatively, the processor 150 can identify the projection surface state based on sensing results obtained from the distance sensor 130 and the image sensor 140, and can set correction weights for each region based on the identified state. The processor 150 can identify the maximum value among the correction weights set for each region, set the correction weight based on the non-uniform region identified as having the maximum correction weight in the image, and then set the correction weights for surrounding regions to decrease sequentially. In this case, the processor 150 can capture an image of the projection surface on which the corrected image is projected using the image sensor 140, and can measure the similarity of the corrected image by comparing the captured image with a reference pattern image stored in the memory 110. For example, the processor 150 can compare the non-uniformity of the corrected image for each region, and when the similarity is equal to or greater than 90% based on the comparison result, the correction weight can be set to the final value.

[0101] Figures 9 to 11This is a diagram illustrating a method for locally correcting the peripheral region of a non-uniform area. Figure 9 It is a diagram showing a pattern image projected onto a uniform projection surface. Figure 10 This shows a patterned image projected onto a non-uniform region. Figure 11 This shows how to apply correction weights Figure 10 The pattern image is corrected for non-uniform regions within the image. Furthermore, for ease of description, although... Figures 9 to 11 Only the horizontal lines of the reference pattern image are shown, but in actual correction, the correction can also be performed in the vertical direction in the same way.

[0102] Figure 10 The illustration shows a non-uniform region 1010 identified within the projection plane, with the upper region 1020 and lower region 1030 of the identified non-uniform region being uniform. In this case, when correction is performed by applying correction weights only to the identified non-uniform region 1010, undesirable image distortion may occur in the surrounding regions of the non-uniform region.

[0103] refer to Figure 11 When Figure 10 When correction is performed only on the non-uniform region 1010 without considering the correction of the surrounding regions 1020 and 1030 of the non-uniform region 1010, it is understandable that distortion occurs in the image of the upper region 1120 and the lower region 1130 due to the effect of the correction applied to region 1110.

[0104] For example, when on Figure 10 When performing correction on the non-uniform region 1010, the processor 150 can apply a correction value in the downward direction to correct the two upper lines distorted in the upward direction, and can apply a correction value in the upward direction to the two lower lines distorted in the downward direction. In this case, when the correction of the surrounding regions 1020 and 1030 is not considered, such as Figure 11 As shown, due to the effect of the correction applied to the upper part of the non-uniform region 1110, the peripheral region 1110 may have image distortion occurring in the downward direction, and due to the effect of the correction applied to the lower part of the non-uniform region 1110, the peripheral region 1130 may have image distortion occurring in the upward direction.

[0105] The processor 150 can set different correction weights for the surrounding areas of the identified non-uniform region than the correction weights set for the non-uniform region itself, and can locally distort and correct the image based on the differentiated correction weights. Compared to the correction weights set for the non-uniform region, the processor 150 can set relatively smaller correction weights for the surrounding areas.

[0106] For example, the processor 150 can subdivide the identified non-uniform region into cell regions of predetermined size, and can set different correction weights for each subdivided cell region. Furthermore, the processor 150 can subdivide the surrounding region of the non-uniform region into cell regions of predetermined size, and can set the correction weights of each subdivided cell region of the surrounding region to decrease sequentially based on the difference in correction weights between the closest cell regions in the surrounding region.

[0107] The processor 150 can control the image sensor 140 to capture images of the projection surface on which the corrected image is projected, and can determine whether recalibration is needed based on the captured images obtained from the image sensor 140.

[0108] For example, processor 150 can correct at least one frame in the actual content image based on correction weights, and can control projection unit 120 to project the corrected frame. After controlling image sensor 140 to capture an image of the corrected frame projected onto the projection surface, processor 150 can determine whether recorrection is needed by comparing the captured image with the original frame.

[0109] Alternatively, the processor 150 can correct the reference pattern image by applying correction weights. After the control projection unit 120 projects the corrected reference pattern image, the processor 150 can control the image sensor 140 to capture the corrected reference pattern image projected onto the projection surface. The processor 150 can determine whether the corrected image needs to be recorrected by comparing the captured image with the reference pattern image stored in the memory 110.

[0110] In other words, the processor 150 can compare the non-uniformity of each region of the non-uniform region based on the calibrated frame and the original frame or based on the calibrated reference pattern image and the reference pattern image before calibration, and can change the calibration weight for each region based on the comparison result to re-perform calibration.

[0111] The processor 150 can periodically determine whether non-uniform regions exist and whether to perform correction. Specifically, when projecting an image that has been corrected by applying correction weights, the processor 150 can control the image sensor 140 to capture an image every predetermined period (e.g., 10 minutes) and can compare the captured image with the original image to check for the presence of non-uniform regions. As a result of the check, if non-uniform regions are found even in the corrected image, the correction weights can be readjusted.

[0112] Alternatively, the processor 150 can periodically project a calibrated reference pattern image, compare the calibrated reference pattern image with a reference pattern image stored in the memory 110, and determine whether the calibrated image needs to be recalibrated based on the comparison result. For example, when the projection surface is implemented as a screen, a portion of the screen may bulge upwards or be concave downwards, and may be bent due to wind or external forces. Therefore, the screen state at the time of calibration and the screen state after calibration can be changed. Alternatively, the projection surface state may change even when insects or other foreign objects attach to and then detach from the projection surface. Therefore, the processor 150 can periodically perform the above-described recalibration operation to adaptively respond to changes in the projection surface state.

[0113] Simultaneously, multiple reference pattern images can be set and stored in memory 110. In this case, processor 150 can perform correction by using different reference pattern images for each cycle. For example, memory 110 can store data associated with multiple reference pattern images, which can be implemented in various forms, such as a grid pattern including multiple grids, or a matrix pattern in which multiple points, lines, symbols, characters, etc. are arranged like a matrix.

[0114] The processor 150 can apply a set of correction weights to each of a plurality of reference pattern images to correct the image, and can control the projection unit 120 to continuously project the plurality of corrected reference pattern images onto a projection surface. The processor 150 can control the image sensor 140 to capture images of the projection surface on which the plurality of corrected reference pattern images are projected at predetermined periods, and can periodically determine whether the corrected image needs to be recorrected by periodically comparing the captured image obtained from the image sensor 140 with the reference pattern images.

[0115] Therefore, processor 150 can periodically determine whether a corrected image needs to be recalibrated based on multiple reference pattern images implemented with various angles, sizes, and shapes, and can recalibrate the correction weights based on the determination results to prevent image distortion, even when images of various forms are projected onto non-uniform regions of the projection surface. Processor 150 can control image sensor 140 to capture the entire projection surface on which the corrected image is projected, and can identify the uniformity of the entire projection surface by comparing the captured image obtained from image sensor 140 with the reference pattern images stored in memory 140. In this case, when image correction is performed based on the correction weights of non-uniform regions, processor 150 can identify unwanted image distortion occurring in the peripheral regions of the non-uniform regions or image distortion occurring at specific locations in the image.

[0116] The processor 150 can determine whether the identified uniformity is within a predetermined reference range, and can re-perform the correction by changing the correction weight of each region based on the determination result. The processor 150 can repeat the above operation multiple times to set the correction weight of the state with the lowest degree of non-uniformity as the final value, and can continuously apply the correction weight to correct the image.

[0117] When determining the correction weights for the projection surface, the processor 150 can match information related to the projection surface with information related to the correction weights for each region, and store the matched information in the memory 110. Therefore, when a situation arises where an image needs to be projected onto the same projection surface again, the processor 150 can immediately obtain the correction weights for each region stored in the memory 110 to correct the image. Information related to the projection surface may include the position of the projection surface within the entire space where the projector 100 is located, the position of the projector 100, the size of the image, etc. When the position and projection direction of the projector 100 are identified using a LIDAR sensor or similar device included in the projector 100, the processor 150 can determine whether the projection surface has a previous image projection history based on the information stored in the memory 110. When a projection history exists, the processor 150 can obtain the correction weights for each region corresponding to the projection surface or information related to the position of non-uniform regions, and can use the obtained information for correction.

[0118] Meanwhile, when the image is identified as having a different height on the left and right sides based on the sensing values ​​obtained from the image sensor 140, the processor 150 can control the projection unit 120 to perform the trapezoidal correction as described above, or it can control the projection unit 120 to project the image onto a region with high uniformity.

[0119] The processor 150 can identify the state of the projection surface based on sensing results obtained from at least one of the distance sensor 130, the image sensor 140, or the orientation detection sensor 160, and can identify the number of planes included in the projection surface. For example, the processor 150 can generate coordinate information based on a reference pattern image projected onto the projection surface, and can control the distance sensor 130 based on the generated coordinate information to sense the distance to the projection surface for each region of the projection surface or for each coordinate point.

[0120] The processor 150 can identify angular axes that form symmetry with respect to the distance values ​​of each coordinate point based on the sensing results obtained from the distance sensor 130, and can identify the number of planes included in the projection plane based on the number of angular axes. For example, when there is one angular axis, the processor 150 can identify the number of planes included in the projection plane as two planes, and when there are three angular axes, the processor 150 can identify the number of planes included in the projection plane as three planes.

[0121] Alternatively, the processor 150 can identify the number of planes included in the projection surface based on sensing results obtained from the image sensor 140. The processor 150 can generate coordinate information based on a reference pattern image projected onto the projection surface, and can identify the number of planes based on corner detection in an image obtained by capturing an image of the projection surface according to the generated coordinate information. For example, when the number of corners in the captured image is four, the processor 150 can identify the number of planes included in the projection surface as one plane; when the number of corners is six, the processor 150 can identify the number of planes included in the projection surface as two planes; and when the number of corners is seven, the processor 150 can identify the number of planes included in the projection surface as three planes.

[0122] When multiple planes included in the projection plane are identified, the processor 150 can correct the image by applying correction weights to each region of the projection plane, including connecting regions, and can control the projection unit 120 to project the corrected image onto the projection plane, where the multiple planes are connected to each other. For example, the processor 150 can set correction weights based on regions on the projection plane including angular axes where the planes are connected, and can set the correction weights of peripheral regions to decrease sequentially. In this case, when a non-uniform region exists in at least one of the multiple planes, the processor 150 can apply correction weights set for that non-uniform region to correct the image.

[0123] When multiple planes included in the projection plane are identified, the processor 150 can control the projection unit 120 to perform keystone correction on the image to be projected onto the projection plane based on the state of the identified projection planes. Furthermore, the processor 150 can correct the image to which keystone correction has been performed by applying correction weights to each region, and can control the projection unit 120 to project the corrected image onto the projection plane.

[0124] In the above description, processor 150 controls projection unit 120 to first perform keystone correction on the image, and then applies correction weights for each region to the image to which keystone correction has been performed to correct the image. However, the order is not limited to this; the order of the two operations can be reversed, or the two operations can be performed in parallel.

[0125] As described above, in general, even when the projection surface includes multiple planes or there are non-uniform regions within the projection surface, keystone correction can be performed without considering these conditions, and therefore, image distortion may not be resolved. According to this disclosure, at least one of keystone correction and correction processing for non-uniform regions can be performed on the image to be projected onto the projection surface based on the identified state of the projection surface, thus resolving such image distortion.

[0126] When multiple planes included in the projection plane are identified, and the correction weight set for the connection area where these multiple planes are connected to each other exceeds a settable threshold, the processor 150 can control the projection unit 120 to project the image onto one of the multiple planes. The threshold represents the limit value of the correction weight, at which the processor 150 can correct non-uniform areas of the image by controlling the projection unit 120.

[0127] When the correction weight set for the connection area where multiple planes are connected to each other exceeds the settable threshold, the processor 150 can adjust at least one of the image projection angle, focal length or projection distance of the projection unit 120 to reduce the size of the image projected onto the projection surface or change the projection angle.

[0128] When multiple planes included in the projection plane are identified, and the correction weight for the non-uniform region is identified as equal to or greater than a threshold in at least one of the multiple planes, the processor 150 can control the projection unit 120 to project the image onto a region other than the region having a value equal to or greater than the threshold.

[0129] Specifically, when a non-uniform region with a value equal to or greater than the threshold is identified among multiple planes included in the projection plane, the processor 150 can control the projection unit 120 to project the image onto at least one of the remaining planes other than the plane containing the region with a value equal to or greater than the threshold. The processor 150 can reduce the size of the image projected onto the projection plane, or it can control the projection unit 120 to project the image onto a plane selected from the remaining planes other than the plane identified as having a value equal to or greater than the threshold. For example, the processor 150 can control the projection unit 120 to project the image onto the largest plane among the remaining planes other than the plane identified as having a value equal to or greater than the threshold.

[0130] Alternatively, the processor 150 may adjust at least one of the image projection angle, focal length, or projection distance of the projection unit 120 to project the image onto at least one plane other than the plane identified as having a value equal to or greater than a threshold.

[0131] Figures 12 to 20 This is a flowchart illustrating a control method for a projector according to various embodiments.

[0132] refer to Figure 12 The projector can project a reference pattern image onto the projection surface (S1210). The projector can store data related to the reference pattern image in its memory. The projector can sense the distance to and size of the projection surface using a distance sensor, and can adjust the size of the reference pattern image projected onto the projection surface based on the sensing results.

[0133] The projector can sense the distance to each area of ​​the projection surface on which a reference pattern image is projected using a distance sensor (S1220). In this case, the projector can generate coordinate information based on the reference pattern image projected onto the projection surface, and can control the distance sensor based on the generated coordinate information to sense the distance to the projection surface for each area or for each coordinate point. Alternatively, when the distance sensor has the function of generating and providing coordinate information itself, the projector can sense the distance to the projection surface for each area or for each coordinate point based on the coordinate information provided by the distance sensor.

[0134] The projector can capture an image of the projection surface on which a reference pattern image is projected using an image sensor (S1230). For example, the projector can be set to a wide-angle capture mode that allows it to capture a large area by adjusting the angle of the camera lens, and can capture the entire projection surface on which the reference pattern image is projected at once. Alternatively, the projector can divide the entire projection surface into screen areas, each with a predetermined size, and can capture an image for each of the divided screen areas.

[0135] The projector can identify the projection surface state based on the sensing values ​​obtained from the distance sensor and the image capture data obtained from the image sensor (S1240). The projector can identify individual non-uniform regions based on the sensing values ​​obtained from the distance sensor and the image capture data obtained from the image sensor, and can identify non-uniform regions to be corrected by comparing the identification results with each other.

[0136] The projector can set and store correction weights for each region of the projection surface based on the identified state of the projection surface (S1250). For example, the projector can measure the degree of distortion of the reference pattern image in a non-uniform region based on image capture data obtained from an image sensor, and can set correction weights for each region based on the degree of distortion in the non-uniform region. Alternatively, based on sensing values ​​obtained from a distance sensor, the projector can set correction weights for each region according to the difference between the distance to the non-uniform region and the distance to the uniform region on the projection surface. The projector can aggregate the correction weights extracted based on the sensing values ​​obtained from the distance sensor and the correction weights extracted based on the image capture data obtained from the image sensor, and can set correction weights for each region of the projection surface using the aggregated correction weights.

[0137] The projector can correct the image by applying correction weights for each region to the image to be projected onto the projection surface (S1260), and then project the corrected image onto the projection surface (S1270). Specifically, the projector can correct the image by applying correction weights for each region of the projection surface, set based on a reference pattern image, to the projected image to be projected onto the projection surface, and then project the corrected image onto the projection surface.

[0138] refer to Figure 13 The projector can identify the projection surface state by comparing the non-uniformity state of the projection surface sensed by the distance sensor with the non-uniformity state of each region in the captured image obtained from the image sensor (S1310). For example, the projector can identify a first non-uniform region of the projection surface based on the sensing value of the distance sensor, and can identify a second non-uniform region based on the image capture data obtained from the image sensor. The projector can identify the projection surface state by comparing the first non-uniform region with the second non-uniform region.

[0139] The projector can identify non-uniform regions within the projection surface based on the comparison results (S1320). For example, based on the comparison results of the first non-uniform region and the second non-uniform region, the projector can determine the common region between the first non-uniform region and the second non-uniform region, or each of the two regions, as the non-uniform region to be corrected on the projection surface.

[0140] refer to Figure 14 The projector can set and store different correction weights for the identified non-uniform region and its surrounding region (S1410). For example, compared to the correction weight set for the identified non-uniform region, the projector can set a relatively small correction weight for the surrounding region of the non-uniform region.

[0141] In this way, when performing image correction on non-uniform areas, the projector can prevent unwanted image distortion from occurring in the surrounding areas.

[0142] refer to Figure 15 The projector can capture an image of the projection surface on which a corrected image is projected (S1510). For example, the projector can project a reference pattern image onto the projection surface to which correction weights for each region of the projection surface are applied, and can capture the reference pattern image projected onto the projection surface using an image sensor.

[0143] The projector can determine whether the image needs to be recalibrated by comparing the captured image with a reference pattern image (S1520). The projector can compare the captured image corresponding to the calibrated reference pattern image with the reference pattern image stored in the memory, and can perform recalibration of non-uniform regions based on the comparison result.

[0144] refer to Figure 16 The projector can capture the entire projection surface on which the corrected image is projected (S1610). In this case, by using an image sensor, the projector is able to capture not only the non-uniform area but also the entire projection surface, which includes the peripheral area of ​​the non-uniform area of ​​the projection surface on which the corrected reference pattern image is projected.

[0145] The projector can identify the uniformity of the entire projection surface by comparing the captured image with a reference pattern image (S1620). The projector can also identify the uniformity of the entire projection surface by comparing the captured image of the entire projection surface with a reference pattern image stored in memory. The projector can then determine whether the identified uniformity is within a predetermined reference range (S1630).

[0146] When the uniformity of the entire projection surface is identified to be within a predetermined reference range, the projector can determine and store the correction weights for each region of the projection surface (S1640). However, when the uniformity of the entire projection surface is identified to be equal to or greater than the predetermined reference range, the projector can change the correction weights and perform image recalibration (S1650).

[0147] refer to Figure 17 The projector can select one of multiple reference pattern images at each predetermined time interval and can correct the selected reference pattern image based on correction weights (S1710). In this case, the projector can store data associated with the multiple reference pattern images in its memory. The multiple reference pattern images can include various reference pattern images of various forms to examine non-uniform areas of the projection surface based on various angles, shapes, sizes, etc.

[0148] The projector can project a calibrated reference pattern image onto a projection surface and can capture an image of the projection surface on which the calibrated reference pattern image is projected (S1720). For example, the projector can project multiple reference pattern images, to which calibration weights for each region of the projection surface are applied, onto the projection surface at predetermined intervals, and can capture the reference pattern image projected onto the projection surface using an image sensor.

[0149] The projector can determine whether the image needs to be recalibrated by comparing the captured image with a selected reference pattern image (S1730). The projector can periodically compare the captured image corresponding to the calibrated reference pattern image with the reference pattern image stored in memory, and can perform recalibration on non-uniform regions based on the comparison results. For example, the projector can set the maximum value or the average value of the calibration weights obtained by comparing multiple calibrated reference pattern images with the reference pattern image stored in memory as the final calibration weight, and can apply the set final calibration weight to the image to be projected onto the projection surface.

[0150] refer to Figure 18 The projector can identify the projection direction and tilt state of the projector by using an orientation detection sensor (S1810). In this case, the orientation detection sensor is a component for sensing the projection direction of the image projected from the projector, and may include at least one of a gyroscope sensor, a geomagnetic sensor, a tilt sensor, or a compass.

[0151] The projector can adjust at least one of the image projection angle, focal length, or projection distance based on the identified projection direction and tilt state (S1820). The projector can perform these operations based on sensing values ​​obtained from the orientation detection sensor. However, when these operations are performed in conjunction with sensing values ​​obtained from the distance sensor, the image projection angle, focal length, and projection distance of the projection surface can be adjusted more accurately.

[0152] Furthermore, before performing corrections for each area of ​​the projection surface, the projector can improve the uniformity of the corrected image by adjusting the projector image based on the projector's projection direction and tilt state.

[0153] refer to Figure 19 Based on at least one of the sensing values ​​obtained from the distance sensor, the image capture data obtained from the image sensor, or the projection direction and tilt state of the projector, the projector can identify the projection surface state (S1910).

[0154] The projector can identify the number of planes included in the projection surface based on the state of the identified projection surface (S1920). For example, the projector can identify reference lines where the direction of increase / decrease in distance values ​​changes as angular axes based on sensing values ​​obtained from a distance sensor, and can identify the number of planes included in the projection surface based on the number of angular axes. When there is one angular axis, the projector can identify the number of planes included in the projection surface as two planes; when there are three angular axes, the projector can identify the number of planes included in the projection surface as three planes.

[0155] The projector can determine whether the number of planes included in the projection surface is multiple planes (S1930), and when the number of planes included in the projection surface is identified as one plane rather than multiple planes, the projector can correct the image by applying correction weights for each region to the image to be projected onto the projection surface (S1940).

[0156] When the number of planes included in the projection surface is identified as multiple planes, the projector can correct the image based on the identified state of the projection surface by applying correction weights to each region of the projection surface, including connecting regions, and can project the corrected image onto the projection surface (S1950), wherein multiple planes are connected to each other in the connecting regions. For example, the projector can apply correction weights to correct connecting regions including angular axes where wall surfaces are connected to each other, and can correct the image by setting the correction weights of the surrounding regions of the connecting regions to be different from those of the connecting regions. In this case, based on the identified state of the projection surface, when at least one plane included in the projection surface has a non-uniform region, the projector can correct the image by applying correction weights to the non-uniform region.

[0157] A projector can perform keystone correction on an image to be projected onto a projection surface to adjust the left and right heights of the image projected onto multiple planes. The projector corrects the keystone-corrected image by applying correction weights to the image for each region, and then projects the corrected image onto the projection surface.

[0158] refer to Figure 20 The projector can identify whether the number of planes included in the projection surface is multiple planes (S2010), and when the number of planes included in the projection surface is identified as multiple planes, the projector can set the correction weight of each region of the projection surface, including the connecting region, based on the identified state of the projection surface (S2020), wherein, in the connecting region, multiple planes are connected to each other.

[0159] Based on the identified projection surface state, the projector can set correction weights for each region, including the connecting regions where multiple planes are connected to each other and the surrounding plane regions, and store these weights in memory.

[0160] When the number of planes included in the projection plane is identified as a single plane, the projector can correct the image by applying correction weights for each region to the image to be projected onto the projection plane (S2050).

[0161] The projector can identify whether the correction weights set for the connecting regions where multiple planes are connected exceed a settable threshold (S2030). When the correction weights set for the connecting regions based on the identification result exceed the threshold, the projector can adjust at least one of the projector's image projection angle, focal length, or projection distance to project the image onto one of the multiple planes (S2040).

[0162] When the correction weight set for the connected region based on the recognition result is equal to or less than the threshold, the projector can apply the correction weight for each region to the image to be projected onto the projection surface to perform correction (S2050).

[0163] The projector can also identify regions where a correction weight equal to or greater than a threshold is set for at least one of a plurality of planes. When a region with a correction weight equal to or greater than the threshold is identified in at least one plane, the projector can adjust at least one of its image projection angle, focal length, or projection distance to project the image onto at least one plane other than the plane identified as having a correction weight equal to or greater than the threshold. For example, the projector can reduce the size of the image projected onto the projection surface to project the image onto the remaining planes other than the plane identified as having a correction weight equal to or greater than the threshold. Alternatively, the projector can select one of the remaining planes other than the plane identified as having a correction weight equal to or greater than the threshold and can adjust the image projection angle to project the image onto the selected plane.

[0164] When no region with a correction weight equal to or greater than the threshold is identified on the projection surface, the projector can apply the correction weight for each region to the image to be projected onto the projection surface to perform correction (S2050).

[0165] Furthermore, according to embodiments of this disclosure, the various embodiments described above can be implemented by software including instructions stored on a machine-readable storage medium, which can be read by a machine (e.g., a computer). The machine can be a device that invokes the stored instructions from the storage medium, can operate based on the invoked instructions, and may include a projector according to the disclosed embodiments. When the instructions are executed by a processor, the processor can directly execute or perform the function corresponding to the instructions by using other components (under the control of the processor). The instructions may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be included in the form of a non-transitory storage medium. Here, the term "non-transitory" simply means that the storage medium is tangible and does not include signals, and does not distinguish whether data is stored semi-permanently or temporarily in the storage medium.

[0166] Furthermore, according to embodiments of this disclosure, the methods according to the various embodiments described above can be provided as a computer program product. This computer program product can be traded as a product between a seller and a buyer. The computer program product can be distributed in the form of a machine-readable storage medium (e.g., an optical disc read-only memory (CD-ROM)) or through an app store (e.g., the Play Store). TM Online distribution. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or generated in a storage medium (e.g., the storage of a manufacturer's server, an app store server, or a relay server).

[0167] Furthermore, each component (e.g., a module or program) according to the various embodiments described above may include a single entity or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in the various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into one entity and may perform the functions performed by the respective corresponding components prior to integration in the same or similar manner. Operations performed by modules, programs, or other components according to the various embodiments may be performed sequentially, in parallel, iteratively, or heuristically, and at least some operations may be performed in a different order or omitted, or other operations may be added.

[0168] Although preferred embodiments of the present disclosure have been shown and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications can be made by those skilled in the art to which this disclosure pertains without departing from the scope of the present disclosure as claimed in the appended claims. These modifications should also be understood to fall within the spirit of this disclosure.

Claims

1. A projector, comprising: Memory; Projection unit; A distance sensor is configured to sense the distance to the projection surface; An image sensor is configured to capture an image of the projection surface; as well as processor, The processor is configured as follows: The projection surface state is identified based on the sensing results obtained from the distance sensor and the image sensor. Based on the identified state, a correction weight is set for each region, and the correction weight is stored in the memory. The image is corrected by applying the correction weights for each region to the image to be projected onto the projection plane, and The projection unit is controlled to project the corrected image onto the projection surface.

2. The projector according to claim 1, wherein, The memory stores data related to the reference pattern image, and The processor is configured to: The projection unit is controlled to project the reference pattern image onto the projection surface; The image sensor is controlled to capture an image of the projection surface on which the reference pattern image is projected; as well as Non-uniform regions within the projection surface are identified based on the captured image obtained from the image sensor and the sensing value obtained from the distance sensor. Different correction weights are set for the non-uniform regions and the surrounding areas of the non-uniform regions, and the correction weights are stored in the memory.

3. The projector according to claim 2, wherein, The processor is configured to: The reference pattern image is corrected based on the set correction weights, and the projection unit is controlled to project the corrected reference pattern image onto the projection surface. as well as The image sensor is controlled to capture an image of the projection surface on which the corrected reference pattern image is projected, and the need to recalibrate the image is determined by comparing the captured image obtained from the image sensor with the reference pattern image.

4. The projector according to claim 3, wherein, The processor is configured to: The image sensor is controlled to capture an image of the entire projection surface on which the corrected image is projected; as well as The uniformity of the entire projection surface is identified by comparing the captured image obtained from the image sensor with the reference pattern image, and it is determined whether the identified uniformity is within a predetermined reference range.

5. The projector according to claim 2, wherein, The memory stores data associated with multiple reference pattern images, and The processor is configured as follows: In each predetermined time period, one of the plurality of reference pattern images is selected, the selected reference pattern image is corrected based on the correction weight, and the projection unit is controlled to project the corrected reference pattern image onto the projection surface. Control the image sensor to capture an image of the projection surface on which the calibrated reference pattern image is projected; and Whether the image needs to be recalibrated is determined by comparing the captured image obtained from the image sensor with a selected reference pattern image.

6. The projector according to claim 1, further comprising: An orientation detection sensor is configured to sense the projection orientation of the projection unit. The processor is configured as follows: The projection direction and tilt state of the projector are identified based on the sensing results obtained from the direction detection sensor. The projection unit adjusts at least one of the image projection angle, focal length, or projection distance based on the identified state.

7. The projector according to claim 6, wherein, The processor is configured to: When multiple planes included in the projection plane are identified, the results are based on sensing results obtained from at least one of the distance sensor, the image sensor, and the orientation detection sensor. The image is corrected by applying correction weights to each region, including the connected regions, wherein the plurality of planes are connected to each other in the connected regions. The projection unit is controlled to project the corrected image onto the projection surface.

8. The projector according to claim 7, wherein, The processor is configured to: When the correction weight set for the connection region exceeds a settable threshold, the projection unit is controlled to project the image onto one of the plurality of planes.

9. A method for controlling a projector, the method comprising: Project the reference pattern image onto the projection surface; The distance to each region of the projection surface on which the reference pattern image is projected is sensed by using a distance sensor; An image of the projection surface on which the reference pattern image is projected is captured using an image sensor; The projection surface state is identified based on the sensing values ​​obtained from the distance sensor and the image capture data obtained from the image sensor; Based on the identified state, set and store the correction weights for each region of the projection surface; The image is corrected by applying the correction weights for each region to the image to be projected onto the projection surface; as well as The corrected image is projected onto the projection surface.

10. The method according to claim 9, wherein, Identifying the state of the projection surface includes: The non-uniformity of the projection surface sensed by the distance sensor is compared with the non-uniformity of each region in the captured image obtained from the image sensor; and Non-uniform regions within the projection plane are identified based on the comparison results.

11. The method according to claim 10, wherein, Setting and storing the correction weights for each region includes: Different correction weights are set and stored for the identified non-uniform regions and the surrounding regions of the identified non-uniform regions.

12. The method according to claim 9, further comprising: Capture an image of the entire projection plane on which the corrected image is projected; The uniformity of the entire projection surface is identified by comparing the captured image with the reference pattern image; as well as Determine whether the identified uniformity is within the predetermined reference range.

13. The method of claim 9, further comprising: In each predetermined time period, one of a plurality of reference pattern images is selected, and the selected reference pattern image is corrected based on the correction weight; The calibrated reference pattern image is projected onto the projection surface, and an image of the projection surface on which the calibrated reference pattern image is projected is captured. as well as Whether the captured image needs to be recalibrated is determined by comparing it with a selected reference pattern image.

14. The method of claim 10, further comprising: Identify the projection direction and tilt state of the projector; as well as The projector adjusts at least one of the image projection angle, focal length, or projection distance based on the identified state.

15. The method of claim 14, further comprising: The projection surface state is identified based on the sensed value obtained from the distance sensor, the image capture data obtained from the image sensor, and at least one of the projection direction and the tilt state of the projector; The number of planes included in the projection plane is identified based on the identified state; as well as When the number of planes included in the projection plane is identified as multiple planes, the image is corrected by applying the correction weights for each region of the projection plane, including the connecting regions, and the corrected image is projected onto the projection plane, wherein the multiple planes are connected to each other in the connecting regions.

Images