Projection device and method for recognizing object using image projection

Abstract

The projection apparatus includes: a light source configured to output image light; a projection lens configured to project image light to the outside; a camera configured to photograph an object located between the camera and an external projection surface of the projected image light; a light blocker configured to block the image light output by the light source during a blocking period; and at least one processor configured to obtain illuminance information about illuminance of the external projection surface, set a blocking period during which image light output by the light source is blocked, set a photographing period within the blocking period based on the illuminance information, control the light blocker to block the image light output by the light source within the blocking period, and control the light blocker to block the image light output by the light source within the blocking period. And controlling the camera to photograph the object within the photographing period.

Description

Technical Field

[0001] The present disclosure relates to a method for identifying an object using image projection and a projection device for performing the method. In more detail, various embodiments of the present disclosure relate to a method for experiencing augmented reality (AR) through mutual interaction between an image and an object by simultaneously projecting an image through a projection device and identifying an object located on the image. Background Art

[0002] Augmented reality (AR) is a technology used to overlay and display virtual images on real-world objects or real-world physical environment spaces. AR projectors that project images using AR technology can be used in daily life.

[0003] The AR projector can not only project images, but also identify objects located on the projection surface of the image. The AR projector can provide an AR experience through the mutual interaction between the projected image and the identified object.

[0004] Since the AR projector must identify objects on the projection surface of the image, the AR projector must correctly distinguish between images and objects. In addition, the AR projector can easily identify objects located in an environment with a specific illumination. That is, depending on the illumination change, the AR projector may not easily recognize images located in an environment where the illumination is too dark or too bright. Therefore, it is necessary to correctly identify objects located on the projection surface of the image regardless of how the ambient illumination changes. Summary of the invention

[0005] Solution

[0006] A projection device for recognizing an object using image projection is provided.

[0007] According to one aspect of the present disclosure, a projection device configured to identify an object using image projection includes: a light source configured to output image light; a projection lens configured to project the image light to the outside; a camera configured to photograph an object located between the camera and an external projection surface on which the image light is projected; a light blocker configured to block the image light output by the light source during a blocking period; and at least one processor configured to control operations of the light source, the camera, and the light blocker, wherein the at least one processor is further configured to: obtain illumination information related to the illumination of the external projection surface, set a blocking period during which the image light output by the light source is blocked, set a shooting period within the blocking period based on the illumination information, control the light blocker to block the image light output by the light source during the blocking period, and control the camera to shoot the object during the shooting period.

[0008] According to one aspect of the present disclosure, a method for identifying an object using image projection includes: obtaining illuminance information about the illuminance of an external projection surface of projected image light; setting a blocking period during which the output image light is blocked based on the illuminance information; setting a shooting period within the blocking period; blocking the output image light during the blocking period; and shooting the object located on the external projection surface during the shooting period.

[0009] According to an embodiment of the present disclosure, there is provided a computer-readable recording medium having recorded thereon a program for executing the method on a computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent through the following description in conjunction with the accompanying drawings, in which:

[0011] Figure 1 A conceptual diagram illustrating an operation of a projection device simultaneously projecting an image and recognizing an object according to an embodiment of the present disclosure;

[0012] Figure 2 A flowchart showing an operating method of a projection device according to an embodiment of the present disclosure;

[0013] Figure 3 An operation of a projection device recognizing an object by matching captured images when the illumination is dark according to an embodiment of the present disclosure is shown;

[0014] Figure 4 A flowchart illustrating an operation method by which a projection device repeatedly captures an image including an object by using a photographing period according to an embodiment of the present disclosure;

[0015] Figure 5 A flowchart showing an operation method of recognizing an object by matching captured images through a projection device thereof according to an embodiment of the present disclosure;

[0016] Figure 6 shows a graph illustrating an operation of a projection device to photograph an object by using different photographing cycles according to illumination according to an embodiment of the present disclosure;

[0017] Figure 7 A diagram for describing an operation of a projection device to obtain a matching image by using an object image captured according to illumination according to an embodiment of the present disclosure is shown;

[0018] Figure 8 A flowchart showing an operation method of recognizing an object by matching captured images through a projection device thereof according to an embodiment of the present disclosure;

[0019] Fig. 9A diagram showing a projector among elements of a projection device according to an embodiment of the present disclosure is used to describe in detail;

[0020] Fig.10 A diagram showing a light blocker among elements of a projection device according to an embodiment of the present disclosure for describing in detail;

[0021] Fig.11 A diagram showing a light blocker among elements of a projection device according to an embodiment of the present disclosure for describing in detail;

[0022] Fig.12 A conceptual diagram for describing a feature by which an operation of a projection device is determined based on an operation of a light blocker according to an embodiment of the present disclosure is shown;

[0023] Fig.13 A graph showing an operation of a projection device to photograph an object by using different photographing cycles according to illumination according to an embodiment of the present disclosure; and

[0024] Fig.14 A block diagram for describing a configuration of a projection device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0025] Throughout this disclosure, the expression “at least one of a, b, or c” indicates only a, only b, only c, both a and b, both a and c, both b and c, or all of a, b, and c.

[0026] All terms including descriptive terms or technical terms used in the embodiments of the present disclosure should be interpreted as having meanings that are obvious to those of ordinary skill in the art. However, according to the intention of those of ordinary skill in the art, precedent situations or the emergence of new technologies, terms can have different meanings. In addition, the applicant can arbitrarily select some terms, and in this case, the meaning of the selected terms will be described in detail in the detailed description of the present disclosure. Therefore, the terms used in the present disclosure should not be interpreted based only on their names, but must be defined based on the meaning of the terms and the description in the entire specification.

[0027] As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. All terms used herein (including descriptive terms or technical terms) should be interpreted as having meanings that are obvious to one of ordinary skill in the art.

[0028] Throughout the disclosure, when a component "includes" or "comprising" an element, unless there is a specific description to the contrary, the component may also include other elements without excluding other elements. As used in the specification, the term "unit" or "module" means an entity for performing at least one function or operation, and may be implemented as hardware, software, or a combination of hardware and software.

[0029] The expression "configured to" used in the present disclosure may be interchangeable with, for example, "suitable for", "capable of...", "designed to", "suitable for", "manufactured to", or "capable of", depending on the circumstances. The term "configured to" may not necessarily imply "specially designed to" in hardware. Alternatively, in some cases, the expression "a system configured to..." may mean that the system, together with other devices or components, "is capable of...". For example, the phrase "a processor adapted (or configured) to perform A, B, and C" may mean a dedicated processor (e.g., an embedded processor) that is used only to perform the corresponding operations, or a general-purpose processor (e.g., a central processing unit (CPU) or an application processor (AP)) that can perform the corresponding operations by executing one or more software programs stored in a memory.

[0030] Furthermore, in the present disclosure, when an element is referred to as being “connected to” another element, it should be understood that the element may be directly connected to the other element but may be connected or coupled via another element in between, unless otherwise specified.

[0031] Hereinafter, the present disclosure will now be described more fully with reference to the accompanying drawings so that those skilled in the art can perform the embodiments of the present disclosure without difficulty. However, the present disclosure can be implemented in many different forms and should not be construed as being limited to the embodiments of the present disclosure set forth herein.

[0032] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0033] Figure 1 A conceptual diagram illustrating an operation of a projection device simultaneously projecting an image and recognizing an object according to an embodiment of the present disclosure.

[0034] refer to Figure 1 , the projection device 100 according to an embodiment of the present disclosure can project an image.

[0035] The projection device 100 may include a projector to project an image. In the disclosure, the projector may include a light source unit and a projection lens. Fig. 9 The detailed configuration of the projection device 100 is described, and reference will now be made to Figure 1 The operation of the projection device 100 is described.

[0036] The projection device 100 may output image light by using a light source unit. The image light output by the light source unit may be white light. The projection device 100 may project the image light output by the light source unit to the outside by using a projection lens. The image light projected to the outside by the projection device 100 may indicate an image 120 projected by the projection device 100. That is, the projection device 100 may project the image 120 by using a light source unit and a projection lens.

[0037] However, the projection device 100 may project the image 120 by using more or other elements, and the configuration of the projection device 100 is not limited to the light source unit and the projection lens.

[0038] According to an embodiment of the present disclosure, the projection device 100 may further include a camera to photograph an object 160 positioned relative to the external projection surface 140 on which the image light is projected. In other words, the object 160 may be located on the external projection surface 140, for example, Figure 1 The projection device 100 may obtain an object image including the object 160 by using a camera. The object image may be an image including the object 160 positioned relative to the external projection surface 140.

[0039] The AR projector 100 may obtain an object image by using an RGB camera, but this is merely an example and the present disclosure is not limited thereto. For example, the projection device 100 may obtain an object image by using one of a stereo camera, an infrared (IR) camera, an RGB depth (RGB-D), and a time of flight (ToF) camera.

[0040] In the present disclosure, the external projection surface 140 may be a surface on which image light is projected via a projection lens of the projection device 100 .

[0041] In the present disclosure, the object 160 positioned relative to the external projection surface 140 may be: an object 160 positioned in contact with the external projection surface 140; an object 160 located on the external projection surface 140; and an object 160 located between the external projection surface 140 and the camera and therefore located in the area on which the image is projected.

[0042] According to an embodiment of the present disclosure, the projection device 100 may perform an operation of projecting the image 120 and an operation of photographing the object 160. The operation of projecting the image 120 and the operation of photographing the object 160 may not be performed simultaneously but may be performed separately.

[0043] According to an embodiment of the present disclosure, the projection device 100 may set a shooting period and a projection period. The projection device 100 may control the camera to shoot the object 160 during the shooting period. The projection device 100 may control the light source unit and the projection lens to project an image during the projection period. The shooting period and the projection period may not overlap.

[0044] According to an embodiment of the present disclosure, the projection device 100 may detect an object based on an object image including the object 160. The projection device 100 may detect the object 160 included in the object image by using an object detection algorithm.

[0045] For example, as shown, the object 160 positioned relative to the external projection surface 140 may be a hand. When the projection device 100 includes a stereo camera, the projection device 100 may detect the hand by using hand skeleton detection and tracking technology, and may also recognize the three-dimensional posture of the hand via depth information.

[0046] The hand skeleton detection and tracking technology is a technology that detects the moving joint part on the human hand image, segments a predetermined skeleton structure, and then projects the segmented image. The skeleton structure of the human hand may include the end points (5) of the fingers, the joint points (5*3=15) of the fingers, the palm point (1), and the wrist point (1), but the present disclosure is not limited thereto.

[0047] That is, according to an embodiment of the present disclosure, the projection device 100 can identify the hand by using hand skeleton detection and tracking technology, can track the hand skeleton via the joint points, palm points and wrist points of the fingers, and thus can specifically identify hand movements such as touching, swiping, pinching, etc.

[0048] Figure 2 Flow chart showing the operation method of the projection device according to the embodiment of the present disclosure. Figure 1 Those elements whose descriptions overlap are briefly described.

[0049] refer to Figure 2 In operation S210, the projection device may obtain illuminance information indicating illuminance of an external projection surface.

[0050] According to an embodiment of the present disclosure, the projection device may further include an illuminance sensor to measure the illuminance of the external projection surface. The projection device may measure the illuminance of the external projection surface by using the illuminance sensor.

[0051] However, the projection device including an illuminance sensor for obtaining illuminance information is merely an example, and the method of obtaining illuminance information is not particularly limited. For example, the projection device may obtain an object image relative to an external projection surface by using a camera, may calculate the illuminance of the external projection surface based on the object image relative to the external projection surface, and thus may obtain illuminance information.

[0052] In operation S220, the projection apparatus may set a blocking period for blocking the output image light based on the illuminance information.

[0053] According to an embodiment of the present disclosure, the projection device may project an image by using a light source unit and a projection lens. The projection device may further include a light blocker, and a blocking period may be set in which the image light output from the light source unit is blocked by controlling the light blocker.

[0054] In this disclosure, the image light output from the light source unit being blocked may mean that the image is not projected from the projection device. That is, the blocking period may indicate a period during which the image is not projected by the projection device. In addition, the period during which the image is projected by the projection device may be a projection period. The projection period may be a period excluding the blocking period and may not overlap with the blocking period.

[0055] According to an embodiment of the present disclosure, the projection device can set the blocking period based on the illumination information. For example, when the interval of the blocking period increases as the illumination increases according to the illumination information, the shooting period can be increased, or when the blocking period decreases, the exposure time of the camera can be increased. The projection device can set the blocking period based on the illumination information, and thus can control the operation of the camera and the projector corresponding to the change in ambient illumination.

[0056] According to an embodiment of the present disclosure, the projection device can set the projection period and the shooting period so that the ratio of the shooting period to the projection period decreases when the illumination increases according to the illumination information. That is, the projection device can project an image and shoot an object according to the projection period and the blocking period repeated in an alternating manner, however, when the illumination is bright, the projection device can control the camera to shoot the object frequently.

[0057] In operation S230, the projection device may set the photographing period to be included in the blocking period. The photographing period may be shorter than the blocking period.

[0058] According to an embodiment of the present disclosure, the starting point of the shooting period can match the starting point of the blocking period. That is, when the blocking of the image projection by the projector starts, the projection device can control the camera to shoot the object located on the external projection surface.

[0059] In operation S240, the projection device may control the light blocker to block the output image light during the blocking period. In operation S250, the projection device may control the camera to photograph the subject during the photographing period.

[0060] According to an embodiment of the present disclosure, the projection device may output image light by using a light source unit. The projection device may control the light blocker to block the output image light during a set blocking period. That is, the projection device may not project an image on an external projection surface during the blocking period.

[0061] According to an embodiment of the present disclosure, the projection device can control the camera to shoot the object during the shooting period included in the blocking period. That is, the projection device can shoot the object located on the external projection surface while the image is not projected.

[0062] The projection device divides the operation of projecting an image and the operation of photographing an object, so that a clear object image including the object can be obtained. The projection device can recognize the object based on the obtained clear object image. According to an embodiment of the present disclosure, the operation of projecting an image and the operation of photographing an object can be repeated very quickly so that the human eye cannot distinguish them, and the projection device can provide an augmented reality (AR) image to the user, and from the user's perspective, it seems as if the object on the projected image is recognized.

[0063] Figure 3 An operation of the projection device recognizing an object by matching captured images when illumination is dark according to an embodiment of the present disclosure is illustrated.

[0064] in addition, Figure 3 The separation between the operation of projecting an image and the operation of photographing an object is shown, the operation being performed by the projection device by comparing the projection signal with the camera signal. Each of the projection signal and the camera signal may indicate a clock signal, which is a periodic signal used in synchronization.

[0065] refer to Figure 3 According to an embodiment of the present disclosure, the projection device can obtain a projection signal and a camera signal.

[0066] The projection signal may be a clock signal in which 1 and 0 are repeated according to a constant cycle. The projection signal may be a signal about a time point at which the projection device projects an image. The projection device may periodically project an image based on the projection signal. For example, the projection device may periodically project an image based on the projection signal, the image being generated by processing image light output from the light source unit.

[0067] According to an embodiment of the present disclosure, the projection device may project an image whenever a projection signal is received. The projection device may repeatedly project an image based on a period of the projection signal. For example, the projection device may repeatedly project an image based on a projection signal having a frequency of 120 hertz (Hz). As another example, the projection device may repeatedly project an image based on a projection signal having a period of 8.3 milliseconds (ms).

[0068] The camera signal may be a clock signal in which 1 and 0 are repeated according to a constant cycle, or may be a transistor-to-transistor logic (TTL) signal composed of 1 and 0. The camera signal may be a signal indicating a time point at which the projection device photographs the object. The projection device periodically photographs the object based on the camera signal. For example, the projection device may control an operation of opening a shutter of a camera so as to photograph the object based on the camera signal.

[0069] According to an embodiment of the present disclosure, the projection device may shoot an object after a preset time has passed from a time point T1 at which a camera signal is obtained. The camera signal may refer to a signal for indicating a shooting operation of a camera. The preset time from the time point T1 at which the camera signal is obtained may indicate a time for requesting a light blocker operation so that the projection device can block image projection by using the light blocker. That is, since the light blocker is a mechanical element, after the projection device receives the camera signal, a preset time may be requested for the projection device to operate the light blocker. For example, the preset time requested for operating the light blocker may be a time from the time point T1 to a second time point T2.

[0070] However, this is merely an example, and the present disclosure is not limited thereto.For example, once the projection device obtains a camera signal, the projection device may control a light blocker to block image projection.

[0071] According to an embodiment of the present disclosure, the projection device may set a blocking period in which the image light output by the light source unit is blocked. The projection device may set the blocking period to repeat according to a preset interval. As shown in the figure, the blocking period may include the time from the second time point T2 to the fourth time point T4. In addition, the blocking period may include the time from the sixth time point T6 to the eighth time point T8. Each blocking period may be the same time period. That is, the blocking period may include the same time period repeated according to a preset cycle.

[0072] According to an embodiment of the present disclosure, the projection device may control the light blocker to block the output image light during the blocking period. When the output image light is externally projected via the projection lens, the projection device may project an image on an external projection surface. The projection device may not project an image by controlling the light blocker to block the image light during the blocking period.

[0073] In addition, the projection device may project an image during the projection period. The projection device may project an image by controlling the output image light not to be blocked by the light blocker during the projection period. The projection period and the blocking period may not overlap. For example, the projection period may include the time from the fourth time point T4 to the fifth time point T5. Although not shown, the projection period may include the same time period repeated according to a preset cycle.

[0074] According to an embodiment of the present disclosure, the projection device may set a shooting period in which a camera is used to shoot an object. As shown in the figure, the shooting period may include a time from a second time point T2 to a third time point T3. In addition, the shooting period may include a time from a sixth time point T6 to a seventh time point T7. Each shooting period may be the same time period. That is, the shooting period may include the same time period repeated according to a preset cycle.

[0075] According to an embodiment of the present disclosure, a photographing period may be included in the blocking period. The photographing period may indicate a period during which an aperture of a camera is opened to achieve exposure for sensing light.

[0076] The projection device may control the camera to shoot the object during the shooting period. That is, when the shooting period starts, the projection device may control the camera to open the aperture of the camera. The projection device may sense light via exposure, and may generate an object image based on the sensed light during the shooting period.

[0077] According to an embodiment of the present disclosure, the projection device can obtain a matching image via an object image obtained during a shooting period. The projection device obtains a matching image when the illumination is dark, and thus an image with appropriate illumination can be obtained. That is, the projection device can obtain an object image with darker illumination during each shooting period, and can match the obtained object image with darker illumination, and thus a matching image with brighter illumination can be obtained.

[0078] For example, the projection device may obtain the first object image 310 during a first photographing period. Here, the first photographing period may indicate a photographing period from a second time point T2 to a third time point T3.

[0079] The projection device may obtain the second object image 320 during the second photographing period. Here, the second photographing period may indicate a photographing period from a sixth time point T6 to a seventh time point T7.

[0080] The projection device may obtain the matching image 330 by matching the first object image 310 and the second object image 320. In the present disclosure, matching may refer to an image conversion process for obtaining an image with brighter illumination by synthesizing an image with darker illumination. The image conversion method is not particularly limited.

[0081] According to an embodiment of the present disclosure, the illuminance of the matching image 330 may be greater than the illuminance of the first object image 310 and the illuminance of the second object image 320 .

[0082] Although it is shown that the projection device obtains the matching image 330 by matching the first object image 310 and the second object image 320, the number of images requested for obtaining the matching image 330 is not particularly limited. For example, the projection device can obtain the matching image by matching the first object image, the second object image, and the third object image.

[0083] According to an embodiment of the present disclosure, the projection device may detect the object 335 based on the matching image 330 including the object 335. For example, as shown in the figure, the object 335 may be a user's hand. However, this is merely an example, and the present disclosure is not limited thereto.

[0084] The projection device may use a deep learning-based object detection technology in order to detect the object 335 included in the matching image 330. However, a method in which the projection device recognizes the object 335 is not particularly limited.

[0085] Figure 4 A flowchart illustrating an operation method by which a projection device repeatedly captures an image including an object by using a photographing period according to an embodiment of the present disclosure.

[0086] refer to Figure 4 , according to an embodiment of the present disclosure, Figure 2 Operation S250 may include operation S410 and operation S420.

[0087] In operation S410, the projection device may determine a photographing cycle from the start of a first photographing period to the start of a second photographing period. That is, the projection device may determine the photographing cycle to allow the photographing period to be repeated at regular intervals.

[0088] In addition, refer to Figure 3 , the first shooting period may indicate a period from the second time point T2 to the third time point T3. The second shooting period may indicate a period from the sixth time point T6 to the seventh time point T7. The projection device may determine a shooting period having a period from the second time point T2 to the sixth time point T6.

[0089] In operation S420, the projection device may control the camera to repeatedly photograph the subject during the photographing period based on the photographing cycle.

[0090] According to an embodiment of the present disclosure, the projection device may obtain an object image including an object during a shooting period repeated at regular intervals based on the determined shooting cycle. For example, the first shooting period and the second shooting period may be examples of shooting periods repeated at regular intervals. The projection device may obtain a first object image including an object during the first shooting period. The projection device may obtain a second object image including an object during the second shooting period.

[0091] According to an embodiment of the present disclosure, the projection device may detect an object based on a plurality of object images obtained during a photographing period.

[0092] Figure 5 A flowchart illustrating an operation method of recognizing an object by matching captured images through a projection device thereof according to an embodiment of the present disclosure.

[0093] refer to Figure 5 , according to an embodiment of the present disclosure, Figure 2 Operation S250 may include operation S510 and operation S520.

[0094] In operation S510, the projection device may obtain a first object image including the object by controlling the camera to photograph the object during a first photographing period. In operation S520, the projection device may obtain a second object image including the object by controlling the camera to photograph the object during a second photographing period.

[0095] According to an embodiment of the present disclosure, the projection device can obtain a first object image and a second object image both including an object during a first shooting period and a second shooting period that are not connected. For example, the projection device can expose the aperture of the camera by opening the aperture, thereby obtaining the first object image during the first shooting period. After obtaining the first object image, the projection device can control the aperture of the camera to close. Thereafter, the projection device can expose the aperture of the camera by opening the aperture again, thereby obtaining the second object image during the second shooting period.

[0096] In operation S260, the projecting device may obtain a matching image based on the first object image and the second object image. However, the number of images requested in order to obtain the matching image is not particularly limited.

[0097] According to an embodiment of the present disclosure, the projection device can obtain a matching image with bright illumination by matching the first object image and the second object image. The illumination of the matching image can be greater than the illumination of the first object image and the illumination of the second object image. For example, the illumination of the matching image can be equal to the sum of the illumination of the first object image and the illumination of the second object image. However, this is only an example, and the specific calculation equation of the illumination of the matching image is not particularly limited.

[0098] According to an embodiment of the present disclosure, the projection device may detect an object based on a matching image.

[0099] Figure 6 A graph illustrating an operation of a projection apparatus photographing a subject by using different photographing cycles according to illumination according to an embodiment of the present disclosure is shown.

[0100] For ease of description, Figure 3 Descriptions of overlapping elements are briefly described or not provided.

[0101] refer to Figure 6 According to an embodiment of the present disclosure, the projection device can obtain a projection signal as well as a first camera signal, a second camera signal, and a third camera signal.

[0102] The first camera signal, the second camera signal, and the third camera signal may each be a clock signal or a TTL signal in which 1 and 0 are repeated according to a constant cycle. Specifically, the projection device may obtain camera signals with different cycles according to the illumination. The projection device may periodically photograph an object based on the first camera signal, the second camera signal, and the third camera signal.

[0103] As shown in the figure, the projection device can obtain a first camera signal when the illumination is dark. When the illumination is medium, the projection device can obtain a second camera signal. When the illumination is bright, the projection device can obtain a third camera signal. In the present disclosure, medium illumination can indicate that the illumination is greater than the illumination in the case of obtaining the first camera signal and lower than the illumination in the case of obtaining the third camera signal. That is, the case of medium illumination is relatively defined and therefore is not particularly limited.

[0104] According to an embodiment of the present disclosure, the projection device can shoot the object after a preset time has passed from the time point T1 at which the camera signal is obtained. Here, the camera signal may include a first camera signal, a second camera signal, and a third camera signal. The preset time may refer to the time from the first time point T1 to the second time point T2.

[0105] According to an embodiment of the present disclosure, the projection device may set a blocking period in which the image light output by the light source unit is blocked. The projection device may set the blocking period to repeat according to a preset interval. As shown in the figure, the blocking period may include a time from a second time point T2 to a fourth time point T4.

[0106] According to an embodiment of the present disclosure, the projection device may set a shooting period in which a subject is shot by using a camera. The shooting period may be included in the blocking period. Hereinafter, setting the shooting period, the projection period, and the shooting cycle according to the illumination will now be described in detail.

[0107] According to an embodiment of the present disclosure, the projection device may determine the length of the shooting period according to the illumination. The projection device may determine the length of the shooting period based on illumination information indicating the illumination of the external projection surface on which the image is projected. The projection device may set the length of the shooting period to decrease as the illumination increases. That is, when the illumination is dark, the projection device may set the shooting period so that the exposure is maintained for a long time by opening the aperture of the camera for a long time.

[0108] For example, when the illumination is dark, the projection device may shoot the object based on the first camera signal. When the illumination is dark, the shooting period may include the time from the second time point T2 to the 1_3 time point T13. The projection device may obtain the 1_1 object image 610 by shooting the object during the shooting period from the second time point T2 to the 1_3 time point T13.

[0109] When the illumination is medium, the projection device may shoot the object based on the second camera signal. When the illumination is medium, the shooting period may include a time from the second time point T2 to a 2_3 time point T23. The projection device may obtain a 2_1 object image 620 by shooting the object during the shooting period from the second time point T2 to a 2_3 time point T23.

[0110] When the illumination is bright, the projection device may shoot the object based on the third camera signal. When the illumination is bright, the shooting period may include the time from the second time point T2 to the 3_3 time point T33. The projection device may obtain the 3_1 object image 630 by shooting the object during the shooting period from the second time point T2 to the 3_3 time point T33.

[0111] The time from the second time point T2 to the 1_3 time point T13 may be longer than the time from the second time point T2 to the 2_3 time point T23. The time from the second time point T2 to the 2_3 time point T23 may be longer than the time from the second time point T2 to the 3_3 time point T33. That is, the projection device may set the shooting period to decrease as the illumination increases.

[0112] According to an embodiment of the present disclosure, when the illumination is dark, the projection device may obtain a 1_1 object image 610 including an object. When the illumination is medium, the projection device may obtain a 2_1 object image 620 including an object. When the illumination is bright, the projection device may obtain a 3_1 object image 630 including an object.

[0113] The illuminance of the 1_1 object image 610 may be lower than that of the 2_1 object image 620. The illuminance of the 2_1 object image 620 may be lower than that of the 3_1 object image 630.

[0114] According to an embodiment of the present disclosure, the projection device may determine the length of the projection period according to the illumination. The projection device may determine the length of the projection period based on illumination information indicating the illumination of the external projection surface on which the image is projected. The projection device may set the length of the projection period to decrease as the illumination decreases. That is, when the illumination is dark, the projection device reduces the projection period so that many object images for identifying the object can be obtained in the projection period. The projection device can obtain a matching image with appropriate illumination by matching many object images. The projection device can detect the object by using the matching image.

[0115] For example, when the illumination is dark, the projection device may project an image based on the first camera signal. When the illumination is dark, the projection period may include a time from the fourth time point T4 to a 1-5 time point T15. The projection device may project an image during the projection period from the fourth time point T4 to a 1-5 time point T15.

[0116] When the illumination is medium, the projection device may project an image based on the second camera signal. When the illumination is medium, the projection period may include a time from the fourth time point T4 to a 2-5 time point T25. The projection device may project an image during the projection period from the fourth time point T4 to a 2-5 time point T25.

[0117] When the illumination is bright, the projection device may project an image based on the third camera signal. When the illumination is bright, the projection period may include a time from the fourth time point T4 to a 3-5 time point T35. The projection device may project an image during the projection period from the fourth time point T4 to a 3-5 time point T35.

[0118] The time from the fourth time point T4 to the 1_5 time point T15 may be shorter than the time from the fourth time point T4 to the 2_5 time point T25. The time from the fourth time point T4 to the 2_5 time point T25 may be shorter than the time from the fourth time point T4 to the 3_5 time point T35. That is, the projection device may set the projection period to decrease as the illumination decreases.

[0119] According to an embodiment of the present disclosure, the projection device may set the length of the shooting period to decrease as the illumination increases. The projection device may set the length of the projection period to increase as the illumination increases. Therefore, the projection device may set the projection period and the shooting period so that the ratio of the projection period to the shooting period decreases as the illumination increases according to the illumination information.

[0120] According to one embodiment of the present disclosure, the shooting period may be shorter than the projection period. The projection device may repeatedly project images during the projection period. The projection device may repeatedly project images so that the projected images appear seamless to the human eye. Therefore, the projection device may set a minimum shooting period and an effective projection period so that the gaps between the projection periods are not observable to the human eye.

[0121] Figure 7 A diagram for describing an operation of a projection device to obtain a matching image by using an object image captured according to illumination according to an embodiment of the present disclosure is shown. Figures 1 to 6 Descriptions of overlapping elements are briefly described or not provided.

[0122] refer to Figure 7 In an embodiment of the present disclosure, the projection device may obtain an object image including the object. The projection device may obtain a matching image 740 based on the object image.

[0123] According to an embodiment of the present disclosure, the projection device may obtain illuminance information indicating the illuminance of an external projection surface on which image light is projected via a projection lens. When the illuminance is dark according to the illuminance information, the projection device may obtain a matching image 740 by matching many object images.

[0124] For convenience, the following description will now be made based on a case where the illumination is dark according to the illumination information, a case where the illumination is medium according to the illumination information, and a case where the illumination is bright according to the illumination information. Figure 7 However, the illuminance may indicate a relative illuminance. In addition, for convenience, the illuminance is divided into three cases, but the present disclosure is not limited thereto.

[0125] According to an embodiment of the present disclosure, when the illumination according to the illumination information is dark, the projection device can obtain the 1_1 object image 710. Figure 6 When the projection device is in the state of being illuminated, the projection device may obtain 1_1 object images 610 and 710 by photographing the object during the photographing period according to the first camera signal. Although the object is shown as a hand, this is merely an example and the present disclosure is not limited thereto.

[0126] According to an embodiment of the present disclosure, the projection device may identify an object based on an object image having a reference illumination requested for identifying the object. That is, based on the object image being lower than the reference illumination, the projection device may not be able to identify the object. The illumination of the 1_1 object image 710 may not meet the reference illumination. The illumination of the 1_1 object image 710 may be lower than the reference illumination.

[0127] The projection device may obtain the matching image 740 by matching at least two 1_1 object images 710. When the illumination is dark, the projection device may obtain the matching image 740 by matching multiple 1_1 object images 710. For example, the projection device may obtain the matching image 740 by matching three 1_1 object images 710. However, the number of 1_1 object images 710 used for matching is merely an example, and the present disclosure is not limited thereto.

[0128] According to an embodiment of the present disclosure, when the illumination according to the illumination information is medium, the projection device can obtain the 2_1 object image 720. Figure 6 When the projection device 200 is used, the projection device can obtain the 2_1 object image 720 by photographing the object during the photographing period according to the second camera signal. Although the object is shown as a hand, this is only an example and the present disclosure is not limited thereto.

[0129] According to an embodiment of the present disclosure, the illuminance of the 2_1 object image 720 may not satisfy the reference illuminance. The illuminance of the 2_1 object image 720 may be lower than the reference illuminance.

[0130] The projection device may obtain the matching image 740 by matching at least two 2_1 object images 720. When the illumination is medium, the projection device may obtain the matching image 740 by matching a plurality of 2_1 object images 720. For example, the projection device may obtain the matching image 740 by matching two 2_1 object images 720.

[0131] The number of 2_1 object images 720 for matching may be smaller than the number of 1_1 object images 710 for matching. That is, when the illumination of the 2_1 object image 720 is greater than the illumination of the 1_1 object image 710, the projection device may obtain the matching image 740 by matching a relatively small number of 2_1 object images 720. However, the number of 2_1 object images 720 for matching is merely an example, and the present disclosure is not limited thereto.

[0132] According to an embodiment of the present disclosure, when the illuminance according to the illuminance information is brighter, the projection device can obtain the 3_1 object image 730. Figure 6 When the projection device 730 is used, the projection device can obtain a 3_1 object image 730 by photographing the object during the photographing period according to the third camera signal. Although the object is shown as a hand, this is only an example and the present disclosure is not limited thereto.

[0133] According to an embodiment of the present disclosure, the illuminance of the 3_1 object image 730 may satisfy the reference illuminance.

[0134] The projecting device may obtain the matching image 740 via the 3_1 object image 730. The matching image 740 may be the same as the 3_1 object image 730. When the illumination is bright, the projecting device may obtain the matching image 740 via only one 3_1 object image 730.

[0135] That is, when the illumination of the object image is dark, the projection device may obtain the matching image 740 by matching a plurality of object images. However, when the illumination of the object image is appropriate, the projection device may use the object image as the matching image 740.

[0136] According to an embodiment of the present disclosure, when the illuminance according to the illuminance information is very bright, the projection device may obtain 4_1 the object image. The illuminance according to the illuminance information may exceed the reference illuminance.

[0137] The projection device may obtain the matching image 740 via the 4_1 object image. The projection device may obtain the 4_1 object image during a preset shooting period. When the illuminance according to the illuminance information exceeds the reference illuminance, the projection device may obtain the 4_1 object image satisfying the reference illuminance during a short shooting period. The projection device may use the 4_1 object image as the matching image 740.

[0138] That is, when the illuminance according to the illuminance information exceeds the reference illuminance, the projection device may obtain a 4_1 object image by photographing the object during a short photographing period as the illuminance increases. The projection device may use the obtained 4_1 object image as the matching image 740.

[0139] Figure 8 A flowchart illustrating an operation method of recognizing an object by matching captured images through a projection device thereof according to an embodiment of the present disclosure.

[0140] refer to Figure 8 In operation S810, the projection device may determine a matching amount based on the illumination information. The matching amount may indicate an amount of a matching object image of an object photographed by a camera.

[0141] According to an embodiment of the present disclosure, when the illuminance based on the illuminance information is darker, the projection device may determine a high number of the matching amount. For example, when the illuminance based on the illuminance information is darker, the projection device may determine 4 as the matching amount.

[0142] According to an embodiment of the present disclosure, when the illumination based on the illumination information is medium, the projection device may determine a medium amount of the matching amount. For example, when the illumination based on the illumination information is medium, the projection device may determine 3 as the matching amount. The matching amount for a case where the illumination based on the illumination information is medium may be less than the matching amount for a case where the illumination based on the illumination information is darker.

[0143] According to an embodiment of the present disclosure, when the illuminance based on the illuminance information is brighter, the projection device may determine a small amount of the matching amount. For example, when the illuminance based on the illuminance information is brighter, the projection device may determine 2 as the matching amount. The matching amount for a case where the illuminance based on the illuminance information is brighter may be smaller than the matching amount for a case where the illuminance based on the illuminance information is medium.

[0144] In operation S820, the projecting device may select some object images from among images repeatedly obtained according to a photographing cycle based on the matching amount.

[0145] According to an embodiment of the present disclosure, the projection device may control the camera to repeatedly photograph the object during the photographing period based on the photographing cycle (eg, Figure 4 Operation S420). The projecting device may obtain a plurality of object images by repeatedly photographing the object. The projecting device may select some object images from the plurality of object images based on the matching amount.

[0146] For example, when the illuminance based on the illuminance information is dark, the projection device may determine 4 as the matching amount. The projection device may select four object images from the plurality of object images based on the matching amount.

[0147] As another example, when the illuminance based on the illuminance information is medium, the projection device may determine 3 as the matching amount. The projection device may select three object images from the plurality of object images based on the matching amount.

[0148] As another example, when the illuminance based on the illuminance information is brighter, the projection device may determine the matching amount as 2. The projection device may select two object images from the plurality of object images based on the matching amount.

[0149] In operation S830, the projecting device may obtain a matching image based on the selected some object images.

[0150] According to an embodiment of the present disclosure, the projection device can obtain a matching image by matching an image selected from a plurality of object images. For example, when the illumination based on the illumination information is dark, the projection device can obtain a matching image by matching four object images selected from a plurality of object images.

[0151] As another example, when the illumination based on the illumination information is medium, the projection device can obtain a matching image by matching three object images selected from the plurality of object images. As another example, when the illumination based on the illumination information is bright, the projection device can obtain a matching image by matching two object images selected from the plurality of object images.

[0152] Fig. 9A diagram showing a projector among the elements of a projection device for describing in detail an embodiment of the present disclosure. In addition, the configuration based on digital light processing (DLP) projection will now be described. Fig. 9 , but this is only an example, and the present disclosure is not limited thereto. According to an embodiment of the present disclosure, a Fig. 9 The blocking period of projection of the light blocker 930 is configurable, but other elements are not particularly limited.

[0153] refer to Fig. 9 , the projection device may include a projector 900 and a camera.

[0154] According to an embodiment of the present disclosure, the projector 900 may include a light source unit (or light source) 910, a light separator 920, a light blocker 930, a mirror unit 940, and a projection lens 950. The projector 900 may project an image 960 on an external projection surface by using the light source unit 910, the light separator 920, the light blocker 930, the mirror unit 940, and the projection lens 950. However, the projector 900 is not limited to Fig. 9 , and may include a comparison Fig. 9 More elements than those shown in FIG. 1 or more elements than those shown in FIG. 1 may be included in projector 900. Fig. 9 Fewer components are shown.

[0155] According to an embodiment of the present disclosure, the projector 900 is configured to project an image to the outside. The projection according to the embodiment of the present disclosure can be implemented in various types (e.g., cathode ray tube (CRT) type, liquid crystal display (LCD) type, DLP type, laser type, etc.). For example, the CRT type has a principle substantially the same as that of a CRT monitor. The CRT type displays an image on a screen by magnifying the image via a lens in front of the CRT. According to the number of CRTs, the CRT type is divided into a single CRT type and a three-CRT type, and in the three-CRT type, CRTs for red, green, and blue are implemented respectively.

[0156] As another example, the LCD type displays an image by transmitting light from a light source to liquid crystal. The LCD type is divided into a single-panel type and a three-panel type, and in the three-panel type, the light from the light source can be separated into red, green, blue via a dichroic mirror (e.g., a mirror that reflects light with a specific color and allows light of other colors to pass through), can pass through the liquid crystal, and can then be collected.

[0157] As another example, the DLP type is a type that displays an image by using a digital micromirror device (DMD) chip. The projection unit of the DLP type may include a light source, a color wheel, a DMD chip, a projection lens, etc. The light output from the light source may have a color by passing through a color wheel. The light passing through the color wheel is input to the DMD chip. The DMD chip includes many micromirrors and reflects the light input to the DMD chip. The projection lens may be configured to magnify the light reflected from the DMD chip to an image size.

[0158] As another example, the laser type includes a diode pumped solid-state (DPSS) laser and a galvanometer. As a laser for outputting various colors, a laser in which three DPSS lasers are provided for each RGB color and the optical axes overlap by using a special mirror is used. The galvanometer includes a mirror and a high-output motor to move the mirror at a fast speed. For example, the galvanometer can rotate the mirror at a maximum of 40kHz. The galvanometer is installed along the scanning direction, and when the projector performs plane scanning, the galvanometer can be set at the X-axis and the Y-axis.

[0159] According to an embodiment of the present disclosure, the light source unit 910 may output the image light L. The light source unit 910 may output the image light L to be emitted to corresponding specific areas of the light separator 920 and the light blocker 930 .

[0160] According to an embodiment of the present disclosure, the light separator 920 can convert the output image light L into red (R), blue (B), and green (G) by transmitting the output image light L. Specifically, as shown in the figure, the light separator 920 may include a color filter for allowing the corresponding red, blue, and green light beams to selectively pass. The light separator 920 may include: a rotating plate, each color filter can be rotatably mounted on the rotating plate; and a driving motor for rotatably driving the rotating plate.

[0161] According to an embodiment of the present disclosure, the light separator 920 may be controlled so that when the rotating plate is rotated by the driving motor, the red filter, the blue filter, and the green filter are sequentially arranged on the optical path along which the image light L travels. Therefore, the projection apparatus may separate the image light L output from the light source unit 910 into the red image light, the blue image light, and the green image light.

[0162] According to an embodiment of the present disclosure, the light separator 920 may indicate a DLP type color wheel.

[0163] According to an embodiment of the present disclosure, the light blocker 930 may block a separate image light. In detail, the light blocker 930 may include a light blocking filter 931 and a light transmitting filter 932. The light blocker 930 may include a rotating plate on which the light blocking filter 931 and the light transmitting filter 932 are rotatably mounted and a driving motor for rotatably driving the rotating plate.

[0164] According to an embodiment of the present disclosure, the light blocker 930 may be controlled so that when the rotating plate is rotated by the driving motor, the light blocking filter 931 and the light transmitting filter 932 are alternately arranged on the optical path on which the image light L travels. Therefore, the projection device may block the image light output from the light source unit 910 during the blocking period. In addition, the projection device may transmit the image light output from the light source unit 910 during the projection period.

[0165] According to an embodiment of the present disclosure, the mirror unit 940 may reflect the image light L transmitted by the light blocker 930. The mirror unit 940 may be an optical semiconductor composed of a plurality of micro mirrors. For example, the mirror unit 940 may indicate a DLP type DMD chip.

[0166] According to an embodiment of the present disclosure, the mirror unit 940 may reflect the image light L transmitted by the light separator 920 and the light blocker 930 based on color (RGB) information composed of an image. The mirror unit 940 may reflect the image light L based on color information composed of an image to correspond to the image.

[0167] According to an embodiment of the present disclosure, the projection lens 950 may magnify the image light L reflected from the mirror unit 940. The projection device may project an image 960 by projecting the magnified image light on an external projection surface.

[0168] Fig.10 A diagram is shown for describing in detail a light blocker among elements of a projection device according to an embodiment of the present disclosure.

[0169] Reference Fig.10 The light blocker 930 may include a light blocking filter 931 , a light transmitting filter 932 , a rotating plate 933 , and a driving motor 934 .

[0170] According to an embodiment of the present disclosure, the light blocking filter 931 may block the image light L output from the light source unit 910 . The light transmitting filter 932 may transmit the image light L output from the light source unit 910 .

[0171] The light transmission filter 932 and the light blocking filter 931 may be connected to each other. As shown in the figure, the light transmission filter 932 and the light blocking filter 931 may be mounted on a rotating plate 933. The rotating plate 933 may be rotated by a driving motor 934. When the rotating plate 933 rotates, the light transmission filter 932 and the light blocking filter 931 may be alternately arranged on the optical path where the image light L travels.

[0172] The driving motor 934 may rotatably drive the rotating plate 933. For example, the driving motor 934 may provide power to allow the rotating plate 933 to rotate once in a constant direction. The driving motor 934 may provide power based on a preset signal to allow the rotating plate 933 to rotate once in a constant direction. When the preset signal is repeated, the driving motor 934 may provide power to allow the rotating plate 933 to repeatedly perform an operation of rotating once.

[0173] According to an embodiment of the present disclosure, the driving motor 934 may provide power to allow the rotating plate 933 to rotate. When the rotating plate 933 rotates, the light transmission filter 932 and the light blocking filter 931 may be alternately arranged on an optical path where the image light L travels.

[0174] Reference Fig.10 When the rotating plate 933 rotates, the light transmission filter 932 may be arranged on the optical path. When the rotating plate 933 rotates, the image light L on the optical path may be transmitted through the first area A1 on the light transmission filter 932. The light transmission filter 932 may transmit the image light L traveling along the optical path. The transmitted image light L may be transmitted through the projection lens 950 (see, for example, Fig. 9 ) is projected outward. In other words, the projection device can project an image.

[0175] Fig.11 A diagram showing a light blocker in an element of a projection device according to an embodiment of the present disclosure is shown in detail. Fig.10 Description of overlapping content.

[0176] Reference Fig.11 , when the rotating plate 933 rotates, the blocking filter 931 may be provided on the optical path. When the rotating plate 933 rotates, the image light L on the optical path may be blocked due to the second area A2 of the blocking filter 931. The blocking filter 931 may block the image light L traveling along the optical path. When the image light L is blocked, the image light L may be projected to the outside via the projection lens 950. That is, the blocking period may start.

[0177] According to an embodiment of the present disclosure, when the image light L is blocked by the light blocking filter 931, the projection device may not project an image. The image light L being blocked by the light blocking filter 931 may indicate a blocking period. That is, the projection device may not project an image during the blocking period. The projection device may obtain an object image by photographing an object located on an external projection surface using a camera during a photographing period included in the blocking period.

[0178] According to an embodiment of the present disclosure, the projection device may detect an object included in the object image based on the object image. According to an embodiment of the present disclosure, the projection device may obtain a matching image based on the object image. The projection device may detect an object included in the matching image based on the matching image.

[0179] Fig.12 A conceptual diagram for describing a feature of determining the operation of a projection device based on the operation of a light blocker according to an embodiment of the present disclosure is shown. Figure 1 , Fig.10 and Fig.11 Descriptions of overlapping content are briefly described or not provided.

[0180] refer to Fig.12 According to an embodiment of the present disclosure, the projection device 100 may respectively perform an operation of projecting an image through a light blocker (S1210) and an operation of photographing an object (S1220). Figure 1 , Fig.10 and Fig.11 Descriptions of overlapping content are briefly described or not provided.

[0181] In operation S1210, the projection device 100 may project an image. The operation of the light blocker corresponding to operation S1210 may be similar to that described in reference to FIG. Fig.10 The described operation is the same. When the rotating plate 933 of the light blocker 930 rotates, the light transmission filter 932 may be provided on the optical path of the image light L. The image light L may be transmitted by the light transmission filter 932. The transmitted image light L may be projected to the outside via the projection lens 950. That is, the projection device 100 may project the image 120.

[0182] According to an embodiment of the present disclosure, a period during which the image light L is transmitted by the light-transmitting filter 932 may indicate a projection period. The projection device 100 may project the image 120 during the projection period.

[0183] In operation S1220, the projection device 100 may photograph the subject. The operation of the light blocker 930 corresponding to operation S1220 may be similar to the operation described in the reference Fig.11 When the rotating plate 933 of the light blocker 930 rotates, the light blocking filter 931 may be provided on the optical path along which the image light L travels. The image light L may be blocked by the light blocking filter 931. That is, the projection device 100 may not project the image 120.

[0184] According to an embodiment of the present disclosure, the projection device 100 may photograph the object 160 while the image light L is blocked. The projection device 100 may obtain an object image including the object 160. The projection device 100 may detect the object 160 included in the object image based on the object image.

[0185] According to an embodiment of the present disclosure, a period during which the image light L is blocked by the light blocking filter 931 may indicate a blocking period. The projection device 100 may set a photographing period to be included in the blocking period. The projection device 100 may control the camera to photograph the object 160 during the photographing period.

[0186] According to an embodiment of the present disclosure, the projection device 100 may receive a preset clock signal to control the light blocker 930 to rotate. The projection device 100 may receive a repeated clock signal to control the light blocker 930 to rotate repeatedly based on the signal. The projection device may control the light blocker 930 to alternately repeat the operation of S1210 and the operation of S1220.

[0187] When the operation of S1220 is repeatedly performed, the projection device 100 may obtain a plurality of object images including the object 160. The projection device 100 may obtain a matching image by matching the plurality of object images. The projection device 100 may detect the object 160 included in the matching image based on the matching image.

[0188] Fig.13 A graph showing an operation of a projection device according to an embodiment of the present disclosure to photograph an object by using different photographing cycles according to illumination. Figure 3 and Figure 6 Descriptions of overlapping elements are briefly described or not provided.

[0189] refer to Fig.13 , the projection device may obtain a projection signal and a plurality of camera signals. The projection signal and the plurality of camera signals received by the projection device may not be synchronized.

[0190] According to an embodiment of the present disclosure, the projection signal may start at an initial time point T0, and 0 and 1 may be repeated at regular intervals. In contrast, the first camera signal and the second camera signal may not start at the initial time point T0. For example, as shown in the figure, the first camera signal and the second camera signal may start at a first time point T1, and 0 and 1 may be repeated at regular intervals.

[0191] refer to Figure 6 According to an embodiment of the present disclosure, the projection signal and the plurality of camera signals may start at the same time at the first time point T1 and may be repeated at regular intervals. That is, the projection signal and the plurality of camera signals may be synchronized.

[0192] On the contrary, Fig.13 As shown, the projection signal and the plurality of camera signals received by the projection device may not be synchronized with each other.

[0193] According to an embodiment of the present disclosure, when the illumination is dark, the projection device can obtain a first camera signal. When the illumination is bright, the projection device can obtain a second camera signal.

[0194] According to an embodiment of the present disclosure, the projection device may shoot the object after a preset time has passed since the time point T1 at which the camera signal is obtained. Here, the camera signal may include a first camera signal and a second camera signal. The preset time may refer to the time from the first time point T1 to the second time point T2.

[0195] According to an embodiment of the present disclosure, the projection device may set a blocking period in which the image light output by the light source unit is blocked. The projection device may set the blocking period to repeat according to a preset interval. As shown in the figure, when the illumination is dark, the blocking period may include a time from the second time point T2 to a 1-3 time point T13. When the illumination is bright, the blocking period may include a time from the second time point T2 to a 2-3 time point T23.

[0196] According to an embodiment of the present disclosure, the projection device may set a photographing period in which a subject is photographed using a camera. The photographing period may be included in the blocking period.

[0197] For example, when the illumination is dark, the projection device may shoot the object based on the first camera signal.When the illumination is dark, the shooting period may include the time from the second time point T2 to the 1_3 time point T13.

[0198] When the illumination is bright, the projection device may shoot the object based on the second camera signal.When the illumination is bright, the shooting period may include a time from the second time point T2 to a 2_3 time point T23.

[0199] According to an embodiment of the present disclosure, the projection device may determine the length of the projection period according to the illumination. The projection device may determine the length of the projection period based on illumination information indicating the illumination of the external projection surface on which the image is projected. The projection device may set the length of the projection period to decrease as the illumination decreases.

[0200] For example, when the illumination is dark, the projection device may photograph the object based on the first camera signal. When the illumination is dark, the projection period may include a time from a 1_3 time point T13 to a 1_4 time point T14.

[0201] When the illumination is bright, the projection device may photograph the object based on the second camera signal.When the illumination is bright, the projection period may include a time from a 2_3 time point T23 to a 2_4 time point T24.

[0202] Fig.14 A block diagram for describing a configuration of a projection device according to an embodiment of the present disclosure is shown.

[0203] refer to Fig.14 , the projection device 1000 may include a processor 1100, a memory 1200, a camera 1300, and a projector 1400. The projector 1400 may include a light source unit 1410, a light blocker 1420, and a projection lens 1430. The processor 1100, the memory 1200, the camera 1300, and the projector 1400 may be electrically and / or physically connected to each other.

[0204] However, Fig.14 The elements shown in FIG. 1 are based only on the embodiments of the present disclosure, and therefore, the elements included in the projection device 1000 are not limited to the following. Fig.14 The projecting device 1000 may not include Fig.14 Some of the elements shown in, or may further include Fig.14 For example, the projection device 1000 may further include a power source (eg, a battery) for supplying driving power to the camera 1300 , the projector 1400 , the processor 1100 , and the memory 1200 .

[0205] The light source unit 1410 may output image light. The image light may include white light.

[0206] The light blocker 1420 may block the image light output from the light source unit 1410. The light blocker 1420 may block the image light during a preset period, and may project the image light during a different preset period.

[0207] According to an embodiment of the present disclosure, the light blocker 1420 may include a blocking filter for blocking image light, a projection filter for projecting image light, and a drive motor for rotating the blocking filter and the projection filter. The light blocker 1420 may be provided as a rotatable wheel. When the light blocker 1420 is rotated by the drive motor, the image light incident on the light blocker 1420 may be blocked by the blocking filter or may be projected by the projection filter.

[0208] The image light projected by the light blocker 1420 may be projected to the outside via the projection lens 1430. That is, the processor 1100 may project an image by projecting the image light to the outside using the projection lens 1430.

[0209] The camera 1300 may photograph an object located on the external projection surface. The processor 1100 may obtain an object image of the object by using the camera 1300.

[0210] According to an embodiment of the present disclosure, the processor 1100 may set a blocking period in which image light is blocked by the light blocker 1420. The processor 1100 may set a photographing period to be included in the blocking period. The processor 1100 may obtain an object image by using the camera 1300 during the photographing period.

[0211] The processor 1100 may execute one or more instructions or program codes stored in the memory 1200, and may perform functions and / or operations corresponding to the instructions or program codes. The processor 1100 may include hardware elements for performing arithmetic, logic and input / output calculations and signal processing. The processor 1100 may include, for example, at least one of a CPU, a microprocessor, a graphics processor (graphics processing unit (GPU)), an AP, an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), or a field programmable gate array (FPGA), but the present disclosure is not limited thereto.

[0212] Although the processor 1100 Fig.14 In the embodiment of the present disclosure, the processor 1100 may include one or more processors.

[0213] In an embodiment of the present disclosure, the processor 1100 may be configured as a dedicated hardware chip for performing AI learning.

[0214] The memory 1200 may store instructions and program codes that can be read by the processor 1100. The memory 1200 may include at least one of a flash memory type, a hard disk type, a micro multimedia card type, a memory card type (e.g., a secure digital (SD) or extreme digital (XD) memory card), a random access memory (RAM) (a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a mask ROM, a flash ROM, etc.), a hard disk drive (HDD), or a solid state drive (SSD).

[0215] The memory 1200 may store instructions or program codes for executing functions or operations of the projection device 1000. According to an embodiment of the present disclosure, the memory 1200 may store instructions, algorithms, data structures, program codes, or applications that can be read by the processor 1100. The instructions, algorithms, data structures, and program codes stored in the memory 1200 may be implemented in, for example, a programming or scripting language such as C, C++, Java, assembler, etc.

[0216] The memory 1200 may store instructions, algorithms, data structures, or program codes for a model for setting a blocking period for blocking image light and controlling a camera to photograph an object during a photographing period included in the blocking period. A “module” included in the memory 1200 refers to a unit for processing a function or operation performed by the processor 1100, which may be implemented in software such as instructions, algorithms, data structures, or program codes.

[0217] In the following embodiments, the processor 1100 may be implemented by executing instructions or program codes stored in the memory 1200 .

[0218] According to an embodiment of the present disclosure, the processor 1100 may obtain illumination information indicating the illumination of the external projection surface. The processor 1100 may set a blocking period during which the output image light is blocked based on the illumination information. The processor 1100 may set a shooting period to be included in the blocking period. The processor 1100 may control the light blocker 1420 to block the output image light during the blocking period. The processor 1100 may control the camera 1300 to shoot the object during the shooting period.

[0219] According to an embodiment of the present disclosure, a projection device for identifying an object by image projection may include a light source unit, a projection lens, a camera, a light blocker, and at least one processor. The light source unit may be configured to output image light. The output image light may be projected to the outside via the projection lens. The camera may be configured to capture an object between the camera and an external projection surface of the projected image light. The light blocker may be configured to block the image light output by the light source during a blocking period.

[0220] According to an embodiment of the present disclosure, the processor 1100 may obtain illumination information about the illumination of the external projection surface. The processor 1100 may set a blocking period during which the image light output by the light source is blocked based on the illumination information. The processor 1100 may set a shooting period within the blocking period. The processor 1100 may control the light blocker 1420 to block the image light output by the light source during the blocking period. The processor 1100 may control the camera 1300 to shoot the object during the shooting period.

[0221] In an embodiment of the present disclosure, the starting point of the photographing period may match the starting point of the blocking period.

[0222] In an embodiment of the present disclosure, the shooting period may include a first shooting period and a second shooting period after the first shooting period. The at least one processor may also be configured to determine a shooting cycle from the beginning of the first shooting period to the beginning of the second shooting period. The at least one processor may also be configured to control the camera to repeatedly shoot the object during the shooting period based on the shooting cycle.

[0223] In an embodiment of the present disclosure, a shooting period and a projection period that does not overlap with the shooting period are repeated during a shooting cycle. The shooting period may be shorter than the projection period.

[0224] In an embodiment of the present disclosure, the at least one processor may be further configured to set the projection period and the photographing period such that a ratio of the photographing period to the projection period decreases as the illuminance according to the illuminance information increases.

[0225] In an embodiment of the present disclosure, the at least one processor may be further configured to: based on the illuminance information, set the photographing period to decrease when the illuminance according to the illuminance information increases.

[0226] In an embodiment of the present disclosure, at least one processor may also be configured to obtain a first object image of the object by controlling the camera to photograph the object during a first photographing period. At least one processor may also be configured to obtain a second object image of the object by controlling the camera to photograph the object during a second photographing period. At least one processor may also be configured to obtain a matching image based on the first object image and the second object image.

[0227] In an embodiment of the present disclosure, the illuminance of the matching image may be greater than the illuminance of the first object image and the illuminance of the second object image.

[0228] In an embodiment of the present disclosure, at least one processor may also be configured to determine a matching amount based on the illumination information, the matching amount indicating the amount of matching object images of the object captured by the camera. At least one processor may also be configured to: based on the matching amount, select an object image from a plurality of object images repeatedly captured according to a capture cycle. At least one processor may also be configured to obtain a matching image based on the selected object image. The matching amount may decrease when the illumination according to the illumination information increases.

[0229] In an embodiment of the present disclosure, the light blocker may include a light-transmitting filter configured to transmit image light output by the light source. The light blocker may include a light-blocking filter connected to the light-transmitting filter and configured to block the image light output by the light source. The light blocker may include a rotating plate on which the light-transmitting filter and the light-blocking filter are rotatably mounted. The light blocker may include a drive motor configured to rotatably drive the rotating plate so that the light-transmitting filter and the light-blocking filter are alternately arranged on the optical path of the image light.

[0230] In an embodiment of the present disclosure, when the light blocker rotates during the blocking period, the image light is blocked by the light blocking filter. When the light blocker rotates during the projection period, the image light is transmitted by the light transmitting filter.

[0231] According to an embodiment of the present disclosure, a method for identifying an object using image projection may include: obtaining illumination information about the illumination of an external projection surface of image light projected via a projection lens. The method may include setting a blocking period during which image light output by a light source unit is blocked based on the illumination information. The method may include setting a shooting period within the blocking period. The method may include blocking the output image light during the blocking period. The method may include shooting an object located on the external projection surface by using a camera during the shooting period.

[0232] In an embodiment of the present disclosure, the shooting period may include a first shooting period and a second shooting period after the first shooting period. Setting the shooting period may include determining a shooting cycle from the start of the first shooting period to the start of the second shooting period. The shooting of the object may include repeatedly shooting the object during the shooting period based on the shooting cycle.

[0233] In an embodiment of the present disclosure, the photographing cycle may be a cycle in which a photographing period and a projection period not overlapping with the photographing period are repeated. The photographing period may be shorter than the projection period.

[0234] In an embodiment of the present disclosure, setting the photographing period may include setting the photographing period and the projection period such that a ratio of the photographing period to the projection period decreases as the illuminance according to the illuminance information increases.

[0235] In an embodiment of the present disclosure, setting the photographing period may include, based on the illuminance information, setting the photographing period to decrease when the illuminance according to the illuminance information increases.

[0236] In an embodiment of the present disclosure, repeatedly photographing the object may include obtaining a first object image of the object by photographing the object during a first photographing period. Repeating photographing the object may include obtaining a second object image of the object by photographing the object during a second photographing period. The method further includes obtaining a matching image based on the first object image and the second object image.

[0237] In an embodiment of the present disclosure, the illuminance of the matching image may be greater than the illuminance of the first object image and the illuminance of the second object image.

[0238] In an embodiment of the present disclosure, the method may further include determining a matching amount indicating an amount of matching object images of the photographed object based on the illumination information. The method may further include: selecting an object image from a plurality of object images repeatedly captured according to a photographing cycle based on the matching amount. The method may further include obtaining a matching image based on the selected object image. The matching amount may decrease when the illumination according to the illumination information increases.

[0239] According to an embodiment of the present disclosure, there is provided a computer-readable recording medium having recorded thereon a program for executing the method on a computer.

[0240] The machine-readable storage medium may be provided in the form of a non-transitory storage medium. In this regard, the term "non-transitory storage medium" only means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), and the term does not distinguish between a case where data is semi-permanently stored in the storage medium and a case where data is temporarily stored in the storage medium. For example, a non-transitory storage medium may include a buffer in which data is temporarily stored.

[0241] According to an embodiment of the present disclosure, the method according to various embodiments disclosed in this specification may be included and provided in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or distributed online (e.g., downloaded or uploaded) via an application store, or distributed directly between two user devices (e.g., smart phones). For online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily generated or at least temporarily stored in a machine-readable storage medium (e.g., a memory of a manufacturer's server, an application store's server, or a relay server).

Claims

1. A projection device, configured to identify an object using image projection, the projection device comprising: a light source configured to output image light; a projection lens configured to project image light to the outside; a camera configured to capture an object between the camera and an external projection surface onto which the image light is projected; a light blocker configured to block image light output by the light source during a blocking period; as well as at least one processor configured to control the operation of the light source, the camera, and the light blocker, Wherein, at least one processor is further configured to: obtaining illumination information about the illumination of an external projection surface, setting a blocking period during which image light output by the light source is blocked based on the illuminance information, Set the shooting period within the blocking period. controlling the light blocker to block the image light output by the light source during the blocking period, and The camera is controlled to capture the subject during the capture period.

2. The projection device according to claim 1, wherein: The start point of the shooting period matches the start point of the blocking period.

3. The projection device according to any one of claims 1 and 2, wherein: The shooting period includes a first shooting period and a second shooting period after the first shooting period, Wherein, at least one processor is further configured to: determining a shooting period from the start of the first shooting period to the start of the second shooting period, and Based on the photographing cycle, the camera is controlled to repeatedly photograph the subject during the photographing period.

4. The projection device according to claim 3, wherein: During the shooting cycle, a shooting period and a projection period not overlapping with the shooting period are repeated, and The shooting period is shorter than the projection period.

5. The projection device according to any one of claims 1 to 4, wherein: The at least one processor is further configured to, based on the illuminance information, set the photographing time period to decrease as the illuminance according to the illuminance information increases.

6. The projection device according to any one of claims 3 to 5, wherein: The at least one processor is further configured to: obtaining a first object image of the object by controlling the camera to photograph the object during a first photographing period, obtaining a second object image of the object by controlling the camera to photograph the object during a second photographing period, and A matching image is obtained based on the first object image and the second object image.

7. The projection device according to any one of claims 3 to 6, wherein: The at least one processor is further configured to: determining a matching amount based on the illumination information, the matching amount indicating an amount of a matching object image of the object captured by the camera, selecting, based on the matching amount, a subject image from a plurality of subject images that are repeatedly captured according to a photographing cycle; and Obtaining a matching image based on the selected object image, Here, the matching amount decreases when the illuminance according to the illuminance information increases.

8. The projection device according to any one of claims 1 to 7, wherein: Light blockers include: a light transmission filter configured to transmit image light output by the light source; a light blocking filter connected to the light transmitting filter and configured to block image light output by the light source; a rotating plate on which the light transmission filter and the light blocking filter are rotatably mounted; and The driving motor is configured to rotatably drive the rotating plate so that the light transmission filter and the light blocking filter are alternately arranged on an optical path along which the image light travels.

9. The projection device according to claim 8, wherein: When the light blocker rotates during the blocking period, the image light is blocked by the light blocking filter, and Among them, when the light blocker rotates during the projection period, the image light is transmitted by the light transmission filter.

10. A method for identifying an object using image projection, the method comprising: obtaining illumination information regarding an illumination of an external projection surface on which image light is projected; setting a blocking period during which output image light is blocked based on the illuminance information; Set up shooting time periods within the blocking period; blocking the output image light during a blocking period; as well as An object located on the external projection surface is photographed during the photographing period.

11. The method according to claim 10, wherein: Setting the photographing period includes, based on the illuminance information, setting the photographing period to decrease when the illuminance according to the illuminance information increases.

12. The method according to any one of claims 10 to 11, wherein: Subjects include: obtaining a first object image of the object by photographing the object during a first photographing period; and obtaining a second object image of the object by photographing the object during a second photographing period, and The method further includes obtaining a matching image based on the first object image and the second object image.

13. The method according to claim 12, wherein: The illuminance of the matching image is greater than the illuminance of the first object image and the illuminance of the second object image.

14. The method according to any one of claims 10 to 13, further comprising: determining a matching amount indicating an amount of a matching object image of a photographed object based on the illumination information; selecting, based on the matching amount, an object image from a plurality of object images that are repeatedly captured according to a photographing cycle; as well as obtaining a matching image based on the selected object image, and Here, the matching amount decreases when the illuminance according to the illuminance information increases.

15. A computer-readable recording medium having recorded thereon a program for executing the method according to any one of claims 10 to 14 on a computer.

Images