Image projection apparatus and image projection control method

Abstract

The image projection device may include: a ToF sensor configured to measure a distance to a target; a lamp assembly configured to output an image signal including a pattern image; the driving part is used for controlling the lens assembly to move along an axial direction; and a processor controlling to measure a distance to a screen through the ToF sensor and project the output pattern image to the screen through the lamp assembly. The processor may calculate a sharpness value of a pattern image projected to the screen by the lamp assembly, and may adjust a size of the pattern image and a position at which the pattern image is projected based on the calculated sharpness value.

Description

Technical Field

[0001] The present invention relates to an image projection device and an image projection control method, and specifically relates to an automatic correction method for an image projection device. Background Art

[0002] An image projection device such as a projector is configured to be connected to a computer or an electronic device and project an input image onto a screen. In recent years, as image projection devices such as projectors have become smaller, they can be installed not only in a fixed place but also in a movable manner. In this regard, whenever a movable image projection device is installed in a specific place, there is a problem that the user needs to adjust the focus, screen configuration / size, and clarity by himself.

[0003] Therefore, there is a problem that the size, configuration, and clarity of the projected image projected onto the screen may change depending on the user's proficiency in setting up and operating the image projection device. In order to solve this problem, it is necessary to control the image projection device so that it automatically adjusts the focus, screen configuration / size, and clarity every time the image projection device is set up or power is applied. Summary of the invention

[0004] Problems to be solved by the invention

[0005] This specification is used to solve the above problems and other problems. One object of the present invention is to provide an image projection device and an image projection control method.

[0006] Another object of the present invention is to provide an automatic calibration method for an image projection device.

[0007] Another object of the present invention is to automatically adjust the focus, screen configuration / size and clarity of a movable image projection device whenever it is placed in a specific location.

[0008] Another object of the present invention is to control the image projection device to automatically adjust the focus, screen configuration / size and clarity whenever the image projection device is set up or power is applied.

[0009] Technical solutions to the problem

[0010] In order to achieve the above or other purposes, the image projection device of the embodiment may include: a ToF sensor configured to measure the distance to the target; a lamp assembly configured to output an image signal including a pattern image; a driving unit controlled to move the lens assembly along an axial direction; and a processor controlled to measure the distance to the screen through the ToF sensor and project the output pattern image onto the screen through the lamp assembly. The processor may calculate the sharpness value of the pattern image projected onto the screen by the lamp assembly, and may adjust the size of the pattern image and the position of projecting the pattern image based on the calculated sharpness value.

[0011] According to an embodiment, the processor may control the driving part to move the lamp assembly based on the measured distance and the focus position from the screen.

[0012] According to an embodiment, the image projection device may further include a camera configured to capture an image of the screen or a mark associated with the pattern image. The processor may detect the coordinates of a plurality of corner points of the screen or the coordinates of a plurality of corner points of a mark associated with the pattern image and a credibility value associated with the detection accuracy of the coordinates based on deep learning, and may determine which of the plurality of corner points is the optimal corner point for detecting the screen or the mark by detecting the linear components in the vertical and horizontal directions based on the plurality of corner points. The processor may control the drive unit to move the lens assembly based on the distance from the midpoint of the ToF sensor to the optimal corner point.

[0013] According to an embodiment, if the screen detection based on deep learning fails, the processor can be controlled to generate a mark related to the pattern image and project the mark through the light assembly, and the coordinates of multiple corner points of the projected mark and the credibility value can be detected through the camera.

[0014] According to an embodiment, the processor can determine whether the lines of the upper end, the lower end, one end, and the other end of the screen can be obtained from the point of view of the camera, and can determine whether the surface of the screen is displayed as a uniform color within a specified range, and can determine whether the display area of ​​the screen, i.e., the first area, does not have a portion blocked by a person or other objects. If it is determined that the lines of the screen can be obtained and the surface is displayed as a uniform color and the first area does not have a blocked portion, the processor can detect the coordinates of the plurality of corner points of the screen and the credibility value based on deep learning.

[0015] According to an embodiment, the processor may measure the distance from the midpoint of the ToF sensor to the screen, and if the measured distance is less than the ToF effective distance, the processor may move the light assembly to a first position adjacent to the target position along a specified direction and measure the first sharpness value of the pattern image, and if the measured distance is greater than the ToF effective distance, the processor may move the light assembly to a second position corresponding to the ToF effective distance and measure the second sharpness value of the pattern image. The processor may adjust the size of the pattern image and the position at which the pattern image is projected based on the measured first and second sharpness values.

[0016] According to an embodiment, the processor may compare the display area of ​​the screen, i.e., the first area, and the projection area, i.e., the second area, where the pattern image is projected, and may control to perform a first zoom action to move the focus position to the first position, so that the second area is formed to be larger than the first area by a specified ratio or more. The processor may extract a first center point of the first area and a second center point of the second area, and may adjust the center position of the pattern image so that the second center point of the second area is moved to the first center point of the first area, and may perform a second zoom action to move the focus position to the second position when the center position is adjusted.

[0017] According to an embodiment, the processor can measure the first sharpness value or the second sharpness value by the first zoom action at a first interval in an axial direction, and can measure the first sharpness value or the second sharpness value by the second zoom action at a second interval narrower than the first interval in the axial direction, and while the center position is adjusted, fine-adjustment can be performed to move the focus to the second position.

[0018] According to an embodiment, the processor may determine a mapping matrix P, which defines a conversion relationship between a source image frame of the pattern image and a projection image frame projected onto the screen, and may determine a pre-warping matrix W, which enables the projection image frame to have a prescribed shape on the screen. The processor may be controlled to convert the source image frame based on the pre-warping matrix W, and may enable the converted source image frame to be projected onto the screen in the prescribed shape.

[0019] According to an embodiment, the processor may detect four corner points by detecting the screen, and may determine a first mapping matrix T based on the detected four corner points, wherein the first mapping matrix T defines a conversion relationship between the source image frame and the camera image frame.

[0020] According to an embodiment, the processor may determine a second mapping matrix C based on the detected four corner points, wherein the second mapping matrix C enables the projection image frame projected onto the detected screen to become the prescribed shape from the perspective of the camera.

[0021] According to an embodiment, the processor may determine the mapping matrix P as C based on the first mapping matrix T and the second mapping matrix C. -1 The processor may determine an optimal point for adjusting an offset and a scale of the rectified screen modified into the specified shape, and may determine whether the offset and the scale of the modified screen are appropriate based on the coordinates of the optimal point and the coordinates of the corresponding points of the screen before being modified into the specified shape.

[0022] According to an embodiment, the processor may apply the inverse transformation C of the second mapping matrix C to the camera image frame. -1 The projected image frame before being modified into the image of the prescribed shape (rectified screen) is estimated.

[0023] According to an embodiment, the processor may control the image frame converted by multiplying the source image frame by the pre-deformation matrix W to be projected onto the screen as a projection image frame of the prescribed shape.

[0024] According to an embodiment, the processor may perform an evaluation related to the degree to which the projection image frame is modified into the prescribed shape by detecting a plurality of corner points of the projection image frame. The processor may modify the mapping matrix P to P1 based on the distance difference between the coordinates of the plurality of detected corner points and the coordinates of the plurality of optimal corner points for modification into the prescribed shape, and may modify the pre-deformation matrix to W1 based on the modified mapping matrix P1. The processor may be controlled to transform the source image frame based on the modified pre-deformation matrix W1, and the transformed source image frame is projected onto the screen in the prescribed shape.

[0025] According to an embodiment, the image projection device may further include a user input unit configured to receive user input to implement correction of the pattern image projected onto the screen. The processor may be controlled to display the projection image frame projected onto the screen in the prescribed shape and to maintain the center position of the projection image frame at a position within a prescribed distance from the center position on one axis of the screen based on the user input.

[0026] Another embodiment of the present invention is to control an image projection control method for outputting a pattern image, which can be executed by a processor of a projector. The control method may include: a screen / marker detection process, detecting a screen or a mark associated with the pattern image; a drive control process, based on the distance to the screen and the focus position from the screen measured by a ToF sensor, controlling a drive unit to move a lamp assembly; a sharpness value calculation process, calculating the sharpness value of the pattern image projected to the screen by the lamp assembly; and a pattern image size / position adjustment process, adjusting the size of the pattern image and the position of projecting the pattern image based on the calculated sharpness value.

[0027] According to an embodiment, during the screen / marker detection process, the coordinates of a plurality of corner points of the screen or the coordinates of a plurality of corner points of a mark associated with the pattern image and a credibility value associated with the detection accuracy of the coordinates can be detected based on deep learning, and it can be determined which corner point among the plurality of corner points is the optimal corner point for detecting the screen or the mark by detecting straight line components in the vertical and horizontal directions based on the plurality of corner points.

[0028] According to an embodiment, the control method may further include a zoom / position adjustment process, which compares the display area of ​​the screen, i.e., the first area, and the projection area, i.e., the second area, where the pattern image is projected, and adjusts the position after executing the sharpness value calculation process. In the zoom / position adjustment process, it may be controlled to perform a first zoom action to move the focus position to the first position, so that the second area is formed to be larger than the first area by a specified ratio or more. In the zoom / position adjustment process, the first center point of the first area and the second center point of the second area may be extracted, and the center position of the pattern image may be adjusted so that the second center point of the second area is moved to the first center point of the first area, and in the state where the center position is adjusted, a second zoom action may be performed to move the focus position to the second position.

[0029] According to an embodiment, during the pattern image size / position adjustment process, a mapping matrix P may be determined, the mapping matrix P defining the conversion relationship between the source image frame of the pattern image and the projection image frame projected onto the screen, and a pre-warping matrix W may be determined, the pre-warping matrix W enabling the projection image frame to have a prescribed shape on the screen. During the pattern image size / position adjustment process, the source image frame may be controlled to be converted based on the pre-warping matrix W, and the converted source image frame may be projected onto the screen in the prescribed shape.

[0030] According to an embodiment, during the pattern image size / position adjustment process, the image frame converted by multiplying the source image frame by the pre-deformation matrix W can be controlled to be projected onto the screen as a projection image frame of the specified shape, and an evaluation related to the degree to which the projection image frame is modified into the specified shape can be performed by detecting multiple corner points of the projection image frame.

[0031] Effects of the Invention

[0032] The technical effects of the image projection device and the image projection control method of the present invention are as follows.

[0033] According to the embodiment, the lens assembly can be driven and controlled according to the measured distance to the screen, so that an image projection control method can be provided based on the measured sharpness of the pattern image.

[0034] According to the embodiment, the size and projection position of the pattern image can be adjusted based on the calculated sharpness value, thereby providing an automatic calibration method for an image projection device.

[0035] According to the embodiment, whenever the movable image projection device is placed in a specific place, the focus, screen configuration / size and definition can be automatically adjusted.

[0036] According to an embodiment, whenever the image projection device is set up or power is applied, the image projection device can be controlled to automatically adjust the focus, screen configuration / size, and clarity.

[0037] The additional scope of the applicability of the present invention will become clear from the following detailed description. However, those skilled in the art can clearly understand the various changes and modifications made within the thought and scope of the present invention, and therefore, the specific embodiments such as the detailed description and the preferred embodiments of the present invention should be understood to be exemplary only. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1This is a block diagram for explaining the image projection device of the present invention.

[0039] Figure 2 It is a perspective view showing a projector according to an embodiment of the present invention.

[0040] Figure 3 FIG. 1 is a diagram showing an optical system of a projector according to an embodiment of the present invention.

[0041] Figure 4a A block diagram showing an image projection device for correcting a projected image and an image projection system including the image projection device according to the present invention.

[0042] Figure 4b Show Figure 4a Detailed structure of the processor of the image projection device.

[0043] Figure 5a A flow chart showing the sharpness measurement method based on measuring distance of the present invention.

[0044] Figure 5b A conceptual diagram showing a method of measuring sharpness by filtering detected edge components according to an embodiment.

[0045] Figure 5c A pattern including edge components in multiple directions in order to extract sharpness components according to the present invention is shown.

[0046] Figure 6 A flow chart showing a method for adjusting the size and position of a pattern image executed in the image projection device of the present invention.

[0047] Figure 7 A conceptual diagram showing that a pattern image formed by the image projection device of the present invention is displayed as a projection image projected onto a screen.

[0048] Figure 8 The conversion relationship between the source image frame and the projection image frame of the image projection device of the present invention is shown.

[0049] Fig. 9 A flow chart showing a keystone correction method for an image projection device according to the present invention.

[0050] Fig.10 The structure of determining mapping parameters for projector-camera calibration according to the present invention and example pattern images related thereto are shown.

[0051] Fig.11a A diagram showing a structure for executing a screen area detection algorithm in the image projection device of the present invention.

[0052] Fig.11b Shown with Fig.11aThe decoder outputs relevant information and screen detection results.

[0053] Fig.12 The present invention shows the structure of detecting the screen area by using a pattern image displayed on the screen.

[0054] Fig.13 The structure of detecting a mark in an image projection device without using a screen according to the present invention is shown.

[0055] Fig.14 A flow chart showing an image projection control method for outputting a pattern image according to the present invention. DETAILED DESCRIPTION

[0056] Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings. Regardless of the figure numbers, the same or similar constituent elements are given the same figure numbers and their descriptions are omitted. The suffixes "module" and "unit" of the constituent elements used in the following description are given or mixed for the sake of ease of writing the description, and they themselves do not have meanings or functions that distinguish each other. In addition, when describing the embodiments disclosed in this specification, if it is determined that the specific description of the relevant known technology may make the gist of the embodiments disclosed in this specification unclear, its detailed description will be omitted. In addition, the drawings are provided only to facilitate the understanding of the embodiments disclosed in this specification. The technical ideas disclosed in this specification are not limited to the drawings, and it should be understood that the present invention includes all changes, equivalents and even substitutes made within the technical ideas and technical scope of the present invention.

[0057] Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited to the terms. The terms are used only to distinguish one component from another component.

[0058] When it is mentioned that a certain component is "coupled" or "connected" to another component, it should be understood that the component can be directly coupled or connected to the other component, or it can also be understood that there are other components between them. On the contrary, when it is mentioned that a certain component is "directly coupled" or "directly connected" to another component, it should be understood that there are no other components between them.

[0059] Unless the context clearly indicates otherwise, an expression in the singular includes an expression in the plural.

[0060] In the present application, the terms "including" or "having" should be understood as intending to specify the existence of the features, numbers, steps, actions, constituent elements, parts or their combinations disclosed in the present specification, but not intending to preclude the existence or additional possibilities of one or more other features, numbers, steps, actions, constituent elements, parts or their combinations.

[0061] The electronic device described in this specification can be applied to fixed terminals such as digital TVs, desktop computers, information kiosks, digital signage, etc. In particular, the electronic device described in this specification can be applied to non-contact image projection devices such as information kiosks, digital signage, etc., that is, image display devices.

[0062] Figure 1 This is a block diagram for explaining the image projection device of the present invention. Figure 1 The image projection device may be a projector, but is not limited thereto. Figure 1 The image projection device 100 may include a wireless communication unit 10, a detection unit 40, a projection unit 50, a position adjustment unit 60, and a power supply unit 90. Figure 3 The components shown in the figure are not essential for realizing the projector 100 , and thus the projector 100 described in this specification may have more components than those listed above, or may have less components than those listed above.

[0063] The wireless communication unit 10 can transmit and receive signals by using a mobile communication module or a short-range communication module. Specifically, the wireless communication unit 10 can transmit and receive specific instructions or specific information with an external terminal. Alternatively, the wireless communication unit 10 can receive images such as photos and videos from an external terminal. In addition, a function such as a remote control that can wirelessly control the projector 100 can be provided by using a short-range communication module. In addition, the wireless communication unit 10 can receive broadcast signals and / or broadcast-related information from an external broadcast management server by using a broadcast receiving module and a broadcast channel. In addition, the wireless communication unit 10 can obtain the current location of the projector 100 by using a location information module.

[0064] The detection unit 40 can detect the presence or absence of an object approaching a predetermined detection surface or an object existing nearby without mechanical contact by using the force of an electromagnetic field or infrared rays.

[0065] The projection unit 50 can function to project an image signal, and can also be referred to as an optical system. The projection unit 50 can be implemented as an image projection device including a light source 51 for providing light and a light-transmitting light source 51. In addition, the projection unit 50 can include a projection lens 59 for adjusting the size of the projected image on the projection area.

[0066] The position adjustment unit 60 has the function of adjusting the direction of the projected image of the projector 100. Specifically, the position adjustment unit 60 has the function of adjusting the position of the projection unit 50. To this end, the position adjustment unit 60 may include a vertical moving unit 61 and a horizontal moving unit 62. The vertical moving unit 61 may move the projection unit 50 up and down, and the horizontal moving unit 62 may move the projection unit 50 left and right. The camera 20, the detection unit 40 and / or the projection unit 50 of the projector 100 may be moved up and down, left and right by the position adjustment unit 60.

[0067] The memory 70 may store images to be projected, surrounding space information and / or projection area information.

[0068] The control unit 80 may be combined with the wireless communication unit 10, the detection unit 40, the projection unit 50, the position adjustment unit 60 and / or the memory 70, and may analyze the surrounding space structure, analyze the projection area, perform image correction, project an image, or transmit and receive signals with an external terminal. That is, in order to realize various embodiments described below, the control unit 80 may combine any one or more of the above-mentioned components and control.

[0069] The power supply unit 90 receives external power and internal power under the control of the control unit 80, and supplies power required for the operation of each component. The power supply unit 90 may include a detachable battery.

[0070] Figure 2 1 is a perspective view showing a projector according to an embodiment of the present invention. It shows a housing 20 of the projector 100 and a projection lens 59 exposed on the front of the housing 20. Since heat generating components such as a light source 51 are arranged in the housing 20, a vent 25 for discharging heat may be formed on the side of the housing. The focus or projection direction of the lens may be adjusted by an operating unit 30 exposed to the outside of the housing.

[0071] The housing 20 includes a wireless communication unit 10, a detection unit 40, a projection unit 50, a memory 70, a control unit 80, and a power supply unit 90. Light emitted from the light source 51 is combined with image information in the display device 58, and is emitted through the projection lens 59 to display an image on a screen or a wall.

[0072] Figure 3: is a diagram showing the optical system of the projector according to an embodiment of the present invention. The optical system 50 of the projector 100 of the present invention uses a laser diode as the light source 51. Since the light emitted from the laser diode is a laser having the same wavelength, the energy density is high, and the phase is constant and has a straight-line property, thereby improving the brightness. In addition, since the life of the laser diode itself is longer than that of the existing lamp or light-emitting diode, it has the advantages of increasing the life and low manufacturing cost, and can also achieve a smaller size, thereby reducing the size of the projector 100. Laser is light of a specific wavelength, and the efficiency of blue light is higher than that of green light. Therefore, from the perspective of efficiency, it is more advantageous to use a laser diode that emits blue light.

[0073] On the other hand, the image projection device (image projection device) of the present invention is configured to project an image and can be controlled to automatically correct the image projected onto the screen. In this regard, the image projection device (image projection device) such as a projector is configured to be connected to a computer or an electronic device and project an input image onto the screen. In recent years, with the trend of miniaturization of image projection devices such as projectors, they can be set not only in a fixed place but also in a movable manner. In this regard, whenever the image projection device that can be movably set is set in a specific place, there is a problem that the user needs to directly adjust the focus, screen configuration / size, and clarity.

[0074] Therefore, there is a problem that the size, configuration, and clarity of the projected image projected onto the screen may change depending on the user's proficiency in setting up and operating the image projection device. In order to solve this problem, it is necessary to control the image projection device so that it automatically adjusts the focus, screen configuration / size, and clarity every time the image projection device is set up or power is applied.

[0075] This specification is used to solve the above-mentioned problems and other problems. One object of the present invention is to provide an image projection device and an image projection control method. Another object of the present invention is to provide an automatic correction method for an image projection device. Another object of the present invention is to automatically adjust the focus, screen configuration / size and clarity whenever a movable image projection device is set in a specific place. Another object of the present invention is to control the image projection device so that it automatically adjusts the focus, screen configuration / size and clarity whenever the image projection device is set or power is applied.

[0076] the following, Figure 4a A block diagram of an image projection device for correcting a projected image and an image projection system including the image projection device of the present invention is shown. Figure 4b Show Figure 4a Detailed structure of the processor of the image projection device.

[0077] Reference Figure 4a and Figure 4b The image projection device 1000 may include a ToF (Time of Flight) sensor 1100 , a light assembly 1200 , a driving unit 1300 , and a processor 1400 . The image projection device 1000 may also include a camera 1500 .

[0078] The ToF sensor 1100 may be configured to measure the distance to a target. The ToF sensor 1100 of the image projection device 1000 provided in the image projection system may be configured to measure the distance to the screen or to a mark associated with the pattern image. The lamp assembly 1200 may be configured to output an image signal including a pattern image as a source image. The drive unit 1300 may be controlled so that the lens assembly 1200 moves along an axial direction. Since the drive unit 1300 controls the drive of the mechanical structure including the lens assembly 1200, it may also be referred to as an operational controller. The processor 1400 may be configured to measure the distance to the screen through the ToF sensor 1100. The processor 1400 may be controlled so that the pattern image output by the lamp assembly 1200 is projected onto the screen. The camera 1500 may be configured to capture an image of the screen or a mark associated with the pattern image.

[0079] Reference Figure 4b The processor 1400 may be configured to include a detection module 1410, a focus adjustment module 1420, a size / position adjustment module 1430, and a fine adjustment module 1440. The size / position adjustment module 1430 may include: a zoom action module 1431, which adjusts the size of the pattern image through a zoom action; and a position adjustment module 1432, which adjusts the position of the pattern image.

[0080] The micro-adjustment module 1440 may include a trapezoidal correction module 1441, a micro-adjustment module 1442, and a focus adjustment module 1443. The trapezoidal correction module 1441 may perform trapezoidal correction on the pattern image in the screen. The trapezoidal correction module 1441 may forcibly move the corner portion of the pattern image in the screen and project it, thereby adjusting it to a quadrilateral shape of the original shape. The micro-adjustment module 1442 may adjust the center point of the pattern image in the screen to be configured on one axis. The micro-adjustment module 1442 may physically move the position of the lens, thereby performing a lens shift action of projecting the image to the center of the screen. The micro-adjustment module 1442 may perform an image shift action of moving the position of the image in parallel along one axis and / or another axis in a software manner. The focus adjustment module 1443 may adjust the aspect ratio of the pattern image in the screen on one axis and / or another axis by adjusting the focus position of the pattern image in the screen.

[0081] Reference Figure 4a and Figure 4b , the processor 1400 may control the driving unit 1400 based on the measured distance to the screen and the focal position of the screen to move the lamp assembly 1200. The processor 1400 may calculate the sharpness value of the pattern image projected onto the screen by the lamp assembly 1200. The processor 1400 may adjust the size of the pattern image and the position of projecting the pattern image based on the calculated sharpness value.

[0082] The processor 1400 may be configured to detect a screen or a mark associated with the pattern image through a detection module 1410 and based on deep learning. The processor 1400 may detect the coordinates of a plurality of corner points of a screen or a mark associated with the pattern image and a credibility value associated with the detection accuracy of the coordinates based on deep learning. If the detection fails through the detection module 1410, the processor 1400 may treat it as an exception and perform additional actions. As an example, the processor 1400 may display a mark in software through a user interface (UI: User Interface) / user experience (UX: User Experience) module and detect the displayed mark.

[0083] The processor 1400 may be configured to detect the linear components in the vertical direction and the horizontal direction by using the detection module 1410 and taking the plurality of corner points as reference. The processor 1400 may determine whether any one of the plurality of corner points is an optimal corner point for detecting the screen or the mark. The processor 1400 may control the driving unit 1300 to move the lens assembly 1200 based on the distance to the optimal corner point taking the midpoint of the ToF sensor as reference.

[0084] On the other hand, the image projection device configured to project an image of the present invention can compare the distance measured by the ToF sensor 1100 with the reference distance, and can measure the sharpness value of the pattern image in different ways to perform focus adjustment. In this regard, Figure 5a FIG. 1 is a flow chart showing a method for measuring sharpness based on measuring distance of the present invention. Figure 5b A conceptual diagram showing a sharpness measurement method based on filtering edge component detection according to an embodiment. Figure 5c A pattern including edge components in multiple directions for extracting sharpness components according to the present invention is shown.

[0085] Reference Figure 5aThe sharpness measurement method can be constructed to include a distance measurement process (S110), a distance comparison process (S120), a first movement process (S130a), a second movement process (S130b), a first pattern sharpness measurement process (S140a) and a second pattern sharpness measurement process (S140b). Figure 5a The process may be executed by the processor 1400 of the image projection device 1000. Specifically, Figure 5a The process may be executed by the focus adjustment module 1420 of the processor 1400 .

[0086] Reference Figures 4a to 5a In the distance measurement process (S110), the distance to the screen or mark is measured based on the midpoint of the ToF sensor 1100. In the distance comparison process (S120), it can be determined whether the measured distance to the screen or mark is less than the ToF effective distance. In the distance comparison process (S120), if it is determined that the measured distance is less than the ToF effective distance, the first movement process (S130a) can be executed. For example, it can be determined whether the measured distance is less than the effective distance of 5m. In the distance comparison process (S120), if it is determined that the measured distance is above the effective distance, the second movement process (S130b) can be executed. For example, it can be determined whether the measured distance is greater than the effective distance of 5m.

[0087] If the measured distance is less than the ToF effective distance, the image projection device or the light assembly may be moved to a first position adjacent to the target position in the first movement process (S130a). The user interface may also be displayed on a screen or a display area provided in the image projection device, so that the image projection device moves. On the other hand, as the image projection device moves, the distance between the light assembly provided in the image projection device and the screen may change. Therefore, it can be considered that as the image projection device moves, the position of the light assembly also moves.

[0088] In the first moving process (S130a), the current position may be moved a threshold distance from the target position. For example, when the target position is 3 m and the current position is 1 m, the driving unit may be driven to move the current position by about 2.5 m.

[0089] In the first pattern sharpness measurement process (S140a), a first sharpness value of the pattern image may be measured. The lamp assembly may be moved along a prescribed direction to a first position adjacent to a target position and the first sharpness value of the pattern image may be measured. Figure 5b, edge components can be detected by using a specific filter to measure sharpness. For example, the specific filter used to measure sharpness can be a 2D Laplacian filter of Mathematical Formula 1, but is not limited thereto and can be changed according to the application.

[0090] [Formula 1]

[0091]

[0092] It can be obtained from the original image by using a specific filter as shown in Mathematical Formula 1 Figure 5b The filtered image of Figure 5b In the filtered image, pattern sharpness measurement can be performed by detecting specific edge components. The specific edge components can be formed in the vertical direction, horizontal direction, diagonal direction or any direction. Figure 5b As shown, for a circular pattern image, the edge component can be formed as a circular pattern within a specific angle range.

[0093] In the first pattern sharpness measurement process (S140a), the first sharpness value of the pattern image can be measured while moving in the vertical direction, horizontal direction, diagonal direction or any direction. Figure 5c In order to extract the sharpness component, a composite pattern 500 including a plurality of edge components in the vertical / horizontal / diagonal directions may be used. The composite pattern 500 may be configured to include a radial pattern 510 and a correction pattern 520. The radial pattern 510 and the correction pattern 520 may be referred to as a first pattern and a second pattern, respectively. The radial pattern 510 and / or the correction pattern 520 may correspond to a marker associated with the pattern image, and a screen area may be detected based on the marker.

[0094] The radial pattern 510 can be composed of a plurality of fan-shaped patterns as a pattern for adjusting auto focus (AF). The fan-shaped patterns constituting the radial pattern 510 are alternately configured by a first fan-shaped pattern of a first color and a second fan-shaped pattern of a second color. The first fan-shaped pattern and the second fan-shaped pattern can also be formed by a first pattern and a second pattern different from each other. The correction pattern 520 can be configured to be adjacent to the radial pattern 510. The correction pattern 520 is a pattern for keystone calibration of a beam projector device. The correction pattern 520 can include a first correction pattern to a fourth correction pattern 521, 522, 523, 524 configured at the upper left side, the upper right side, the lower left side, and the lower right side based on the center point of the radial pattern 510. Each of the first correction pattern to the fourth correction pattern 521, 522, 523, 524 can be configured to include a first edge component E1 in the horizontal direction and a second edge component E2 in the vertical direction.

[0095] Refer again Figure 5a If the measured distance is greater than the ToF effective distance, the image projection device or the light assembly may be moved to a second position corresponding to the ToF effective distance in the second movement process (S130b). The user interface may also be displayed on a screen or a display area provided on the image projection device to move the image projection device. The ToF effective distance may be defined as the maximum measurable distance that can measure the distance between the beam projector of the image projection device and the screen. If the measured distance is greater than the ToF effective distance of 5 m, the image projection device or the light assembly may be moved to a second position corresponding to the ToF effective distance in the second movement process (S130b).

[0096] When the image projection device is arranged in an outdoor area, the ToF effective distance, which is the maximum measurable distance, can be increased compared to when the image projection device is arranged in an indoor area. For example, in an indoor area, the ToF effective distance can be set to about 4m. On the other hand, in an outdoor area, the ToF effective distance can be set to about 5m or more.

[0097] In the second pattern sharpness measurement process (S140b), the second sharpness value of the pattern image can be measured. The second sharpness value of the pattern image can be measured while the image projection device or the lamp assembly is moved to a second position corresponding to the ToF effective distance. If the configuration of the image projection device in the indoor area leads to an increase in the ToF effective distance, in the second pattern sharpness measurement process (S140b), the second sharpness value of the pattern image can be measured while moving in the long-distance area. Therefore, the pattern sharpness can be measured and the focus can be adjusted based on the ToF distance measurement not only at a short distance indoors, but also at a long distance of 4m or more outdoors.

[0098] Reference Figures 4a to 5c , the processor 1400 can measure the sharpness value based on the ToF distance measurement. The processor 1400 can adjust the size of the pattern image and the position of the projected pattern image based on the measured sharpness value. In this regard, the processor 1400 can measure the distance to the screen based on the midpoint of the ToF sensor 1100. If the measured distance is less than the ToF effective distance, the processor 1400 can move the lamp assembly 1200 to a first position adjacent to the target position in a specified direction while measuring the first sharpness value of the pattern image. If the measured distance is above the ToF effective distance, the processor 1400 can move the lamp assembly 1200 to a second position corresponding to the ToF effective distance while measuring the second sharpness value of the pattern image. The processor 1400 can adjust the size of the pattern image and the position of the projected pattern image based on the measured first sharpness value and the second sharpness value.

[0099] On the other hand, the image projection device of the present invention configured to project an image can be configured to adjust the size and position of the pattern image. In this regard, Figure 6 A flow chart showing a method for adjusting the size and position of a pattern image executed in the image projection device of the present invention.

[0100] Reference Figure 4a , Figure 4b as well as Figure 6 , Figure 6 The process may be executed by the processor 1400 of the image projection device 1000. Specifically, Figure 6 The process may be performed by the size / position adjustment module 1430 of the processor 1400 .

[0101] Reference Figure 6 The method for adjusting the size and position of the pattern image can be constructed to include a region comparison process (S210), a first zoom execution process (S220), a center point extraction process (S230), a center point movement process (S240) and a second zoom execution process (S250).

[0102] In the area comparison process (S210), the display area of ​​the screen, i.e., the first area, and the projection area of ​​the projected pattern image, i.e., the second area, may be compared. In the first zoom execution process (S220), the first zoom action may be performed to move the focus position from the screen to the first position. In the first zoom execution process (S220), the projection area, i.e., the second area, may be controlled to be larger than the display area of ​​the screen, i.e., the first area, by a predetermined ratio or more.

[0103] In the center point extraction process (S230), the first center point of the display area of ​​the screen, i.e., the first area, and the second center point of the projection area, i.e., the second area, can be extracted. In the center point movement process (S240), the center position of the pattern image can be adjusted so that the second center point of the projection area, i.e., the second area, can be moved to the first center point of the display area of ​​the screen, i.e., the first area. In the second zoom execution process (S250), in a state where the center position is adjusted, a second zoom action can be executed so that the focus position is moved to the second position.

[0104] The size and position adjustment method of the pattern image described above can be performed by the processor 1400. In this regard, the processor 1400 can compare the display area of ​​the screen, i.e., the first area, and the projection area of ​​the projected pattern image, i.e., the second area. Based on the comparison result, the processor 1400 can be controlled to perform a first zoom operation to move the focus position to the first position, so that the second area is formed to be larger than the first area by a specified ratio or more.

[0105] The processor 1400 may extract a first center point of the first area and a second center point of the second area. The processor 1400 may adjust the center position of the pattern image so that the second center point of the second area moves to the first center point of the first area. The processor 1400 may perform a second zoom operation in a state where the center position is adjusted so that the focus position moves to the second position.

[0106] The processor 1400 performs a fine adjustment of the pattern image through a second zoom action. In this regard, the processor 1400 can measure the first sharpness value or the second sharpness value through a first zoom action of a first interval in an axial direction. The processor 1400 can measure the first sharpness value or the second sharpness value through a second zoom action of a second interval narrower than the first interval in an axial direction. The processor 1400 can perform a fine adjustment through a second zoom action of a second interval narrower than the first interval in a state where the center position is adjusted, so that the focus position moves to the second position.

[0107] On the other hand, the image projection device of the present invention is configured to project an image and can be corrected by converting between images. Figure 7A conceptual diagram showing a pattern image formed by the image projection device of the present invention and displayed as a projection image projected onto a screen. Figure 8 The conversion relationship between the source image frame and the projection image frame of the image projection device of the present invention is shown.

[0108] Reference Figure 4a , Figure 4b , Figure 7 as well as Figure 8 , it is necessary to determine a mapping matrix P representing the relationship between the image projection device 1000 as a beam projector and the screen 200. In addition, it is necessary to determine a pre-warping matrix W for making the projected image have a predetermined shape on the screen. The predetermined target shape of the projected image on the screen may be a rectangular shape, but is not limited thereto.

[0109] Thus, the processor 1400 can determine a mapping matrix P that defines a conversion relationship between a source image frame SF of the pattern image and a projected image frame PF projected onto the screen 200. In this regard, the original shape of the screen 200 can be seen, but from the user's perspective, a distorted projected image can be seen.

[0110] The processor 1400 may determine a pre-warping matrix W that enables the projection image frame PF to have a prescribed shape on the screen 200. The processor 1400 may transform the source image frame SF based on the pre-warping matrix W. The processor 1400 may control the transformed source image frame to be projected onto the screen 200 as the projection image PF of the prescribed shape. The pre-warping matrix W may be expressed as P -1 S. Here, P is a mapping matrix defining a conversion relationship between the source image frame SF and the projection image frame PF, and S is a matrix defining a conversion relationship including position shift between specific points of the pre-deformed image and image scaling.

[0111] The processor 1400 may control the camera 1500 to capture an image of the screen 200 or a mark associated with the pattern image. The processor 1400 may detect four corner points by detecting the screen 200. The processor 1400 may determine a first mapping matrix T that defines a conversion relationship between a source image frame SF and a camera image frame CF based on the detected four corner points. In this regard, from the perspective of the camera 1500, the original shape of the projected image frame PF before distortion can be seen. In contrast, the detected four corner points of the screen 200 are formed into a shape that is not a prescribed shape, so that the distorted state of the screen 200 can be seen.

[0112] The processor 1400 can determine the second mapping matrix C so that the projection image frame PF projected onto the detected screen 200 has a prescribed shape from the perspective of the camera 1500 based on the detected four corner points. In this regard, the source image frame SF can be a rectangular shape formed by four corner points A, B, C, and D. The source image frame SF can be converted into a camera image frame CF by using the first mapping matrix T as a mapping parameter. The camera image frame CF can detect four corner points A', B', C', and D'. Thus, the correction sliding pattern as a pattern image can be detected on the camera image frame CF.

[0113] The processor 1400 may determine the mapping matrix P as C based on the first mapping matrix T and the second mapping matrix C. -1 The processor 1400 may determine an optimal point for adjusting the offset and scale of the rectified screen. The processor 1400 may determine whether the offset and scale of the rectified screen are appropriate based on the coordinates of the optimal point and the coordinates of the corresponding points of the screen before the rectified screen is rectified.

[0114] The trapezoidal correction method of the image projection device of the present invention can be performed by switching between the screen / marker detected during correction and the image frame. In this regard, Fig. 9 A flowchart showing a method for trapezoidal correction of an image projection device according to the present invention is shown. Fig. 9, the trapezoidal correction method may include a projector-camera correction process (S310a), a screen detection process (S310b), a mapping matrix derivation process (S320a), and a screen area modification process (S320b). In this regard, the projector-camera correction process (S310a) and the screen detection process (S310b) may be performed simultaneously, or the other process may be started before any one process ends. The mapping matrix derivation process (S320a) and the screen area modification process (S320b) may be performed simultaneously, or the other process may be started before any one process ends. The trapezoidal correction method may also include an optimal point configuration process (S330) and a pre-deformation process (S340).

[0115] on the other hand, Fig.10 The structure of determining mapping parameters for projector-camera calibration according to the present invention and example pattern images related thereto are shown. Fig.10 (a) shows the conversion between the source image frame SF and the camera image frame CF based on the mapping parameters of the projector-camera calibration of the present invention. Fig.10 (b) shows an example of a correction sliding pattern as a pattern image generated and projected by the image projection device of the present invention.

[0116] Reference Figures 8 to 10 , the source image frame SF may be a rectangular shape formed by four corner points A, B, C, and D. For example, the coordinates of the multiple corner points A, B, C, and D of the source image frame SF may be (0, 0), (0, 720), (1280, 0), and (1280, 720), respectively. The source image frame SF may be converted into a camera image frame CF by using a first mapping matrix T as a mapping parameter. The camera image frame CF may detect four corner points A′, B′, C′, and D′. Thus, the four corner points A′, B′, C′, and D′ may be detected on the camera image frame CF. Fig.10 (b) Pattern image of the corrected sliding pattern.

[0117] The first mapping matrix T as a homography matrix may be calculated by at least four corresponding points (eg, a plurality of corner points) between the source image frame SF and the calibrated sliding pattern on the camera image frame CF.

[0118] Reference Figure 4a , Figure 4b as well as Figures 7 to 10In the projector-camera calibration process (S310a), a first mapping matrix T between the projector as an image projection device and the camera image frame CF may be derived. In the screen detection process (S310b), four corner points may be derived after detecting the area where the screen 200 is configured. In this regard, calibration may be performed by projecting a pattern image such as a chessboard pattern from the projector.

[0119] In the screen area modification process (S320b), a second mapping matrix C may be determined to make the projection image frame PF projected onto the screen 200 detected from the perspective of the camera 1500 have a prescribed shape. In the screen area modification process (S320b), a relational expression may be derived to convert the screen area seen from the perspective of the camera into a prescribed shape, for example, a quadrilateral shape. The screen area modification process (S320b) may be used to remove projective and affine distortion from the projection image frame.

[0120] In the mapping matrix derivation process (S320a), a mapping matrix P representing the relationship between the projector and the screen image can be derived. Specifically, the mapping matrix P defining the conversion relationship between the source image frame SF of the pattern image and the projected image frame PF projected onto the screen can be derived. The mapping matrix P can be de-positioned by C -1 T. Therefore, the mapping matrix derivation process (S320a) can use the first mapping matrix T derived in the projector-camera calibration process (S310a) and the second mapping matrix C derived in the screen area modification process (S320b).

[0121] In the optimal point configuration process (S330), the optimal point can be determined in the modified screen area. In this regard, the offset and scale of the image can be adjusted. In the pre-deformation process (S340), the source image frame SF can be multiplied by the pre-deformation matrix W to project the converted image frame onto the screen through a projector.

[0122] On the other hand, the image projection device configured to project an image of the present invention can derive a projection image frame PF for projection without pre-deformation by using the camera image frame CF. In this regard, the processor 1400 can apply the inverse transformation C of the second mapping matrix C to the camera image frame CF. -1The projection image frame PF before being modified into the image (rectified screen) of the prescribed shape is derived. Thus, the projected image frame can be indirectly obtained. The processor 1400 can control the image frame converted by multiplying the source image frame SF by the pre-deformation matrix W to be projected onto the screen as a projection image frame PF2 of the prescribed shape.

[0123] On the other hand, the correction performance can be evaluated by detecting a plurality of corner points of the projection image frame for keystone correction of the image projection device of the present invention. In this regard, the processor 1400 can evaluate the degree of modification to the prescribed shape of the projection image frame PF2 by detecting a plurality of corner points of the projection image frame PF2.

[0124] The processor 1400 may modify the mapping matrix P to P1 based on the distance difference between the coordinates of the detected plurality of corner points and the coordinates of the plurality of optimal corner points for modification to the prescribed shape. The processor 1400 may modify the pre-deformation matrix W to W1 based on the modified mapping matrix P1. The processor 1400 may transform the source image frame SF based on the modified pre-deformation matrix W1 and control the transformed source image frame to be projected onto the screen 200 in the prescribed shape. Thus, by reflecting the correction result before the image projection device of the present invention performs the trapezoidal correction, the correction can be further accurately and quickly performed.

[0125] Reference Figure 4a The image projection device 1000 may further include a user input unit 1600. On the other hand, the image projection device configured to project an image of the present invention may automatically perform calibration according to a user input through the user input unit 1600.

[0126] Reference Figure 4a , Figure 4b as well as Figures 7 to 10 The user input unit 1600 may be configured to receive user input to implement correction of the pattern image projected onto the screen 200. Since the pattern image may be automatically corrected according to the user input inputted through the user input unit 1600, it may be referred to as one-click correction.

[0127] The processor 1400 may control the projection image frame PF2 projected onto the screen to be displayed in a predetermined shape based on the user input inputted through the user input unit 1600. The processor 1400 may control the projection image frame PF2 projected onto the screen 200 to be displayed in a predetermined shape and to be maintained at a position within a predetermined distance from the center position on one axis of the screen 200 based on the user input.

[0128] In the image projection device configured to project an image of the present invention, as described above, screen area detection can be performed by a pre-processing process based on deep learning or artificial intelligence (AI) and a post-processing process based on subsequent intersection extraction. In this regard, Fig.11a FIG. 1 is a diagram showing a structure of a screen area detection algorithm executed in the image projection device of the present invention. Fig.11b Shown in Fig.11a The decoder outputs relevant information and screen detection results.

[0129] Reference Fig.11a , the input image can be input to the detection module 1410 that performs screen area / corner detection. The detection module 1410 may include: a deep learning module 1411 for screen area / corner detection; and a decoder 1412 that outputs information about the screen area. The detection module 1410 may also include a post-processing module 1413, which is configured to extract intersection points and detect corner points of the screen area.

[0130] The input image input to the deep learning module 1411 may be a rectangular or square shaped image. The detection module 1410 may detect the screen area and a plurality of corner points of the screen by using an object detection network. A center network (CenterNet) may be used as an object detection network, but is not limited thereto and may be changed according to the application. In order to detect the screen area and a plurality of corner points separately, two deep learning models may also be used. The straight line components in the vertical / horizontal direction may be detected based on the plurality of corner points detected by the deep learning module 1411, thereby determining a plurality of optimal corner points.

[0131] The map information, offset information, and size information of the screen area may be predicted by the deep learning module 1411. The screen area and a plurality of corner points of the screen may be output by the decoder 1412. The intersections may be extracted by line fitting by the post-processing module 1413, thereby determining a plurality of optimal corner points of the screen area configuring the screen 200. To this end, the optimal corner point may be determined by detecting the straight line component in the vertical / horizontal direction related to the screen area.

[0132] Fig.11b (a) shows that Fig.11a The decoder 1412 outputs the frame area information of the screen and the corner point information of the screen. Fig.11b (b) shows coordinate information P1, P2, P3, and P4 of a frame area BR and a plurality of corner points detected from the screen captured by the camera of the image projection device.

[0133] Reference Fig.11a and Fig.11b, the coordinate information (LT.x, LT.y, RB.x, RB.y) and reliability information of the frame area BR of the display screen area can be derived through the decoder 1412. Here, (LT.x, LT.y) and (RB.x, RB.y) represent the (x, y) coordinates of the upper left end and the lower right end of the frame area, and the box score is the credibility value of the coordinate information of the frame area. The coordinate information P1, P2, P3, P4 and reliability information of multiple corner points of the screen can be derived through the post-processing module 1413. The coordinate information P1, P2, P3, P4 can be expressed as (LT.x, LT.y), (RT.x, RT.y), (RB.x, RB.y) and (LB.x, LB.y). Here, (LT.x, LT.y), (RT.x, RT.y), (RB.x, RB.y) and (LB.x, LB.y) represent the (x, y) coordinates of multiple corner points at the upper left end, upper right end, lower left end and lower right end, and the box score is the credibility value of the coordinate information of the box area.

[0134] On the other hand, the image projection device of the present invention configured to project an image can perform post-processing based on the screen area, or in the absence of a screen, can perform post-processing based on a marker. In this regard, Fig.12 The structure of the screen area is shown to be detected by using a pattern image displayed on the screen of the present invention.

[0135] Reference Figures 11a to 12 , a post-processing process for reducing the error of the detected multiple corner points can be performed based on deep learning. Fig.12 (a) of the present invention can generate a region of interest (ROI) by using the coordinate values ​​of a plurality of corner points of the screen area. Fig.12 The ROI of (a) can correspond to the representation Fig.12 (b) The frame area BR of the screen area. Fig.12 (a) can detect the edge components inside the ROI, such as Fig.12 In (b), line fitting can be performed and intersection points can be extracted. Fig.12 The intersection of (b) can correspond to Fig.11b The transformation matrix can be calculated by using the extracted intersection points.

[0136] In the case where there is no actual screen and the projected image is displayed on the wall, or in the case where the detection of the screen area fails, a mark generated by software can be projected on the wall. Fig.13 The structure of detecting marks in the image projection device without using a screen according to the present invention is shown.

[0137] Reference Figure 5c and Fig.13 , a plurality of markers may be configured to project a pattern image onto a wall surface without a screen. Therefore, in an environment without a screen, a conversion matrix estimation method is applied by using markers. A first marker 500a and a second marker 500b may be configured in the left area and the right area. For example, the positions where the first marker 500a and the second marker 500b are displayed may correspond to the positions of the left area and the right area of ​​the screen area.

[0138] After detecting the first mark 500a and the second mark 500b of the left area and the right area, an orthogonalization process can be performed so that the first correction pattern to the fourth correction pattern 521a, 522a, 523a, 524a of the detected first mark 500a are formed into a square shape. Similarly, an orthogonalization process can be performed so that the first correction pattern to the fourth correction pattern 521b to 521b of the detected second mark 500b are formed into a square shape. Whether the screen correction such as the orthogonalization process is performed normally is verified by determining whether the mark that is not detected or selected is a square shape. In the orthogonalization process, the conversion matrix can be selected by confirming whether the correction pattern (521a to 524 and 521b to 521b) is displayed as a square shape.

[0139] For the above screen area and / or landmark detection, refer to Figures 4a to 13 , the actions of the processor are described in detail below. In this regard, if the deep learning-based detection of the screen 200 fails, the control processor 1400 may be controlled to generate a mark related to the pattern image and project the mark through the light assembly 1200. The processor 1400 may detect the coordinates of the projected screen area or the plurality of corner points of the mark and the credibility value through the camera 1500.

[0140] The processor 1400 can determine whether it is possible to obtain the lines of the upper end, the lower end, one end, and the other end of the screen 200 from the camera's point of view. The processor 1400 can determine whether the surface of the screen is displayed as a uniform color within a specified range. The processor 1400 can determine whether there is a portion of the display area of ​​the screen, that is, the first area, that is blocked by a person or other objects. If it is determined that the lines of the screen can be obtained and the surface is displayed as a uniform color and there is no blocked portion in the first area, the processor 1400 can detect the coordinates and credibility values ​​of multiple corner points of the screen 200 based on deep learning.

[0141] In the above, an image projection device configured to project an image according to one embodiment of the present specification is described. In the following, an image projection control method configured to output a pattern image according to another embodiment of the present specification is described. In this regard, the actions and technical features described in the image projection device can also be applied to the following image projection control method.

[0142] Regarding this, Fig.14 A flowchart showing an image projection control method for outputting a pattern image according to the present invention is shown. Figure 4a and Fig.14 The image projection control method can be executed by a processor of an image projection device such as a projector.

[0143] The image projection control method may be configured to include a screen / marker detection process (S1100), a drive control process (S1200), a sharpness value calculation process (S1300), and an image size / position adjustment process (S1400).

[0144] In the screen / marker detection process (S1100), the screen or a mark associated with the pattern image can be detected. In the drive control process (S1200), the drive unit can be controlled to move the lamp assembly based on the distance to the screen and the screen to the focus position measured by the ToF sensor. In the sharpness value calculation process (S1300), the sharpness value of the pattern image projected onto the screen by the lamp assembly can be calculated. In the image size / position adjustment process (S1400), the size of the pattern image and the position of the pre-projected pattern image can be adjusted based on the calculated sharpness value.

[0145] In the screen / marker detection process (S1100), the coordinates of a plurality of corner points of the screen or a mark associated with the pattern image and the reliability value associated with the detection accuracy of the coordinates can be detected based on deep learning. In the screen / marker detection process (S1100), by detecting the straight line components in the vertical direction and the horizontal direction based on the plurality of corner points, it is determined whether any one of the plurality of corner points is an optimal corner point for detecting the screen or the mark.

[0146] After executing the sharpness value calculation process (S1300), the image projection control method may further include a zoom / position adjustment process (S1350). In the zoom / position adjustment process (S1350), the position may be adjusted by comparing the display area of ​​the screen, i.e., the first area, and the projection area of ​​the projected pattern image, i.e., the second area. In the zoom / position adjustment process (S1350), the first zoom action may be controlled to move the focus position to the first position, so that the second area is formed to be larger than the first area by a specified ratio or more.

[0147] In the zoom / position adjustment process (S1350), a first center point of the first area and a second center point of the second area may be extracted. In the zoom / position adjustment process (S1350), the center position of the pattern image may be adjusted so that the second center point of the second area moves to the first center point of the first area. In the zoom / position adjustment process (S1350), in a state where the center position is adjusted, a second zoom action may be performed so that the focus position moves to the second position.

[0148] The above describes the image projection device and the image projection control method. The technical effects of the image projection device and the image projection control method of the present invention can be summarized as follows, but are not limited thereto.

[0149] According to the embodiment, the lens assembly may be driven and controlled according to the measured distance to the screen, thereby providing an image projection control method based on the measured sharpness of the pattern image.

[0150] According to the embodiment, the size and projection position of the pattern image can be adjusted based on the calculated sharpness value, thereby providing an automatic correction method for an image projection device.

[0151] According to the embodiment, whenever the movable image projection device is placed in a specific place, the focus, screen configuration / size and definition can be automatically adjusted.

[0152] According to an embodiment, whenever the image projection device is set up or power is applied, the image projection device can be controlled to automatically adjust the focus, screen configuration / size, and clarity.

[0153] The additional scope of the applicability of the present invention will become clear from the following detailed description. However, it is clearly understood by those skilled in the art that various changes and modifications can be made within the concept and scope of the present invention, and therefore, the detailed description and specific embodiments such as the preferred embodiments of the present invention should be understood as exemplary only.

[0154] The additional scope of possible applications of the image projection device and the image projection control method of the present invention will become clear from the following detailed description. However, those skilled in the art will clearly understand that various changes and modifications can be made within the concept and scope of the present invention, and therefore, the detailed description and specific embodiments such as the preferred embodiments of the present invention should be understood as exemplary only.

[0155] The above-mentioned image projection device and image projection control method related to the present invention can be implemented by a computer-readable code in a medium storing a program. Computer-readable media include all kinds of storage devices storing data that can be read by a computer system. As computer-readable media, there are, for example, HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic disk, floppy disk, optical data storage device, and in addition, it can also be implemented in the form of a carrier (for example, based on Internet transmission). In addition, the computer may also include a control unit of the terminal. Therefore, the detailed description should not be interpreted as restrictive in all aspects, but should be considered as exemplary. The scope of the present invention should be determined by the reasonable interpretation of the attached claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims

1. An image projection device, wherein: include: A ToF sensor configured to measure the distance to a target; a light assembly configured to output an image signal including a pattern image; a driving unit, controlling the lens assembly to move along an axial direction; and A processor, controlling to measure the distance to the screen through the ToF sensor and project the output pattern image onto the screen through the light assembly; the processor controls the driving unit to move the lamp assembly based on the measured distance and the focus position from the screen, The processor calculates a sharpness value of the pattern image projected onto the screen by the lamp assembly, The processor adjusts the size of the pattern image and the position at which the pattern image is projected based on the calculated sharpness value.

2. The image projection device according to claim 1, wherein: Also included is a camera configured to capture an image of the screen or a mark associated with the pattern image, The processor detects the coordinates of a plurality of corner points of the screen or the coordinates of a plurality of corner points of a mark associated with the pattern image and a confidence value associated with the detection accuracy of the coordinates based on deep learning, The processor determines which corner point among the plurality of corner points is the best corner point for detecting the screen or the mark by detecting the straight line components in the vertical direction and the horizontal direction based on the plurality of corner points, The processor controls the driving unit to move the lens assembly based on a distance from a midpoint of the ToF sensor to the optimal corner point.

3. The image projection device according to claim 2, wherein: If the screen fails to be detected based on deep learning, the processor controls to generate a mark related to the pattern image and project the mark through the light assembly, The processor detects the coordinates of a plurality of corner points of the projected mark and the credibility value through the camera.

4. The image projection device according to claim 2, wherein: The processor determines whether the lines of the upper end, the lower end, one end, and the other end of the screen can be obtained from the viewing angle of the camera, The processor determines whether the surface of the screen displays a uniform color within a specified range, The processor determines whether the display area of ​​the screen, that is, the first area, has no part blocked by a person or other objects, If it is determined that the line of the screen can be acquired and the surface is displayed as a uniform color and there is no blocked portion in the first area, the processor detects the coordinates of multiple corner points of the screen and the credibility value based on deep learning.

5. The image projection device according to claim 1, wherein: The processor measures the distance from the midpoint of the ToF sensor to the screen. If the measured distance is less than the ToF effective distance, the processor moves the light assembly along a specified direction to a first position adjacent to the target position and measures a first sharpness value of the pattern image. If the measured distance is greater than the ToF effective distance, the processor moves the light assembly to a second position corresponding to the ToF effective distance and measures a second sharpness value of the pattern image. The processor adjusts the size of the pattern image and the position at which the pattern image is projected based on the measured first sharpness value and the second sharpness value.

6. The image projection device according to claim 5, wherein: The processor compares the display area of ​​the screen, i.e., the first area, and the projection area, i.e., the second area, where the pattern image is projected. The processor controls to execute a first zoom operation to move the focus position to a first position so that the second area is larger than the first area by a predetermined ratio or more. The processor extracts a first center point of the first area and a second center point of the second area, The processor adjusts the center position of the pattern image so that the second center point of the second area moves to the first center point of the first area. In a state where the center position is adjusted, the processor performs a second zooming operation to move the focus position to a second position.

7. The image projection device according to claim 6, wherein: The processor measures the first sharpness value or the second sharpness value through the first zoom action at a first interval in an axial direction, the processor measures the first sharpness value or the second sharpness value through the second zooming action of the second interval narrower than the first interval in the one axial direction, In a state where the center position is adjusted, the processor performs fine adjustment to move the focus to a second position.

8. The image projection device according to claim 1, wherein: The processor determines a mapping matrix P, wherein the mapping matrix P defines a conversion relationship between a source image frame of the pattern image and a projection image frame projected onto the screen. The processor determines a pre-deformation matrix W, wherein the pre-deformation matrix W enables the projected image frame to have a prescribed shape on the screen. The processor is controlled to transform the source image frame based on the pre-deformation matrix W, and to project the transformed source image frame onto the screen in the prescribed shape.

9. The image projection device according to claim 8, wherein: Also included is a camera configured to capture an image of the screen or a mark associated with the pattern image, The processor detects four corner points by detecting the screen, The processor determines a first mapping matrix T based on the four detected corner points, where the first mapping matrix T defines a conversion relationship between the source image frame and the camera image frame.

10. The image projection device according to claim 9, wherein: The processor determines a second mapping matrix C based on the detected four corner points, wherein the second mapping matrix C enables the projection image frame projected onto the detected screen to become the prescribed shape from the perspective of the camera.

11. The image projection device according to claim 10, wherein: The processor determines the mapping matrix P as C based on the first mapping matrix T and the second mapping matrix C -1 T, The processor determines an optimal point for adjusting the offset and scale of the image modified to the specified shape, The processor determines whether the offset and the scale of the modified image are appropriate based on the coordinates of the optimal point and the coordinates of the corresponding points of the image before being modified into the prescribed shape.

12. The image projection device according to claim 10, wherein: The processor applies the inverse transformation C of the second mapping matrix C to the camera image frame. -1 To estimate the projected image frame before being modified into the image of the prescribed shape.

13. The image projection device according to claim 10, wherein: The processor controls the image frame converted by multiplying the source image frame by the pre-deformation matrix W to be projected onto the screen as a projection image frame of the prescribed shape.

14. The image projection device according to claim 13, wherein: The processor performs an evaluation related to the degree to which the projected image frame is modified into the prescribed shape by detecting a plurality of corner points of the projected image frame. The processor modifies the mapping matrix P into P1 based on the distance difference between the detected coordinates of the plurality of corner points and the coordinates of the plurality of optimal corner points for modifying the prescribed shape. The processor modifies the pre-deformation matrix to W1 based on the modified mapping matrix P1, The processor controls to transform the source image frame based on the modified pre-deformation matrix W1, and the transformed source image frame is projected onto the screen in the prescribed shape.

15. The image projection device according to claim 8, wherein: The system further comprises a user input unit configured to receive user input to implement correction of the pattern image projected onto the screen. The processor controls, based on the user input, to display the projection image frame projected onto the screen in the prescribed shape and to maintain a center position of the projection image frame at a position within a prescribed distance from a center position on one axis of the screen.

16. A method for controlling image projection, wherein: The method is controlled to output a pattern image, and the method is executed by a processor of the projector. The method comprises: a screen / marker detection process for detecting a screen or a mark associated with the pattern image; a drive control process of controlling a drive unit to move the lamp assembly based on a distance to the screen and a focus position from the screen measured by a ToF sensor; a sharpness value calculation process, calculating the sharpness value of the pattern image projected onto the screen by the lamp assembly; and The pattern image size / position adjustment process adjusts the size of the pattern image and the position of projecting the pattern image based on the calculated sharpness value.

17. The image projection control method according to claim 15, wherein: During the screen / marker detection process, Based on deep learning, the coordinates of a plurality of corner points of the screen or the coordinates of a plurality of corner points of a mark associated with the pattern image and a credibility value associated with the detection accuracy of the coordinates are detected, By detecting the straight line components in the vertical direction and the horizontal direction based on the plurality of corner points, it is determined which corner point among the plurality of corner points is the best corner point for detecting the screen or the mark.

18. The image projection control method according to claim 16, wherein: The method further includes a zoom / position adjustment process, wherein after executing the sharpness value calculation process, the zoom / position adjustment process compares the display area of ​​the screen, i.e., the first area, and the projection area, i.e., the second area, where the pattern image is projected, and adjusts the position. During the zoom / position adjustment process, controlling to execute a first zoom operation to move the focus position to a first position so that the second area is larger than the first area by a predetermined ratio or more, extracting a first center point of the first area and a second center point of the second area, adjusting the center position of the pattern image so that the second center point of the second area moves to the first center point of the first area, In a state where the center position is adjusted, a second zooming operation is performed to move the focus position to a second position.

19. The image projection control method according to claim 16, wherein: During the pattern image size / position adjustment process, Determine a mapping matrix P, wherein the mapping matrix P defines a conversion relationship between a source image frame of the pattern image and a projection image frame projected onto the screen, determining a pre-deformation matrix W, wherein the pre-deformation matrix W enables the projected image frame to have a prescribed shape on the screen, The control is to transform the source image frame based on the pre-deformation matrix W, and project the transformed source image frame onto the screen in the prescribed shape.

20. The image projection control method according to claim 19, wherein: During the pattern image size / position adjustment process, Controlling the image frame converted by multiplying the source image frame by the pre-deformation matrix W to be projected onto the screen as a projection image frame of the prescribed shape, An evaluation is performed on the degree to which the projected image frame is modified into the prescribed shape by detecting a plurality of corner points of the projected image frame.

Images