Projection method, projection device, and program product

Abstract

The invention provides a projection method, a projection device, and a program product, which can reduce the sense of incongruity caused by long waiting time until a corrected projection image is projected. The projection method includes: performing a calculation for correcting a shape of a projection image at a high speed; projecting a provisional correction image on a projection surface based on the result of the calculation; performing a calculation for correcting the shape of the projection image with high precision; and projecting the determined correction image on the projection surface on the basis of the calculation result, the processing time for obtaining the calculation result for correcting the shape of the projection image at a high speed being a first time period, and the processing time for obtaining the calculation result for correcting the shape of the projection image at a high precision being a second time period longer than the first time period. The provisional correction image is projected onto the projection surface before the determined correction image is projected onto the projection surface.

Description

Technical Field

[0001] The invention relates to a projection method, a projection device and a program product. Background Art

[0002] Patent Document 1 proposes a projector that corrects trapezoidal distortion caused by the inclination of the projection surface by measuring the inclination of the projection surface based on a measurement image projected onto the projection surface. In addition, various schemes have been proposed in the mainland for correcting trapezoidal distortion, including a structure for simply correcting and a structure for correcting with high precision, in addition to the above-mentioned structure.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2015-186080

[0004] However, in the structure for correcting trapezoidal distortion with high accuracy, there is a problem that the correction requires a long time. If the correction of trapezoidal distortion requires a long time, the user is made to wait before being able to visually confirm the desired image, which makes the user feel uncomfortable. On the other hand, in the structure for simply correcting trapezoidal distortion, correction can be performed in a short time, but the accuracy of the correction is insufficient, and the contour of the image may be tilted. Summary of the invention

[0005] A projection method is a projection method in a projection device that projects a projection image onto a projection object, characterized in that the projection method includes: performing a first correction operation for correcting the shape of the projection image; projecting a first projection image as the projection image based on the operation result of the first correction operation onto the projection object; performing a second correction operation for correcting the shape of the projection image; and projecting a second projection image as the projection image based on the operation result of the second correction operation onto the projection object, wherein a processing time for obtaining the operation result of the first correction operation is a first time, a processing time for obtaining the operation result of the second correction operation is a second time longer than the first time, and before projecting the second projection image onto the projection object, the first projection image is projected onto the projection object.

[0006] The projection device is characterized in that it comprises: a projection optical system including a lens for projecting a projection image onto a projection object; and one or more processors, wherein the one or more processors perform the following processing: performing a first correction operation for correcting the shape of the projection image; projecting a first projection image as the projection image based on the operation result of the first correction operation from the projection optical system to the projection object; performing a second correction operation for correcting the shape of the projection image; and projecting a second projection image as the projection image based on the operation result of the second correction operation from the projection optical system to the projection object, wherein a processing time for obtaining the operation result of the first correction operation is a first time, and a processing time for obtaining the operation result of the second correction operation is a second time longer than the first time, and the one or more processors project the first projection image from the projection optical system to the projection object before projecting the second projection image from the projection optical system to the projection object.

[0007] The program product is characterized in that the program product causes a computer that controls a projection device having a projection optical system to execute processing, the projection optical system including a lens for projecting a projection image onto a projection object, the processing being the following processing: performing a first correction operation for correcting the shape of the projection image; projecting a first projection image as the projection image based on the operation result of the first correction operation from the projection optical system to the projection object; performing a second correction operation for correcting the shape of the projection image; and projecting a second projection image as the projection image based on the operation result of the second correction operation from the projection optical system to the projection object, the processing time for obtaining the operation result of the first correction operation is a first time, and the processing time for obtaining the operation result of the second correction operation is a second time longer than the first time, so that the computer projects the first projection image from the projection optical system to the projection object before projecting the second projection image from the projection optical system to the projection object. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 It is a block diagram showing the internal structure of a projector.

[0009] Figure 2 It is a schematic diagram showing a schematic structure of an image projection unit included in a projector.

[0010] Figure 3 This is an explanatory diagram for explaining the high-speed correction process.

[0011] Figure 4 It is an explanatory diagram for explaining the high-precision correction processing.

[0012] Figure 5 : is a flowchart showing the correction process of keystone distortion.

[0013] Figure 6 This is a flowchart showing the high-speed correction process according to the first embodiment.

[0014] Figure 7 This is a flowchart showing the high-precision correction process according to the first embodiment.

[0015] Figure 8 This is a flowchart showing the high-precision correction process according to the second embodiment.

[0016] Fig. 9 This is a flowchart showing the high-precision correction process according to the third embodiment.

[0017] Fig.10 This is a flowchart showing the high-speed correction process according to the fourth embodiment.

[0018] Description of symbols

[0019] 1: projector; 5: image providing device; 10: control unit; 10a: correction control unit; 11: storage unit; 11a: control program; 12: operation unit; 13: image input unit; 14: image processing unit; 14a: keystone correction unit; 15: image projection unit; 16: projection surface measuring unit; 16a: infrared irradiation unit; 16b: depth camera; 17: motion detection unit; 21: light source; 22R, 22G, 22B: liquid crystal light valve; 22i: pixel area; 23: projection optical system; 24: light valve driving unit; A1: area; A2: area; Sp: projection surface. DETAILED DESCRIPTION

[0020] 1. First Implementation Method

[0021] Hereinafter, a projector 1 according to a first embodiment will be described with reference to the drawings.

[0022] Figure 1 is a block diagram showing the internal structure of the projector 1. Figure 2 It is a schematic diagram showing a schematic configuration of the image projection unit 15 included in the projector 1 .

[0023] like Figure 1 As shown in FIG. 1 , the projector 1 integrally includes a control unit 10, a storage unit 11, an operation unit 12, an image input unit 13, an image processing unit 14, an image projection unit 15, a projection surface measurement unit 16, and a motion detection unit 17. The projector 1 projects a projection image from the image projection unit 15 onto a projection surface Sp based on image data input to the image input unit 13. The projector 1 is an example of a projection device, and the projection surface Sp is an example of a projection object.

[0024] The control unit 10 includes one or more processors such as a CPU (Central Processing Unit), and controls the operation of the projector 1 in a unified manner by operating according to a control program 11a stored in the storage unit 11. The control unit 10 corresponds to a computer. Alternatively, the computer may be a terminal device such as a personal computer that controls the projector 1.

[0025] The storage unit 11 includes RAM (Random Access Memory) and ROM (Read Only Memory). RAM is used to temporarily store various data, and ROM stores a control program 11a for controlling the operation of the projector 1, control data, image data, etc. The control program 11a is an example of a program.

[0026] The operation unit 12 has a plurality of operation keys (not shown) for the user to give various instructions to the projector 1. When the user operates the various operation keys of the operation unit 12, the operation unit 12 outputs an operation signal corresponding to the user's operation content to the control unit 10. In addition, a remote controller (not shown) capable of remote operation may be used as the operation unit 12. In this case, the remote controller transmits an infrared operation signal corresponding to the user's operation content, and the remote controller signal receiving unit (not shown) receives the operation signal and transmits it to the control unit 10.

[0027] The image input unit 13 includes a plurality of input terminals (not shown), and each input terminal is connected to an external image providing device 5 such as a computer or an image reproduction device. The image input unit 13 receives image data from the image providing device 5, performs data format conversion processing, resolution conversion processing, etc. on the provided image data as required, and outputs the processed image data to the image processing unit 14.

[0028] The image processing unit 14 performs various adjustment processes on the image data input from the image input unit 13 under the control of the control unit 10, and outputs the processed image data to the light valve driving unit 24 (see Figure 2 ). For example, the image processing unit 14 processes the image data to adjust the image quality, correct keystone distortion, and superimpose an OSD (On Screen Display) image such as a message or menu on the displayed image. The image processing unit 14 can be implemented by one or more integrated circuits.

[0029] like Figure 2As shown, the image projection unit 15 includes a light source 21, three liquid crystal light valves 22R, 22G, and 22B as light modulation devices, a projection optical system 23, a light valve driving unit 24, etc. The image projection unit 15 modulates the light emitted from the light source 21 using the liquid crystal light valves 22R, 22G, and 22B to form image light, and projects the image light from the projection optical system 23 including a lens, and displays a projection image on the projection surface Sp.

[0030] The light source 21 includes a solid light source such as a semiconductor laser or a light emitting diode, or a discharge type light source lamp such as an ultra-high pressure mercury lamp or a metal halide lamp. The light emitted from the light source 21 is converted into light with a substantially uniform brightness distribution by an integrator optical system (not shown), and is separated into light components of red, green, and blue, which are the three primary colors of light, by a color separation optical system (not shown), and then enters the liquid crystal light valves 22R, 22G, and 22B respectively.

[0031] The liquid crystal light valves 22R, 22G, and 22B are each composed of a transmissive liquid crystal panel with liquid crystal sealed between a pair of transparent substrates, etc. A rectangular pixel area 22i composed of a plurality of pixels arranged in a matrix is ​​formed on each liquid crystal panel, and a driving voltage can be selectively applied to each liquid crystal element constituting each pixel.

[0032] The light valve driving unit 24 forms a projected image in the pixel area 22i of the liquid crystal light valves 22R, 22G, and 22B. Specifically, the light valve driving unit 24 applies a driving voltage corresponding to the image data input from the image processing unit 14 to each pixel in the pixel area 22i, and sets each pixel to a transmittance corresponding to the image data. The light emitted from the light source 21 passes through the pixel area 22i of the liquid crystal light valves 22R, 22G, and 22B, and is modulated for each pixel, thereby forming image light corresponding to the image data for each color light. The formed image light of each color is synthesized for each pixel by a color synthesis optical system (not shown) to become image light representing a color image, and is enlarged and projected onto the projection surface Sp by the projection optical system 23. As a result, a projection image based on the image data input from the image processing unit 14 is displayed on the projection surface Sp.

[0033] return Figure 1The projection surface measuring unit 16 includes, for example, an infrared irradiation unit 16a and a depth camera 16b, which measures the inclination of the projection surface Sp. The depth camera 16b obtains a depth image in a so-called ToF (Time Of Flight) method by measuring the reflected light of the infrared irradiation unit 16a, and outputs it to the control unit 10. The output of the depth camera 16b, that is, the depth image, is an image that corresponds a depth value corresponding to the distance from the projector 1 to the projection surface Sp in a direction parallel to the optical axis of the depth camera 16b and coordinates on a plane perpendicular to the optical axis of the depth camera 16b. Therefore, the control unit 10 can grasp the inclination of the projection surface Sp by obtaining the difference in depth values ​​at multiple positions on the projection surface Sp based on the depth image. In this way, the depth camera 16b is a sensor for detecting the inclination of the projection surface Sp, which is equivalent to the second sensor.

[0034] The image processing unit 14 includes a trapezoidal distortion correction unit 14a. The trapezoidal distortion correction unit 14a corrects the distortion of the projected image, i.e., the trapezoidal distortion, which is generated when the projection surface Sp is tilted relative to the optical axis of the projector 1, i.e., the optical axis of the projection optical system 23, under the control of the control unit 10. Specifically, the trapezoidal distortion correction unit 14a performs processing to deform the projected image into a shape that can offset the trapezoidal distortion based on the correction information input from the control unit 10, so that the projected image becomes the original shape, i.e., a rectangle, on the projection surface Sp tilted relative to the optical axis of the projection optical system 23. In addition, correcting the trapezoidal distortion is equivalent to correcting the shape of the projected image.

[0035] As a functional unit realized by the control program 11a, the control unit 10 has a correction control unit 10a for controlling the correction process of trapezoidal distortion. The correction control unit 10a calculates the inclination of the projection surface Sp based on the measurement result of the projection surface measurement unit 16, and derives correction information for correcting the trapezoidal distortion generated when the projection image is projected onto the projection surface Sp. Then, the correction control unit 10a outputs the derived correction information to the trapezoidal distortion correction unit 14a. The correction information is information that can determine the shape of the projection image to be formed in the pixel area 22i, for example, it can be an equation representing the normal line of the projection surface Sp, or it can be the coordinates of the four corners of the projection image in the two-dimensional coordinate system set in the pixel area 22i. In addition, the position of the depth camera 16b in the projector 1, the direction of the optical axis of the depth camera 16b, the position of the projection optical system 23, and the direction of the optical axis of the projection optical system 23 are fixed, and the correction control unit 10a can derive the inclination of the projection surface Sp relative to the optical axis of the projection optical system 23 from the depth image based on their mutual relationship. Furthermore, the correction control unit 10 a causes the keystone correction unit 14 a to correct keystone distortion according to the inclination of the projection plane Sp with respect to the optical axis of the projection optical system 23 .

[0036] The motion detection unit 17 includes, for example, a gyro sensor, detects the posture of the projector 1, and outputs the detection result to the control unit 10. If the change range of the posture based on the output of the motion detection unit 17 is within a prescribed range, the control unit 10 determines that the projector 1 is stationary, and if the change range of the posture exceeds the prescribed range, the control unit 10 determines that the projector 1 has moved. In addition, the motion detection unit 17 is not limited to the gyro sensor, and other sensors such as an acceleration sensor can also be used. In addition, the motion detection unit 17 can also detect the change of the posture of the projector 1 based on the measurement result of the projection surface measurement unit 16, that is, the change of the distance from the projector 1 to the projection surface Sp. The motion detection unit 17 is equivalent to the first sensor for detecting the posture of the projector 1. In addition, the above-mentioned prescribed range is equivalent to the first range.

[0037] Next, the correction process for correcting keystone distortion will be described.

[0038] The projector 1 of the present embodiment is capable of executing two types of correction processing, namely a high-speed correction processing and a high-precision correction processing, as processing for correcting keystone distortion. The high-speed correction processing is a correction processing that has a shorter processing time than the high-precision correction processing, but has a lower correction accuracy than the high-precision correction processing. The high-precision correction processing is a correction processing that has a higher correction accuracy than the high-speed correction processing, but has a longer processing time than the high-speed correction processing. In the present embodiment, the number of times the projection surface measuring unit 16 measures the projection surface Sp, that is, the number of times the depth image is obtained, is different between the high-speed correction processing and the high-precision correction processing. Specifically, the high-precision correction processing obtains more depth images than the high-speed correction processing. Therefore, the processing time of the high-precision correction processing is longer than that of the high-speed correction processing.

[0039] Figure 3 is an explanatory diagram for explaining high-speed correction processing. Figure 4 : is an explanatory diagram for explaining the high-precision correction process. In both figures, the area A1 surrounded by the double-dashed line represents the area where the projected image is displayed when the correction process is not performed, and the area A2 surrounded by the dotted line represents the area where the projected image is displayed when the correction process is performed. When the correction process is performed, in the pixel area 22i of the liquid crystal light valve 22R, 22G, and 22B, the area inside the area A1 and corresponding to the outside of the area A2 is set to opaque black. That is, in the pixel area 22i, a meaningful image is formed in the area corresponding to the area A2. In addition, in this specification, the "projected image" refers to the image displayed in the area A2 of the image formed in the pixel area 22i.

[0040] In the high-speed correction processing of the present embodiment, the inclination of the projection surface Sp is calculated based on the result of measuring the projection surface Sp once, that is, one depth image, and the trapezoidal distortion is corrected based on the inclination. On the other hand, in the high-precision correction processing of the present embodiment, the inclination of the projection surface Sp is calculated based on the result of measuring the projection surface Sp multiple times at intervals, that is, multiple depth images, and the trapezoidal distortion is corrected based on the inclination. The measurement result of the projection surface measuring unit 16 may contain errors due to various factors, so the correction accuracy is low in one measurement. That is, the shape of the projection image in this case, that is, the shape of the area A2 is as shown in FIG. Figure 3 As shown in FIG. 1 , it is difficult to form an accurate rectangle, and the outline is often tilted. On the other hand, if the projection surface Sp is measured multiple times at intervals and the tilt of the projection surface Sp is determined by the least square method or averaging, the influence of the error is reduced. Figure 4 As shown, the shape of the projected image, that is, the shape of the area A2, becomes closer to a rectangular shape than when the measurement is performed once. However, in the high-precision correction process, since multiple measurements and relatively complex calculations are required, the processing time required for the correction process becomes longer than that for the high-speed correction process.

[0041] That is, when the processing time for obtaining the calculation result of the inclination of the projection surface Sp by the high-speed correction processing is set as the first time, and the processing time for obtaining the calculation result of the inclination of the projection surface Sp by the high-precision correction processing is set as the second time, the second time is longer than the first time. Here, the processing time for obtaining the calculation result includes the time for measuring the projection surface Sp and the time for performing the calculation for calculating the inclination of the projection surface Sp. Therefore, the first time is the period from the start time of measuring the projection surface Sp, that is, the start time of obtaining the information related to the projection surface Sp to the first calculation end time of obtaining the calculation result of the inclination of the projection surface Sp by the high-speed correction processing, and the second time is the period from the start time to the second calculation end time of obtaining the calculation result of the inclination of the projection surface Sp by the high-precision correction processing. The depth camera 16b starts the measurement of the projection surface Sp in response to the fact that the trigger signal indicating the start of the measurement of the projection surface Sp is received from the control unit 10. The start time of the depth camera 16b starting to obtain information related to the projection surface Sp is, for example, the time when the depth camera 16b starts to shoot the projection surface Sp, or the time when the depth camera 16b receives a trigger signal output from the control unit 10 indicating the start of shooting the projection surface. The time for measuring the projection surface Sp is the time required to obtain information related to the projection surface Sp, that is, the depth image. Therefore, the more the number of measured depth images, the longer the time for measuring the projection surface Sp. In addition, the more the number of measured depth images, the more the amount of data, and therefore, the more the number of measured depth images, the longer the time for performing the calculation for calculating the inclination of the projection surface Sp.

[0042] In addition, it is also possible to perform high-speed correction processing based on the output from the second sensor of the first type, and perform high-precision correction processing based on the output from the second sensor of the second type different from the first type. For example, the second sensor of the first type is an ultrasonic sensor not shown, and the second sensor of the second type is the depth camera 16b. In this case, the start time of starting to obtain information related to the projection surface Sp by the ultrasonic sensor is, for example, the time when the ultrasonic sensor starts to emit ultrasonic waves or the time when the ultrasonic sensor receives the second trigger signal output from the control unit 10 to instruct the start of ultrasonic emission. The ultrasonic sensor measures the projection surface Sp once. The rising timing of the trigger signal output from the control unit 10 to instruct the depth camera 16b to start photographing the projection surface Sp is preferably synchronized with the rising timing of the second trigger signal output from the control unit 10 to instruct the ultrasonic sensor to start emitting ultrasonic waves. By synchronizing the trigger signal with the second trigger signal, it is possible to make the start time of the first type of the second sensor starting the measurement of the projection surface Sp the same as the start time of the second type of the second sensor starting the measurement of the projection surface Sp.

[0043] In the present embodiment, when the projector 1 moves, such as when the user adjusts the posture of the projector 1, the correction control unit 10a repeatedly performs the high-speed correction process. Then, after the projector 1 is stationary, the high-precision correction process is performed. In addition, from now on, the projected image corrected by the high-speed correction process will be referred to as a "tentative correction image", and the projected image corrected by the high-precision correction process will be referred to as a "determined correction image". The tentative correction image is temporarily projected before the final correction image is projected onto the projection surface Sp. In the present embodiment, the content shown in the tentative correction image is the same as the content shown in the final correction image except that the accuracy of the correction, that is, the appearance is different.

[0044] Figure 5 : is a flowchart showing the correction process of keystone distortion. Figure 6 is a flowchart showing the high-speed correction process of the first embodiment. Figure 7 This is a flowchart showing the high-precision correction process according to the first embodiment. Figure 5 to Figure 7 The flowchart shown shows a part of the projection method of the projector 1. When the projector 1 is started, the control unit 10 starts the operation corresponding to the control program 11a, and after completing the predetermined initial operation, starts projection of the projection image based on the image data provided by the image providing device 5.

[0045] After the projection of the projected image starts, the control unit 10 monitors the movement of the projector 1 based on the detection result of the motion detection unit 17. In addition, when the change range of the posture of the projector 1 exceeds a predetermined range, such as when the user moves the projector 1 to adjust the position and size of the projected image on the projection surface Sp, the control unit 10 determines that the projector 1 has moved. When the control unit 10 determines that the projector 1 has moved, it will monitor the movement of the projector 1 based on the detection result of the motion detection unit 17. Figure 5 The flow shown starts the correction process to correct the keystone distortion caused by the motion of the projector 1. The predetermined range is the fluctuation range of the posture of the projector 1 so that the motion of the projector 1 appears to be substantially still when viewed by the user, and is determined in advance based on the results of experiments, simulations, sensory tests, and the like.

[0046] In step S101, the correction control unit 10a of the control unit 10 performs a high-speed correction process. Specifically, Figure 6 As shown in FIG. 1 , the correction control unit 10a obtains a depth image from the projection surface measuring unit 16 (step S111), and calculates the tilt of the projection surface Sp based on the obtained depth image (step S112). Then, the correction control unit 10a outputs correction information corresponding to the calculated tilt of the projection surface Sp to the keystone correction unit 14a (step S113). When the keystone correction unit 14a performs correction processing based on the correction information, as shown in FIG. Figure 3 As shown in FIG. 1 , the image projection unit 15 of the projector 1 projects a tentatively corrected image in a state where the trapezoidal distortion is suppressed to some extent (step S114). That is, the correction control unit 10a outputs the correction information to the trapezoidal distortion correction unit 14a, and the projection optical system 23 of the image projection unit 15 projects the projection image corrected according to the correction information. The operation for calculating the inclination of the projection surface Sp in step S112 is equivalent to the first correction operation. In addition, the tentatively corrected image projected in step S114 is a projection image based on the operation result of the first correction operation, and is equivalent to the first projection image.

[0047] return Figure 5In step S102, the correction control unit 10a determines whether the projector 1 is stationary based on the output from the motion detection unit 17. For example, when the variation range of the posture of the projector 1 is within a predetermined range and this state continues for a predetermined time, the correction control unit 10a determines that the projector 1 is stationary. On the other hand, when the variation range of the posture of the projector 1 still exceeds the predetermined range, the correction control unit 10a determines that the projector 1 has moved. Then, when the projector 1 is stationary (step S102: Yes), the correction control unit 10a transfers the process to step S103. On the other hand, when the projector 1 has moved (step S102: No), the correction control unit 10a returns the process to step S101. Furthermore, the correction control unit 10a repeatedly performs the high-speed correction process until the projector 1 is stationary, that is, the variation range of the posture of the projector 1 is within the predetermined range.

[0048] When the projector 1 is stationary and the process is transferred to step S103, the correction control unit 10a performs the high-speed correction process again. As a result, a tentative correction image corrected based on the depth image obtained in a stationary state is projected onto the projection surface Sp. Therefore, during the high-precision correction process described later, the tentative correction image is projected in a shape close to the original shape compared to the tentative correction image corrected based on the depth image obtained when the projector 1 is moving. However, step S103 may also be omitted.

[0049] Next, in step S104, the correction control unit 10a performs a high-precision correction process. Specifically, Figure 7 As shown, the correction control unit 10a obtains a depth image from the projection surface measuring unit 16 (step S121). Then, the correction control unit 10a determines whether all the required number of depth images have been obtained (step S122). If all the depth images have not been obtained (step S122: No), the depth images are repeatedly obtained (step S121) until all the depth images are obtained. On the other hand, if all the depth images have been obtained (step S122: Yes), the correction control unit 10a calculates the inclination of the projection surface Sp based on the obtained multiple depth images using the least squares method or the like (step S123), and derives correction information corresponding to the inclination. Then, the correction control unit 10a outputs the derived correction information to the keystone correction unit 14a (step S124). When the keystone correction unit 14a performs correction processing based on the correction information, as shown in FIG. Figure 4As shown, the image projection unit 15 of the projector 1 projects a determined corrected image in a state in which the trapezoidal distortion is corrected with high accuracy (step S125). That is, the correction control unit 10a outputs the correction information to the trapezoidal distortion correction unit 14a, so that the projection optical system 23 of the image projection unit 15 projects the projection image corrected according to the correction information. The operation for calculating the inclination of the projection surface Sp in step S123 is equivalent to the second correction operation. In addition, the determined corrected image projected in step S125 is a projection image based on the operation result of the second correction operation, which is equivalent to the second projection image.

[0050] In addition, the number of times the depth image is acquired in the high-precision correction process is not particularly limited and may be several times or more than 100 times. For example, when the depth image is acquired at a frame frequency of 30 Hz, it takes more than 3 seconds to acquire the depth image 100 times.

[0051] After executing the high-precision correction process, the correction control unit 10 a ends a series of correction processes.

[0052] As described above, according to the projection method, the projector 1 , and the control program 11 a of the present embodiment, the following effects can be obtained.

[0053] According to the present embodiment, the projector 1 projects a tentatively corrected image corrected by a high-speed correction process onto the projection surface Sp before projecting a definite corrected image corrected by a high-precision correction process onto the projection surface Sp. That is, before the definite corrected image corrected by a relatively long time is projected, the tentatively corrected image corrected in a relatively short time is projected onto the projection surface Sp. Therefore, the sense of incongruity caused by the long waiting time until the definite corrected image is projected onto the projection surface Sp can be reduced.

[0054] According to the present embodiment, when the projector 1 moves, such as when the user adjusts the posture of the projector 1, a tentative correction image corrected in a short time is projected, and after the posture of the projector 1 is stationary, a final correction image is projected. Therefore, before the posture of the projector 1 is stationary, the projection image can be projected in a calibrated state.

[0055] According to the present embodiment, both the high-speed correction process and the high-precision correction process are performed based on the output from the common depth camera 16 b , so that the correction process algorithm can be made common in the high-speed correction process and the high-precision correction process.

[0056] 2. Second Implementation Method

[0057] Hereinafter, a projector 1 according to a second embodiment will be described with reference to the drawings. The projector 1 according to this embodiment has the same hardware configuration as the projector 1 according to the first embodiment, and therefore its description will be omitted. In the second embodiment, the operation of the projector 1 in the high-precision calibration process is different from that in the first embodiment.

[0058] Figure 8 This is a flowchart showing the high-precision correction process according to the second embodiment.

[0059] The projection method of this embodiment includes steps S131 to S134 between steps S123 and S124. The operations of the steps other than steps S131 to S134 are the same as those of the first embodiment, and thus the description thereof will be omitted.

[0060] After calculating the tilt of the projection plane Sp based on the plurality of depth images in step S123 , the correction control unit 10 a executes steps S131 to S134 before outputting correction information corresponding to the calculated tilt of the projection plane Sp to the keystone correction unit 14 a in step S124 .

[0061] In steps S131 to S134, the correction control unit 10a gradually and stepwise changes the shape of the projected image from the shape of the projected image projected at the current time point, that is, the shape of the provisional correction image determined by the last high-speed correction processing (step S103), to the shape of the definite correction image determined by the high-precision correction processing.

[0062] Specifically, first, in step S131, the correction control unit 10a determines the number of stages for changing the projected image according to the difference between the shape of the tentatively corrected image projected at the current time point and the shape of the final corrected image corresponding to the inclination of the projection surface Sp determined in step S123. Here, it is determined that the greater the difference in the shape of the projected image, the greater the number of stages. And the correction control unit 10a determines the correction information corresponding to the shape of the projected image in each stage, that is, the intermediate correction information. In addition, the number of stages may also be fixed.

[0063] Next, in step S132, the correction control unit 10a outputs the intermediate correction information corresponding to the shape of the projection image at the current stage to the keystone correction unit 14a. When the keystone correction unit 14a performs the correction processing according to the intermediate correction information, in step S133, the image projection unit 15 of the projector 1 projects the projection image in a shape that is one stage closer to the shape of the final correction image.

[0064] Next, in step S134, the correction control unit 10a determines whether the intermediate correction information has been output in all the stages determined in step S131. Then, when the output of the intermediate correction information has not been completed in all the stages (step S134: No), the correction control unit 10a returns the process to step S132, outputs the intermediate correction information of the next stage to the keystone correction unit 14a, and then repeats the above-mentioned operation. On the other hand, when the output of the intermediate correction information has been completed in all the stages (step S134: Yes), the correction control unit 10a transfers the process to step S124, and outputs the correction information corresponding to the inclination of the projection plane Sp calculated in step S123, that is, the correction information corresponding to the shape of the determined correction image, to the keystone correction unit 14a.

[0065] As described above, according to this embodiment, the shape of the projected image gradually changes in stages from the shape of the projected image determined by the final high-speed correction processing toward the shape of the projected image determined by the high-precision correction processing, so the shape of the projected image does not change drastically, which can reduce the user's sense of disharmony.

[0066] 3. Third Implementation Method

[0067] Hereinafter, a projector 1 according to a third embodiment will be described with reference to the drawings. The projector 1 according to this embodiment has the same hardware configuration as the projector 1 according to the first embodiment, and therefore its description will be omitted. In the third embodiment, the operation of the projector 1 in the high-precision calibration process is different from that in the first embodiment.

[0068] Fig. 9 This is a flowchart showing the high-precision correction process according to the third embodiment.

[0069] The projection method of this embodiment includes steps S141 to S144 in addition to steps S121 to S125 in the first embodiment. The operations of the steps other than steps S141 to S144 are the same as those in the first embodiment, and thus description thereof is omitted.

[0070] In step S122, the correction control unit 10a determines whether all the required number of depth images have been acquired. Then, if all the depth images have been acquired (step S122: Yes), the process is transferred to step S123 in the same manner as in the first embodiment. On the other hand, if all the depth images have not been acquired (step S122: No), the correction control unit 10a transfers the process to step S141.

[0071] In step S141, the correction controller 10a determines whether the number of depth images obtained has reached a specified number. Here, the specified number is set to a value that is less than the number of depth images required for high-precision correction processing. For example, when the number of depth images required for high-precision correction processing is 100, the specified number can be set to 20. Then, the correction control unit 10a returns the process to step S121 if the number of depth images obtained has not reached the specified number (step S141: No), and transfers the process to step S142 if the number of depth images obtained has reached the specified number (step S141: Yes).

[0072] In step S142, the correction control unit 10a calculates the inclination of the projection surface Sp based on the specified number of depth images obtained, and derives correction information corresponding to the inclination. Then, in step S143, the correction control unit 10a outputs the derived correction information to the keystone correction unit 14a. When the keystone correction unit 14a performs correction processing based on the correction information, in step S144, the image projection unit 15 of the projector 1 projects a projection image in a state corrected to a certain extent. Thereafter, the correction control unit 10a resets the counter used to determine whether the number of depth images has reached the specified number to 0, and returns the process to step S121.

[0073] In the present embodiment, since the operation is performed as described above, the shape of the projected image is updated each time a specified number of depth images are obtained during the execution of the high-precision correction process. Then, after all the required depth images are obtained, the shape of the projected image becomes the shape of the final determined correction image. For example, when the number of depth images required for the high-precision correction process is 100 and the specified number is 20, the shape of the projected image is updated when the number of depth images obtained becomes a multiple of 20, i.e., 20, 40, 60, 80, and when the number of depth images obtained becomes 100, it is projected in the final shape.

[0074] As described above, according to this embodiment, the same effects as those of the second embodiment can be obtained.

[0075] 4. Fourth Implementation Method

[0076] Hereinafter, the projector 1 of the fourth embodiment will be described with reference to the accompanying drawings. The projector 1 of this embodiment has the same hardware structure as the projector 1 of the first embodiment, so its description is omitted. In the fourth embodiment, the operation of the projector 1 in the high-speed correction process is different from that in the first embodiment. Specifically, the projector 1 applies the operation of the high-precision correction process in the second embodiment to the high-speed correction process.

[0077] Fig.10This is a flowchart showing the high-speed correction process according to the fourth embodiment.

[0078] The projection method of this embodiment includes steps S151 to S154 between step S112 and step S113. The operations of the steps other than steps S151 to S154 are the same as those of the first embodiment, and thus the description thereof is omitted.

[0079] After calculating the tilt of the projection plane Sp from one depth image in step S112 , the correction control unit 10 a executes steps S151 to S154 before outputting correction information corresponding to the calculated tilt of the projection plane Sp to the keystone correction unit 14 a in step S113 .

[0080] In steps S151 to S154, the correction control unit 10a gradually changes the shape of the projected image from the initial shape determined before the high-speed correction process to the shape of the provisional corrected image determined by the high-speed correction process. Here, the initial shape may be the shape before the correction process is performed or the shape determined by the last correction process.

[0081] First, in step S151, the correction control unit 10a determines the number of stages for changing the projection image according to the difference between the initial shape and the shape of the provisional correction image corresponding to the inclination of the projection surface Sp determined in step S112. Here, it is determined that the greater the difference in the shape of the projection image, the greater the number of stages. And the correction control unit 10a determines the correction information corresponding to the shape of the projection image in each stage, that is, the intermediate correction information. In addition, the number of stages may also be fixed.

[0082] Next, in step S152, the correction control unit 10a outputs the intermediate correction information corresponding to the shape of the projection image at the current stage to the keystone correction unit 14a. When the keystone correction unit 14a performs the correction process based on the intermediate correction information, in step S153, the image projection unit 15 of the projector 1 projects the projection image in a shape that is one stage closer to the shape of the provisional correction image.

[0083] Next, in step S154, the correction control unit 10a determines whether the intermediate correction information has been output in all the stages determined in step S151. Then, when the output of the intermediate correction information has not been completed in all the stages (step S154: No), the correction control unit 10a returns the process to step S152, outputs the intermediate correction information of the next stage to the keystone correction unit 14a, and then repeats the above-mentioned operation. On the other hand, when the output of the intermediate correction information has been completed in all the stages (step S154: Yes), the correction control unit 10a transfers the process to step S113, and outputs the correction information corresponding to the inclination of the projection plane Sp calculated in step S112, that is, the correction information corresponding to the shape of the provisional correction image, to the keystone correction unit 14a.

[0084] As described above, according to the present embodiment, the shape of the projected image gradually changes in stages toward the shape of the provisional corrected image determined by the high-speed correction process. Therefore, the shape of the projected image does not change suddenly, and the user's sense of discomfort can be reduced.

[0085] In addition, the high-speed correction processing of this embodiment may be combined with the high-precision correction processing of any one of the first to third embodiments. In addition, the high-speed correction processing in step S101, i.e., the high-speed correction processing before the projector 1 comes to rest, may be performed by the high-speed correction processing of the first embodiment, and the high-speed correction processing in step S103, i.e., the high-speed correction processing after the projector 1 comes to rest, may be performed by the high-speed correction processing of the fourth embodiment.

[0086] 5. Modifications

[0087] The above-mentioned embodiment may be modified as follows.

[0088] In each of the above-mentioned embodiments, the image processing unit 14 may also add a frame-shaped image to the periphery of the tentatively corrected image. Thus, it is possible to notify the user that the current projected image is a tentatively corrected image rather than a final determined corrected image. In addition, the image processing unit 14 may also perform blurring processing on the periphery of the tentatively corrected image. Thus, as described above, it is possible to notify the user that it is a tentatively corrected image, and it is possible to make the tilted outline of the projected image inconspicuous. In this way, when projecting a projected image corrected by high-speed correction processing, it is preferable to notify the user that it is a tentatively corrected image by making the shape of the periphery of the projected image different from the shape of the area different from the periphery. In addition, a message notifying that it is a tentatively corrected image may be overlapped with the projected image.

[0089] In each of the above-mentioned embodiments, in the high-speed correction process, one depth image is used in the calculation of the inclination of the projection surface Sp, and in the high-precision correction process, multiple depth images are used, but it is not limited to this method. For example, if the number of depth images used for the high-precision correction process is greater than the number of depth images used for the high-speed correction process, the number of depth images used for the high-speed correction process may be multiple. In other words, the calculation of the inclination of the projection surface Sp in the high-speed correction process may be based on the output of the first order of quantities from the depth camera 16b, and the calculation of the inclination of the projection surface Sp in the high-precision correction process may be based on the output of the second order of quantities from the depth camera 16b which is greater than the first order. In addition, in the case of using multiple depth images for the high-speed correction process, the inclination of the projection surface Sp may be obtained by using the least squares method or averaging, as in the high-precision correction process.

[0090] In addition, the number of depth values ​​used to calculate the inclination of the projection surface Sp among the depth values ​​contained in the depth image of the specified resolution may be different in the high-speed correction process and the high-precision correction process. Specifically, the number of depth values ​​used to calculate the inclination of the projection surface Sp in the high-speed correction process may be less than the number of depth values ​​used to calculate the inclination of the projection surface Sp in the high-precision correction process. In this case, the number of depth images used for the high-speed correction process and the number of depth images used for the high-precision correction process may also be the same.

[0091] In each of the above-mentioned embodiments, the high-speed correction process and the high-precision correction process are performed based on the output from the depth camera 16b for detecting the inclination of the projection surface Sp, but are not limited to this structure. For example, in addition to the combination of the above-mentioned depth camera 16b and the ultrasonic sensor, the high-speed correction process and the high-precision correction process may also be performed based on the output from a sensor for detecting the posture of the projector 1, such as an acceleration sensor, a gyro sensor, or a geomagnetic sensor, or may be performed only based on the output from another second sensor for detecting the inclination of the projection surface Sp, such as an ultrasonic sensor. In addition, as described above, different sensors may also be used in the high-speed correction process and the high-precision correction process. For example, the calculation of the inclination of the projection surface Sp in the high-speed correction process may be performed based on the output from the sensor for detecting the posture of the projector 1, and the calculation of the inclination of the projection surface Sp in the high-precision correction process may be performed based on the output from the depth camera 16b. According to this structure, the shape of the projected image can be appropriately corrected based on the output from the sensors suitable for the high-speed correction process and the high-precision correction process, respectively. In addition, in this structure, the sensor for detecting the posture of the projector 1 is the same as the sensor constituting the motion detection unit 17, and is equivalent to the first sensor. That is, the sensor constituting the motion detection unit 17 may also be used as a sensor for high-speed correction processing.

[0092] In addition, as in the above-mentioned embodiments, when the same type of second sensor, for example, the depth camera 16b, is used in the high-speed correction process and the high-precision correction process, the number of data related to the projection surface Sp obtained by the second sensor in the high-speed correction process is the same as the number of data related to the projection surface Sp obtained by the second sensor in the high-precision correction process, and the keystone correction algorithm used in the high-speed correction process is different from the keystone correction algorithm used in the high-precision correction process. The calculation time of the keystone correction algorithm used in the high-speed correction process is shorter than the calculation time of the keystone correction algorithm used in the high-precision correction process.

[0093] The acceleration sensor among the sensors for detecting the posture of the projector 1 is suitable for high-speed correction processing, but if averaging is performed or outliers are appropriately excluded, it can also be used for high-precision correction processing. In addition, the geomagnetic sensor can be used not only for high-speed correction processing, but also for high-precision correction processing by averaging multiple detection results or using the least square method. On the other hand, since the gyro sensor drifts, it is preferably used only for high-speed correction processing. In addition, in the high-speed correction processing, when calculating the posture of the projector 1 based on fewer detection values, averaging and smoothing using a Kalman filter can also be performed. In addition, at least one of the high-speed correction processing and the high-precision correction processing can also combine and utilize the detection results of multiple sensors illustrated. In this case, in order to estimate the posture of the projector 1, an extended Kalman filter can also be used.

[0094] In the above-mentioned embodiments, the planar projection surface Sp is used as the projection object of the projector 1, but the projection object may also be an object other than a plane, such as a three-dimensional object. In addition, the depth camera 16b can measure the shape, concave and convex surface conditions of the projection object when it has a corresponding resolution. Therefore, the image processing unit 14 can also perform corrections corresponding to the shape of the projection object and corrections for distortion caused by the surface conditions of the projection object. In this case, the depth camera 16b as the second sensor functions as a sensor for detecting the state of the projection object, such as the tilt, shape, and surface conditions of the projection object.

[0095] In the above-mentioned embodiments, the projection surface measurement unit 16 includes the depth camera 16b that outputs a depth image including a plurality of depth values, but the configuration of the projection surface measurement unit 16 is not limited to this configuration. For example, a plurality of depth sensors that obtain and output a single depth value using the ToF method may be arranged.

[0096] In the above embodiment, as the light modulator, the transmissive liquid crystal light valves 22R, 22G, and 22B are used, but a reflective light modulator such as a reflective liquid crystal light valve may also be used. In addition, a digital micromirror device or the like may be used to modulate the light emitted from the light source 21 by controlling the emission direction of the incident light for each micromirror as a pixel. In addition, the structure is not limited to having a plurality of light modulators for each color light, and a structure may also be configured to modulate a plurality of color lights in a time-sharing manner using one light modulator.

[0097] 6. Summary of the Disclosure

[0098] The following is a summary of the present disclosure.

[0099] (Supplementary Note 1) A projection method in a projection device, characterized in that the projection device projects a projection image onto a projection object, and the projection method includes: performing a first correction operation for correcting the shape of the projection image; projecting a first projection image as the projection image based on the operation result of the first correction operation onto the projection object; performing a second correction operation for correcting the shape of the projection image; and projecting a second projection image as the projection image based on the operation result of the second correction operation onto the projection object, wherein a processing time for obtaining the operation result of the first correction operation is a first time, and a processing time for obtaining the operation result of the second correction operation is a second time longer than the first time, and before projecting the second projection image onto the projection object, the first projection image is projected onto the projection object.

[0100] According to the projection method, the first projection image is projected onto the projection object before the second projection image is projected onto the projection object. That is, the first projection image corrected in a relatively short time is projected before the second projection image corrected in a relatively long time is projected onto the projection object. Therefore, the sense of discomfort caused by the long waiting time until the second projection image is projected onto the projection object can be reduced.

[0101] (Supplementary Note 2) The projection method according to Supplementary Note 1 is characterized in that, based on the output from the first sensor for detecting the posture of the projection device, it is determined whether the variation range of the posture converges within the first range, and when it is determined that the variation range does not converge within the first range, the first projection image is projected onto the projection object, and when it is determined that the variation range converges within the first range, the second projection image is projected onto the projection object.

[0102] According to the projection method, when the projection device moves, such as when the user adjusts the posture of the projection device, the first projection image that is corrected in a relatively short time is projected. And after the posture of the projection device is stationary, the second projection image that takes a relatively long time to be corrected is projected. Therefore, the projection image can be projected in a state where the correction is performed before the posture of the projection device is stationary.

[0103] (Supplementary Note 3) The projection method according to Supplementary Note 1 or 2 is characterized in that the first correction operation is performed based on the output from the first sensor for detecting the posture of the projection device, and the second correction operation is performed based on the output from the second sensor for detecting the state of the projection object.

[0104] According to this projection method, the shape of the projected image can be appropriately corrected based on the outputs from the sensors respectively suitable for the first correction calculation and the second correction calculation.

[0105] (Supplement 4) The projection method according to Supplement 1 or 2 is characterized in that the first correction operation is performed based on the output of a first number of quantities from a second sensor for detecting the state of the projection object, and the second correction operation is performed based on the output of a second number of quantities that is greater than the first number from the second sensor.

[0106] According to this projection method, both the first correction calculation and the second correction calculation are performed based on the output from the common sensor. Therefore, the calculation algorithms can be made common in the first correction calculation and the second correction calculation.

[0107] (Supplementary Note 5) The projection method according to any one of Supplementary Notes 1 to 4, wherein a shape of a periphery of the first projection image is different from a shape of a region of the first projection image different from the periphery.

[0108] According to this projection method, the user can easily determine that the projected image is the first projected image, that is, the correction calculation is still in progress, based on the shape of the periphery of the projected image.

[0109] (Supplementary Note 6) The projection method according to Supplementary Note 5, characterized in that blurring processing is performed on the periphery of the first projection image.

[0110] According to this projection method, even when the outline of the projected image due to the first correction calculation is tilted, the tilt can be made inconspicuous.

[0111] (Supplementary Note 7) A projection device, characterized in that the projection device comprises: a projection optical system, which includes a lens for projecting a projection image onto a projection object; and one or more processors, wherein the one or more processors perform the following processing: performing a first correction operation for correcting the shape of the projection image; projecting a first projection image as the projection image based on the operation result of the first correction operation from the projection optical system to the projection object; performing a second correction operation for correcting the shape of the projection image; and projecting a second projection image as the projection image based on the operation result of the second correction operation from the projection optical system to the projection object, wherein a processing time for obtaining the operation result of the first correction operation is a first time, and a processing time for obtaining the operation result of the second correction operation is a second time longer than the first time, and the one or more processors project the first projection image from the projection optical system to the projection object before projecting the second projection image from the projection optical system to the projection object.

[0112] According to the projection device, the first projection image is projected onto the projection object before the second projection image is projected onto the projection object. That is, the first projection image corrected in a relatively short time is projected before the second projection image corrected in a relatively long time is projected onto the projection object. Therefore, the sense of discomfort caused by the long waiting time until the second projection image is projected onto the projection object can be reduced.

[0113] (Supplementary Note 8) A program, characterized in that the program causes a computer that controls a projection device having a projection optical system to execute processing, the projection optical system including a lens for projecting a projection image onto a projection object, the processing being the following processing: performing a first correction operation for correcting the shape of the projection image; projecting a first projection image as the projection image based on the operation result of the first correction operation from the projection optical system to the projection object; performing a second correction operation for correcting the shape of the projection image; and projecting a second projection image as the projection image based on the operation result of the second correction operation from the projection optical system to the projection object, the processing time for obtaining the operation result of the first correction operation is a first time, and the processing time for obtaining the operation result of the second correction operation is a second time longer than the first time, so that the computer projects the first projection image from the projection optical system to the projection object before projecting the second projection image from the projection optical system to the projection object.

[0114] According to the program, the first projection image is projected onto the projection object before the second projection image is projected onto the projection object. That is, the first projection image corrected in a relatively short time is projected before the second projection image corrected in a relatively long time is projected onto the projection object. Therefore, the sense of discomfort caused by the long waiting time until the second projection image is projected onto the projection object can be reduced.

Claims

1. A projection method in a projection device, characterized in that: The projection device projects a projection image onto a projection object, and the projection method comprises: performing a first correction operation for correcting the shape of the projected image; projecting a first projection image as the projection image based on a calculation result of the first correction calculation onto the projection object; performing a second correction operation for correcting the shape of the projected image; and projecting a second projection image as the projection image based on a calculation result of the second correction calculation onto the projection object, The processing time for obtaining the calculation result of the first correction calculation is a first time, a processing time for obtaining a calculation result of the second correction calculation is a second time that is longer than the first time; Before projecting the second projection image onto the projection object, the first projection image is projected onto the projection object.

2. The projection method according to claim 1, characterized in that: determining whether the variation range of the posture is within a first range based on an output from a first sensor for detecting the posture of the projection device, When it is determined that the variation range does not fall within the first range, the first projection image is projected onto the projection object, and when it is determined that the variation range falls within the first range, the second projection image is projected onto the projection object.

3. The projection method according to claim 1, characterized in that: performing the first correction operation based on an output from a first sensor for detecting a posture of the projection device, The second correction calculation is performed based on an output from a second sensor for detecting a state of the projection object.

4. The projection method according to claim 1, characterized in that: performing the first correction operation based on the output of the first order from the second sensor for detecting the state of the projection object, The second correction calculation is performed based on an output from the second sensor of a second order greater than the first order.

5. The projection method according to claim 1, characterized in that: The shape of the periphery of the first projected image is different from the shape of a region of the first projected image different from the periphery.

6. The projection method according to claim 5, characterized in that: A blurring process is performed on the periphery of the first projected image.

7. A projection device, characterized in that: The projection device comprises: a projection optical system including a lens for projecting a projection image onto a projection object; and one or more processors, The one or more processors perform the following processing: performing a first correction operation for correcting the shape of the projected image; projecting a first projection image as the projection image based on a calculation result of the first correction calculation onto the projection object from the projection optical system; performing a second correction operation for correcting the shape of the projected image; as well as projecting a second projection image as the projection image based on a calculation result of the second correction calculation from the projection optical system onto the projection object, The processing time for obtaining the calculation result of the first correction calculation is a first time, a processing time for obtaining a calculation result of the second correction calculation is a second time that is longer than the first time; The one or more processors project the first projection image from the projection optical system onto the projection object before projecting the second projection image from the projection optical system onto the projection object.

8. A program product, characterized in that The program product causes a computer that controls a projection device having a projection optical system including a lens for projecting a projection image onto a projection object to execute a process, wherein: performing a first correction operation for correcting the shape of the projected image; projecting a first projection image as the projection image based on a calculation result of the first correction calculation onto the projection object from the projection optical system; performing a second correction operation for correcting the shape of the projected image; as well as projecting a second projection image as the projection image based on a calculation result of the second correction calculation from the projection optical system onto the projection object, The processing time for obtaining the calculation result of the first correction calculation is a first time, a processing time for obtaining a calculation result of the second correction calculation is a second time that is longer than the first time; The computer is caused to project the first projection image from the projection optical system onto the projection object before the second projection image is projected from the projection optical system onto the projection object.

Images