Correction method and projector

Abstract

To improve an accuracy of correction when a projector corrects a projection image.SOLUTION: A correction method of a projection image PI projected on a projection surface PP from a projector 1 includes steps of: acquiring depth information DI indicating a plurality of distances from a distance sensor 15 to a plurality of positions on the projection surface PP based on an output of the distance sensor 15 that irradiates the projection surface PP with irradiation light and receives reflected light reflected on the projection surface PP; acquiring intensity information SI indicating the intensity of the reflected light at each of the plurality of positions; generating the parameters defining the projection plane PP based on the depth information DI and the intensity information SI; and correcting a projection image PI based on the parameters.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a correction method and a projector. [Background technology]

[0002] Conventionally, a technique has been used in which a normal vector of the projection surface is determined using a measurement value obtained by a distance sensor provided in the projector, and the determined normal vector is used to correct the projected image.

[0003] For example, a projector disclosed in Patent Document 1 uses a ToF (Time of Flight) sensor to acquire depth information of multiple light points on a projection surface, and determines a normal vector of the projection surface based on the depth information. Furthermore, the projector acquires offset information of the projector based on the normal vector, and corrects the projected image based on the offset information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2022 / 193560 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the technology disclosed in Patent Document 1, the accuracy of the depth information of the multiple light points may be reduced depending on the inclination or shape of the projection surface. If the scale of the original image is corrected based on the depth information with reduced accuracy, the result of the correction will also be reduced. [Means for solving the problem]

[0006] A correction method according to an aspect of the present invention is a method for correcting a projection image projected from a projector onto a projection surface, and is characterized in that it includes: irradiating the projection surface with irradiation light and acquiring depth information indicating multiple distances from the distance sensor to multiple positions on the projection surface based on the output of a distance sensor that receives reflected light reflected from the projection surface; acquiring intensity information indicating the intensity of the reflected light at each of the multiple positions; generating parameters that define the projection surface based on the depth information and the intensity information; and correcting the projection image based on the parameters.

[0007] A projector according to one aspect of the present invention is characterized in that it includes one or more processors that perform the following operations: irradiating a projection surface with irradiation light and acquiring depth information indicating multiple distances from the distance sensor to multiple positions on the projection surface based on the output of a distance sensor that receives reflected light reflected from the projection surface; acquiring intensity information indicating the intensity of the reflected light at each of the multiple positions; generating parameters that define the projection surface based on the depth information and the intensity information; and correcting the projected image based on the parameters. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a projector 1. [Figure 2] 4 is a diagram showing an example of depth information acquired by an acquisition unit 122. FIG. [Figure 3] 4 is an explanatory diagram of the correspondence between depth information and intensity information acquired by an acquisition unit 122. FIG. [Figure 4] 4 is an explanatory diagram of the correspondence between depth information and intensity information acquired by an acquisition unit 122. FIG. [Figure 5] 10A and 10B are diagrams showing depth information DI of pixel IM(1,6) in multiple frames. [Figure 6] 10 is a diagram showing depth information DI of pixel IM(1,6) and pixels IM(1,5), IM(2,5), and IM(2,6) adjacent to pixel IM(1,6) in the first frame. [Figure 7] 4 is a flowchart showing the operation of the projector 1. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, in each drawing, the dimensions and scale of each part are appropriately different from those of the actual parts. Furthermore, since the embodiments described below are preferred specific examples of the present invention, various technically preferable limitations are applied, but the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited.

[0010] 1: First embodiment The projector 1 and correction method according to the first embodiment will be described below with reference to FIGS.

[0011] 1-1: Configuration of the first embodiment FIG. 1 is a block diagram showing an example of the configuration of a projector 1 according to a first embodiment. The projector 1 includes a projection device 11, a processing device 12, a storage device 13, a communication device 14, and a distance sensor 15. The elements of the projector 1 are connected to one another by one or more buses for communicating information. Furthermore, the elements of the projector 1 are made up of one or more devices. Some elements of the projector 1 may be omitted.

[0012] The projection device 11 is a device that projects a projection image PI generated by a projection image generation unit 121 (described later) onto a screen SC, a wall, or the like. The projection device 11 projects various images under the control of the processing device 12. The projection device 11 includes, for example, a light source, a liquid crystal panel, and a projection lens, modulates light from the light source using the liquid crystal panel, and projects the modulated light onto the screen SC, a wall, or the like via the projection lens. The aspect in which the projection device 11 includes a liquid crystal panel is merely an example, and the aspect according to this embodiment is not limited to this. For example, this embodiment can also be applied to a DLP (Digital Light Processing: registered trademark) that includes a DMD (Digital Mirror Device) instead of a liquid crystal panel.

[0013] The processing device 12 is a processor that controls the entire projector 1, and is configured, for example, by one or more chips. The processing device 12 is configured, for example, by a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, a register, etc. Note that some or all of the functions of the processing device 12 may be realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The processing device 12 executes various processes in parallel or sequentially.

[0014] The storage device 13 is a recording medium readable by the processing device 12, and stores a plurality of programs including the control program PR1 executed by the processing device 12. The storage device 13 may be configured by at least one of, for example, a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), and a random access memory (RAM). The storage device 13 may also be called a register, a cache, a main memory, a primary storage device, or the like.

[0015] The communication device 14 is hardware that serves as a transmitting / receiving device for communicating with other devices. In particular, in this embodiment, the communication device 14 is a communication device for connecting the projector 1 to other devices via a wired or wireless connection. The communication device 14 is also called, for example, a network device, a network controller, a network card, or a communication module.

[0016] The distance sensor 15 measures the distance between the distance sensor 15 and an object located around the projector 1. In this embodiment, the distance sensor 15 irradiates the projection surface PP of the screen SC with irradiation light and receives the light reflected by the projection surface PP. The distance sensor 15 measures the distance between the distance sensor 15 and multiple points on the projection surface PP based on the detection result of the reflected light. The distance sensor 15 is preferably a ToF sensor. Alternatively, the distance sensor 15 may be, for example, a LIDAR (Light Detection and Ranging) sensor. The distance sensor 15 of this embodiment has 6px pixels in each of the X-axis direction and the Y-axis direction. Therefore, the number of pixels (resolution) of the distance sensor 15 of this embodiment is 6px x 6px.

[0017] The processing device 12 reads out and executes the control program PR1 from the storage device 13, thereby functioning as a projection image generation unit 121, an acquisition unit 122, a parameter generation unit 123, a correction unit 124, and a projection control unit 125. The control program PR1 may be transmitted from another device, such as a server that manages the projector 1, via a communication network (not shown).

[0018] The projection image generation unit 121 generates the projection image PI based on the input image acquired by the projection image generation unit 121. Note that the projection image generation unit 121 may acquire the input image from outside the projector 1, or may acquire the input image stored in the storage device 13.

[0019] Based on the output of the distance sensor 15, the acquisition unit 122 acquires depth information DI indicating multiple distances from the distance sensor 15 to multiple positions on the projection surface PP, and intensity information SI indicating the intensity of reflected light at each of the multiple positions.

[0020] FIG. 2 is a diagram illustrating an example of depth information DI acquired by the acquisition unit 122. As shown in FIG. 2, the projector 1 and the projection surface PP are assumed to be installed in an XYZ coordinate system, which is a world coordinate system. The optical axis of the projector 1 is assumed to be substantially parallel to the Z axis. The optical axis of the projector 1 is, for example, the optical axis of the projection lens. Here, "substantially parallel" means parallel within a tolerance range. For example, if the distance sensor 15 is a ToF sensor, the distance sensor 15 irradiates multiple linear laser beams L at multiple positions on the projection surface PP. The multiple linear laser beams L irradiated onto the projection surface PP form multiple light points LP on the projection surface PP. The distance sensor 15 outputs depth information DI indicating multiple distances from the distance sensor 15 to the multiple light points LP based on the time from the irradiation to when a light-receiving sensor included in the distance sensor 15 detects reflected light from the multiple light points LP formed on the projection surface PP by the multiple linear laser beams L. The acquisition unit 122 acquires depth information DI output from a ToF sensor serving as the distance sensor 15. Fig. 2 shows an example in which light spots LP are formed at eight positions out of a maximum of 36 positions on the projection surface PP that the distance sensor 15 can detect.

[0021] If the distance sensor 15 is a ToF sensor, the distance sensor 15 may irradiate the projection surface PP with a surface-emitting laser instead of irradiating the projection surface PP with a plurality of linear laser beams L as described above. In this case, a plurality of light-receiving sensors discretely installed in the distance sensor 15 detect the light reflected from the projection surface PP.

[0022] 3 and 4 are diagrams illustrating the correspondence between the depth information DI and the intensity information SI acquired by the acquisition unit 122. FIG.

[0023] As described above, the distance sensor 15 measures the distance between the distance sensor 15 and a plurality of points on the projection surface PP based on the detection result of the reflected light. As an example, let us assume that the projection surface PP is laid out on the xy plane as shown in FIG. 3. The projection surface PP includes a pixel group IMs, which is made up of multiple pixels IM(1,1) to IM(6,6). In FIG. 3, pixels IM(1,1) to IM(6,1) are arranged in this order in the x-axis direction. Pixel IM(1,1) is located on the -x side of pixel IM(6,1). Pixels IM(1,1) to IM(1,6) are also arranged in this order in the y-axis direction. Pixel IM(1,1) is located on the -y side of pixel IM(1,6).

[0024] The acquisition unit 122 acquires depth information DI indicating the distance between the distance sensor 15 and the center point of each of pixels IM(1,1) to IM(6,6) at multiple points in time. The acquisition unit 122 acquires depth information DI for the nth frame at point in time t=tn, where n is a natural number. The depth information DI for the nth frame is composed of depth information DI(1,1,n) to DI(6,6,n) indicating the distance between the distance sensor 15 and the center point of each of pixels IM(1,1) to IM(6,6).

[0025] Furthermore, the acquiring unit 122 acquires intensity information SI indicating the intensity of reflected light reflected from each of pixels IM(1,1) to IM(6,6) at multiple points in time. The acquiring unit 122 acquires intensity information SI of the nth frame at point in time t=tn, where n is a natural number. The intensity information SI of the nth frame is made up of intensity information SI(1,1,n) to SI(6,6,n) indicating the intensity of reflected light reflected from each of pixels IM(1,1) to IM(6,6).

[0026] For example, the intensity information SI may be information on intensity indicated by an output voltage output from a light-receiving sensor corresponding to each of pixels IM(1,1) to IM(6,6). Alternatively, if the distance sensor 15 is a ToF sensor and generates an IR image by capturing an infrared image of the projection surface PP, the intensity information SI may be information on intensity indicated by the intensity of a signal output from the light-receiving sensor based on the IR image. Alternatively, if the light-receiving sensor calculates a pseudo-reflectance indicating the ratio of the amount of reflected light received by the light-receiving sensor to the amount of projected light, the intensity information SI may be information on intensity indicated by the pseudo-reflectance.

[0027] At time t=tn, the acquisition unit 122 acquires depth information DI(1,1,n) and intensity information SI(1,1,n) for pixel IM(1,1) in association with each other. The same applies to other pixels IM. The same applies to time points other than t=tn. In FIG. 4, the acquisition unit 122 acquires, for example, depth information DI(1,6,1) and intensity information SI(1,6,1) in association with each other. The acquisition unit 122 also acquires, for example, depth information DI(4,2,1) and intensity information SI(4,2,1) in association with each other. The acquisition unit 122 also acquires, for example, depth information DI(5,4,1) and intensity information SI(5,4,1) in association with each other.

[0028] In FIG. 1, the parameter generation unit 123 generates parameters that define the projection plane PP based on the depth information DI and intensity information SI acquired by the acquisition unit 122.

[0029] More specifically, the parameter generation unit 123 may generate the above parameters by calculating a weighted average of the distances indicated by the depth information DI using the intensities indicated by the intensity information SI as weights. Alternatively, the parameter generation unit 123 may remove outliers from the distances indicated by the depth information DI based on a comparison result of the intensities indicated by the intensity information SI with a threshold, and generate the above parameters based on the distances indicated by the depth information DI from which the outliers have been removed. Alternatively, the parameter generation unit 123 may generate the above parameters by using the intensities indicated by the intensity information SI as weights and performing a weighted least squares method on coordinate values ​​calculated using the distances indicated by the depth information DI.

[0030] Here, the "parameters that define the projection surface PP" may be, for example, the normal vector of the projection surface PP, or the coefficients of a plane equation that defines the position of the projection surface PP in the XYZ coordinate system.

[0031] A: When calculating a weighted average The parameter generation unit 123 may generate parameters that define the projection surface PP based on a weighted average value of the distances indicated by the depth information DI at multiple points in time, with the weight being the intensity indicated by the intensity information SI corresponding to each piece of depth information DI.

[0032] 5 is a diagram showing depth information DI of pixel IM(1,6) in multiple frames. As an example, if the distance indicated by the depth information DI(1,6,i) of pixel IM(1,6) is distance di and the intensity indicated by the intensity information SI(1,6,i) corresponding to the depth information DI(1,6,i) is weight wi, the parameter generation unit 123 calculates distance dff, which is the weighted average value between frames of the distance from the distance sensor 15 to pixel IM(1,6), using the following equation 1. Here, i is an integer between 1 and 7.

number

[0033] In Fig. 5, the same applies to pixels IM(1,1) to IM(6,6) other than pixel IM(1,6). In addition, in the example shown in Fig. 5, seven frames from the first frame to the seventh frame are shown. However, even if the number of frames is different from that in Fig. 5, the same applies except for the difference in the numbers corresponding to the number of frames in Equation 1.

[0034] The parameter generating unit 123 may also generate parameters that define the projection surface PP based on a weighted average value of distances indicated by depth information DI at at least some of a plurality of positions that are adjacent to each other among a plurality of positions within the projection surface PP. Note that the weight here is the intensity indicated by intensity information SI corresponding to each piece of depth information DI.

[0035] 6 shows the depth information DI of pixel IM(1,6) and the pixels IM(1,5), IM(2,5), and IM(2,6) adjacent to pixel IM(1,6) in the first frame. The distance indicated by the depth information DI(1,6,1) of pixel IM(1,6) is defined as distance d1, and the intensity indicated by the intensity information SI(1,6,1) corresponding to the depth information DI(1,6,1) is defined as weight w1. The distance indicated by the depth information DI(1,5,1) of pixel IM(1,5) is defined as distance d2, and the intensity indicated by the intensity information SI(1,5,1) corresponding to the depth information DI(1,5,1) is defined as weight w2. The distance indicated by the depth information DI(2,5,1) of pixel IM(2,5) is defined as distance d3, and the intensity indicated by the intensity information SI(2,5,1) corresponding to the depth information DI(2,5,1) is defined as weight w3. The distance indicated by the depth information DI(2,6,1) of pixel IM(2,6) is defined as distance d4, and the intensity indicated by the intensity information SI(2,6,1) corresponding to the depth information DI(2,6,1) is defined as weight w4. In this case, the parameter generation unit 123 may calculate a distance dmf, which is a weighted average value of the distances of pixel IM(1,6) between adjacent pixels, using the following equation 2. Here, i is an integer between 1 and 4.

number

[0036] In FIG. 6, the same applies to the pixels IM(1,1) to IM(6,6) other than the pixel IM(1,6).

[0037] In addition, the parameter generation unit 123 may combine both the method of calculating the distance dff, which is the weighted average value between frames, described with reference to Figure 5, and the method of calculating the distance dmf, which is the weighted average value between adjacent pixels IM, described with reference to Figure 6. Specifically, the parameter generating unit 123 may calculate the distance dff, which is the weighted average value between frames corresponding to each of pixels IM(1,1) to IM(6,6), and then use the distance dff to calculate the distance dmf, which is the weighted average value between adjacent pixels IM. Alternatively, the parameter generating unit 123 may calculate a distance dmf, which is a weighted average value between adjacent pixels IM for each of pixels IM(1,1) to IM(6,6), and then use the distance dmf to calculate a distance dff, which is a weighted average value between frames corresponding to each of pixels IM(1,1) to IM(6,6).

[0038] B: When removing outliers The parameter generation unit 123 may extract intensity information SI that indicates an intensity exceeding the threshold by comparing the intensity indicated by the intensity information SI with a threshold, and generate parameters that define the projection surface PP based on the average value of the distance indicated by the depth information DI corresponding to the intensity information SI.

[0039] For example, if the intensities indicated by the intensity information SI(1,6,i) (i is an integer between 1 and 7) of pixel IM(1,6) in multiple frames, and the intensities indicated by the intensity information SI(1,6,2), intensity information SI(1,6,3), and intensity information SI(1,6,5) exceed a threshold, the parameter generation unit 123 determines the average value of the distance d2 indicated by the depth information DI(1,6,2), the distance d3 indicated by the depth information DI(1,6,3), and the distance d5 indicated by the depth information DI(1,6,5) in Figure 5 as the distance dfs from the projection device 11 to pixel IM(1,6).

[0040] In Fig. 5, the same applies to pixels IM(1,1) to IM(6,6) other than pixel IM(1,6). In addition, in the example shown in Fig. 5, seven frames from the first frame to the seventh frame are shown. However, the same applies when the number of frames is different from that shown in Fig. 5.

[0041] Alternatively, if the intensity indicated by the intensity information SI(1,6,1), intensity information SI(1,5,1), and intensity information SI(2,6,1) of the pixel IM(1,6) in the first frame and the pixels IM(1,5), IM(2,5), and IM(2,6) adjacent to the pixel IM(1,6) exceeds a threshold, the parameter generation unit 123 sets the average value of the distance d1 indicated by the depth information DI(1,6,1), the distance d2 indicated by the depth information DI(1,5,1), and the distance d4 indicated by the depth information DI(2,6,1) in FIG. 6 as the distance dms from the projection device 11 to the pixel IM(1,6).

[0042] In addition, the parameter generation unit 123 may combine both the method of calculating the distance dfs, which is the average value between frames, as described with reference to Figure 5, and the method of calculating the distance dms, which is the average value between adjacent pixels IM, as described with reference to Figure 6. Specifically, the parameter generation unit 123 may calculate a distance dfs, which is the average value between frames corresponding to each of pixels IM(1,1) to IM(6,6), and then use the distance dfs to calculate a distance dms, which is the average value between adjacent pixels IM. Alternatively, the parameter generation unit 123 may calculate the distance dms, which is the average value between adjacent pixels IM for each of pixels IM(1,1) to IM(6,6), and then use the distance dms to calculate the distance dfs, which is the average value between frames corresponding to each of pixels IM(1,1) to IM(6,6).

[0043] C: When using weighted least squares The parameter generation unit 123 calculates the coordinate values ​​of the center points in the XYZ coordinate system based on the distance from the projection device 11 to the center points of pixels IM(1,1) to IM(6,6), which are indicated by each piece of depth information DI at multiple points in time. The coordinate values ​​of the center points of pixels IM(1,1) to IM(6,6) are calculated based on depth information DI at multiple points in time, and therefore exist for the number of frames. The parameter generation unit 123 may fit a plane to a point cloud of the center points of pixels IM(1,1) to IM(6,6), each of which exists for the number of frames. When fitting the plane, the parameter generation unit 123 uses a weighted least squares method, using each piece of intensity information SI at multiple points in time as a weight.

[0044] For example, as shown in FIG. 2, when the plane equation of the projection surface PP is expressed as pX+qY+rZ=1 and the coordinates of the point group of the center point are expressed as (X, Y, Z)=(Xk, Yk, Zk), where k is an integer from 1 to u, the parameter generation unit 123 generates p, q, and r that minimize mn shown in the following equation 3.

number

[0045] The parameter generating unit 123 uses each piece of intensity information SI corresponding to the depth information DI at multiple points in time as σ in the above equation 3. Each piece of intensity information SI corresponds to each of the coordinates (X, Y, Z) = (Xk, Yk, Zk) of the point cloud of the center point.

[0046] In FIG. 1, the correction unit 124 corrects the projection image PI generated by the projection image generation unit 121 using the parameters generated by the parameter generation unit 123. As an example, the correction unit 124 performs trapezoidal correction on the projection image PI using the above parameters so that the projection image PI on the projection surface PP becomes rectangular.

[0047] The projection control unit 125 causes the projection device 11 to project the projection image PI corrected by the correction unit 124 onto the projection surface PP.

[0048] 1-2: Operation of the first embodiment FIG. 7 is a flowchart showing the operation of the projector 1 according to this embodiment.

[0049] In step S1, the processing device 12 functions as a projection image generation unit 121. The processing device 12 generates a projection image PI based on an input image.

[0050] In step S2, the processing device 12 functions as the acquisition unit 122. Based on the output of the distance sensor 15, the processing device 12 acquires depth information DI indicating a plurality of distances from the distance sensor 15 to a plurality of positions on the projection surface PP.

[0051] In step S3, the processing device 12 functions as the acquisition unit 122. Based on the output of the distance sensor 15, the processing device 12 acquires intensity information SI indicating the intensity of the reflected light at each of a plurality of positions on the projection surface PP.

[0052] In step S4, the processing device 12 functions as the parameter generation unit 123. The processing device 12 generates parameters that define the projection plane PP based on the depth information DI acquired in step S2 and the intensity information SI acquired in step S3.

[0053] In step S5, the processing device 12 functions as the correction unit 124. The processing device 12 corrects the projection image PI based on the parameters generated in step S4.

[0054] In step S6, the processing device 12 functions as the projection control unit 125. The processing device 12 causes the projection device 11 to project the projection image PI corrected in step S5 onto the projection surface PP.

[0055] In addition, in step S4 of the flowchart in Figure 7, if the processing device 12 is unable to acquire sufficient depth information DI and intensity information SI to generate parameters, it is preferable that the processing device 12 repeats at least one of steps S2 and S3 until it can acquire sufficient depth information DI and intensity information SI.

[0056] 2: Variation The above embodiments can be modified in various ways. Specific modified embodiments are exemplified below. The embodiments exemplified below and the embodiments described above can be combined as appropriate within the scope of not mutually contradicting each other. Note that for elements in the modified embodiments exemplified below that have the same actions and functions as the embodiments, the reference numerals referenced in the above explanation will be used and detailed explanations of each element will be omitted as appropriate.

[0057] 2-1: Variation 1 In the above embodiment, for example, if the intensity information SI is information on intensity indicated by the intensity of a signal output from a light-receiving sensor, if the signal intensity is too strong, the signal may saturate, resulting in a decrease in the accuracy of the distance indicated by the depth information DI. Therefore, if the signal intensity is too strong, it is preferable that the parameter generation unit 123 reduce the weight applied to the distance indicated by the depth information DI corresponding to the light-receiving sensor. Alternatively, it is preferable that the parameter generation unit 123 increase the threshold value used for comparing with the signal intensity, thereby widening the range of distances determined to be outliers.

[0058] 2-2: Variation 2 In the above embodiment, the parameter generation unit 123 generates parameters by calculating a weighted average of the distances indicated by the depth information DI using the intensities indicated by the intensity information SI as weights. Alternatively, the parameter generation unit 123 removes outliers from the distances indicated by the depth information DI based on a comparison result of the intensities indicated by the intensity information SI with a threshold, and generates parameters based on the distances indicated by the depth information DI from which the outliers have been removed. Alternatively, the parameter generation unit 123 uses the intensities indicated by the intensity information SI as weights and generates parameters by a weighted least squares method targeting coordinate values ​​calculated using the distances indicated by the depth information DI.

[0059] However, the parameter generation unit 123 is not limited to these, and may generate parameters by using any method that uses the intensity indicated by the intensity information SI in addition to the distance indicated by the depth information DI.

[0060] 2-3: Variation 3 Depending on the material of the projection surface PP, the parameter generation unit 123 may not be able to calculate parameters with sufficient accuracy and correct the projection image PI. In this case, a highly reflective material may be attached to the surface of the projection surface PP, and then the projector 1 may execute the correction method described in the above embodiment.

[0061] 3: Summary of this disclosure A summary of this disclosure is provided below.

[0062] (Appendix 1) A method for correcting a projection image projected from a projector onto a projection surface, comprising: irradiating the projection surface with irradiation light and acquiring depth information indicating multiple distances from the distance sensor to multiple positions on the projection surface based on the output of a distance sensor that receives reflected light reflected from the projection surface; acquiring intensity information indicating the intensity of the reflected light at each of the multiple positions; generating parameters that define the projection surface based on the depth information and the intensity information; and correcting the projection image based on the parameters.

[0063] The correction method of this embodiment has the above configuration, thereby improving the accuracy of correcting the projection image PI. Specifically, since the correction method calculates parameters using intensity information SI in addition to depth information DI, the accuracy of calculating the parameters is improved compared to when parameters are calculated using only depth information DI. This also improves the accuracy of correcting the projection image PI.

[0064] In particular, when a ToF sensor is used as the distance sensor 15, the measurement accuracy of the distance to the projection surface PP decreases when the distance between the distance sensor 15 and the projection surface PP is long or when the angle between the distance sensor 15 and the projection surface PP is large. Accuracy may also decrease depending on the material of the object that makes up the projection surface PP. Under conditions that decrease accuracy, the measurement variability of the ToF sensor increases, but the average or median of the measurement variability does not necessarily become the true value. Therefore, it is difficult to accurately estimate the distance to the projection surface PP using only the depth information DI acquired from the ToF sensor.

[0065] When the accuracy of measuring the distance to the projection surface PP by the ToF sensor decreases, it is because the ToF sensor is not receiving adequate reflected light due to the reasons described above. Therefore, when measurement accuracy decreases, the signal strength of the light measured by the ToF sensor may decrease. Therefore, the correction method of this embodiment uses depth information DI and intensity information SI in combination to improve measurement accuracy, and is expected to measure the 3D shape of the projection surface PP with high accuracy.

[0066] By extension, in keystone correction in a projector, it is necessary to acquire planar information of the projection surface PP with high accuracy. By measuring the projection surface PP using the correction method of this embodiment, the projector 1 can perform keystone correction with high accuracy.

[0067] (Appendix 2) The correction method described in Appendix 1, characterized in that generating parameters defining the projection surface based on the depth information and the intensity information includes comparing the intensity indicated by the intensity information with a threshold, and generating parameters defining the projection surface based on the depth information corresponding to the intensity information indicating the intensity exceeding the threshold.

[0068] The correction method of this embodiment has the above configuration, which improves the accuracy of the depth information DI, and as a result, improves the accuracy of correcting the projection image PI.

[0069] In conventional technology, only the depth information DI output from the ToF sensor is used, and errors are reduced by averaging, etc., but the depth information DI alone does not determine what distances are outliers, and errors cannot be completely removed. The correction method of this embodiment uses intensity information SI in addition to depth information DI obtained from the ToF sensor, making it possible to perform 3D measurement of the projection surface PP with high accuracy.

[0070] (Appendix 3) The correction method of Appendix 2, wherein the depth information and the intensity information are acquired at multiple points in time, and generating parameters defining the projection surface comprises generating parameters defining the projection surface based on the depth information at one or more points in time corresponding to the intensity information indicating the intensity exceeding the threshold.

[0071] The correction method of this embodiment has the above configuration, which further improves the accuracy of the depth information DI, and as a result, also improves the accuracy of correcting the projection image PI.

[0072] (Appendix 4) The correction method of Appendix 1, characterized in that, at each of the plurality of positions, parameters defining the projection surface are generated based on the depth information and the intensity information corresponding to each other, and using a weighted least squares method in which the intensity indicated by the intensity information is used as a weight.

[0073] The correction method of this embodiment has the above configuration, which improves the accuracy of the depth information DI, and as a result, improves the accuracy of correcting the projection image PI.

[0074] (Appendix 5) The correction method of Appendix 1, characterized in that the depth information and the intensity information are acquired at multiple points in time, and parameters defining the projection surface are generated based on a weighted average value of the distances indicated by the depth information at the multiple points in time, with the intensity indicated by the intensity information corresponding to each piece of depth information being used as the weight.

[0075] The correction method of this embodiment has the above configuration, which further improves the accuracy of the depth information DI, and as a result, also improves the accuracy of correcting the projection image PI.

[0076] (Appendix 6) A correction method according to Appendix 1, characterized in that parameters defining the projection surface are generated based on a weighted average value of the distances indicated by the depth information at at least some of the positions adjacent to each other, with the weight being the intensity indicated by the intensity information corresponding to each of the depth information.

[0077] The correction method of this embodiment has the above configuration, which further improves the accuracy of the depth information DI, and as a result, also improves the accuracy of correcting the projection image PI.

[0078] (Appendix 7) A projector characterized by comprising one or more processors that perform the following: irradiating a projection surface with irradiation light and acquiring depth information indicating multiple distances from the distance sensor to multiple positions on the projection surface based on the output of a distance sensor that receives reflected light reflected from the projection surface; acquiring intensity information indicating the intensity of the reflected light at each of the multiple positions; generating parameters that define the projection surface based on the depth information and the intensity information; and correcting the projected image based on the parameters.

[0079] The projector 1 of this embodiment has the above configuration, which improves the accuracy of correcting the projected image PI. Specifically, the projector 1 calculates parameters using intensity information SI in addition to depth information DI, so the accuracy of calculating parameters is improved compared to when parameters are calculated using only depth information DI. This also improves the accuracy of correcting the projected image PI.

[0080] In particular, when a ToF sensor is used as the distance sensor 15, the measurement accuracy of the distance to the projection surface PP decreases when the distance between the distance sensor 15 and the projection surface PP is long or when the angle between the distance sensor 15 and the projection surface PP is large. Accuracy may also decrease depending on the material of the object that makes up the projection surface PP. Under conditions that decrease accuracy, the measurement variability of the ToF sensor increases, but the average or median of the measurement variability does not necessarily become the true value. Therefore, it is difficult to accurately estimate the distance to the projection surface PP using only the depth information DI acquired from the ToF sensor.

[0081] If the accuracy of measuring the distance to the projection surface PP by the ToF sensor decreases, it is because the ToF sensor is not receiving adequate reflected light due to the reasons described above. Therefore, if the measurement accuracy decreases, the signal strength of the light measured by the ToF sensor may decrease. Therefore, the projector 1 of this embodiment is expected to improve measurement accuracy by using both the depth information DI and the intensity information SI, and to measure the 3D shape of the projection surface PP with high accuracy.

[0082] By extension, in keystone correction in a projector, it is necessary to acquire planar information of the projection surface PP with high accuracy. By measuring the projection surface PP in the projector 1 of this embodiment, the projector 1 can perform keystone correction with high accuracy. [Explanation of symbols]

[0083] 1: Projector, 11: Projection device, 12: Processing device, 13: Storage device, 14: Communication device, 15: Distance sensor, 121: Projection image generation unit, 122: Acquisition unit, 123: Parameter generation unit, 124: Correction unit, 125: Projection control unit, DI: Depth information, IM: Pixel, IMs: Pixel group, LP: Light point, PI: Projected image, SC: Screen, PP: Projection surface, PR1: Control program, SI: Intensity information, d1, d2, d3, d4, d5, dff, dfs, di, dmf, dms: Distance, w1, w2, w3, w4, wi: Weight

Claims

1. A method for correcting a projection image projected onto a projection surface from a projector, comprising: irradiating the projection surface with irradiation light, and acquiring depth information indicating a plurality of distances from the distance sensor to a plurality of positions on the projection surface based on an output of a distance sensor that receives light reflected by the projection surface; acquiring intensity information indicating the intensity of the reflected light at each of the plurality of positions; generating parameters defining the projection plane based on the depth information and the intensity information; correcting the projected image based on the parameters; A correction method comprising:

2. generating parameters defining the projection plane based on the depth information and the intensity information, comparing the intensity indicated by the intensity information with a threshold; generating parameters defining the projection plane based on the depth information corresponding to the intensity information indicating the intensity exceeding the threshold; The correction method according to claim 1 , comprising:

3. the depth information and the intensity information are acquired at multiple time points; Generating parameters defining the projection plane includes: parameters defining the projection plane are generated based on the depth information at one or more points in time corresponding to the intensity information indicating the intensity exceeding the threshold. The correction method according to claim 2 .

4. At each of the plurality of positions, parameters defining the projection plane are generated based on the depth information and the intensity information corresponding to each other and using a weighted least squares method in which the intensity indicated by the intensity information is used as a weight. The correction method according to claim 1 .

5. the depth information and the intensity information are acquired at multiple time points; a parameter defining the projection plane is generated based on a weighted average value of the distances indicated by the depth information at the plurality of times, the weight being the intensity indicated by the intensity information corresponding to each of the depth information; The correction method according to claim 1 .

6. a parameter defining the projection plane is generated based on a weighted average value of distances indicated by the depth information at at least some of the positions adjacent to each other among the plurality of positions, the weight being the intensity indicated by the intensity information corresponding to each of the depth information; The correction method according to claim 1 .

7. irradiating a projection surface with irradiation light, and acquiring depth information indicating a plurality of distances from the distance sensor to a plurality of positions on the projection surface based on an output of a distance sensor that receives light reflected by the projection surface; acquiring intensity information indicating the intensity of the reflected light at each of the plurality of positions; generating parameters defining the projection plane based on the depth information and the intensity information; correcting the projected image based on the parameters; A projector comprising one or more processors that execute the above.

Images