Correction method, information processing apparatus, and program

Abstract

To alleviate a user's discomfort caused by correction of an image.SOLUTION: A correction method includes: on the basis of a detection result at a first time of a sensor for estimating the relative positional relationship between a projector that projects an image on a projection surface and the projection surface, acquiring a first calculated value for correcting the shape of the image at the first time; on the basis of a detection result at a second time different from the first time of the sensor, acquiring a second calculated value for correcting the shape of the image at the second time; on the basis of either one of the average value, the median value, or the mode of the plurality of calculated values including the first calculated value and the second calculated value, determining a correction value for geometric correction; and correcting the image by using the correction value.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a correction method, an information processing device, and a program. [Background technology]

[0002] For example, Patent Document 1 describes a projector that automatically performs trapezoidal distortion correction on an image displayed on a screen based on the vertical tilt angle detected by a gravity acceleration sensor. The projector described in Patent Document 1 calculates a representative angle at a sampling time from multiple tilt angles detected by the gravity acceleration sensor during the sampling time, calculates the difference angle between the representative angle and the representative angle at the immediately preceding sampling time, and performs trapezoidal distortion correction on the image only if the difference angle is equal to or greater than a limit angle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-98836 Summary of the Invention [Problem to be solved by the invention]

[0004] In the projector described in Patent Document 1, keystone distortion correction is not performed when the difference angle is less than the limit angle. If the limit angle is not set, it is possible to perform keystone distortion correction each time a difference angle occurs. However, doing so can cause discontinuous changes in the results of keystone distortion correction due to errors in the detection results of the gravity acceleration sensor, which can cause discomfort to the user. [Means for solving the problem]

[0005] A correction method according to one embodiment of the present disclosure includes: acquiring a first calculated value for correcting the shape of the image at a first time based on a detection result at a first time of a sensor for estimating a relative positional relationship between a projector that projects an image onto a projection surface and the projection surface; acquiring a second calculated value for correcting the shape of the image at a second time based on a detection result at a second time different from the first time of the sensor; determining a correction value for geometric correction based on one of the average, median, or mode of multiple calculated values ​​including the first calculated value and the second calculated value; and correcting the image using the correction value.

[0006] An information processing device according to one embodiment of the present disclosure includes at least one processor that performs the following operations: acquiring a first calculated value for correcting the shape of the image at a first time based on a detection result at a first time of a sensor for estimating a relative positional relationship between a projector that projects an image onto a projection surface and the projection surface; acquiring a second calculated value for correcting the shape of the image at a second time based on a detection result at a second time different from the first time of the sensor; determining a correction value for geometric correction based on one of the average, median, or mode of multiple calculated values ​​including the first calculated value and the second calculated value; and correcting the image using the correction value.

[0007] A program according to one embodiment of the present disclosure causes a computer to perform the following operations: based on a detection result at a first time of a sensor for estimating a relative positional relationship between a projector that projects an image onto a projection surface and the projection surface, obtain a first calculated value for correcting the shape of the image at the first time; based on a detection result at the first time of the sensor at a second time different from the first time, obtain a second calculated value for correcting the shape of the image at the second time; based on a detection result at the sensor at the second time different from the first time, determine a correction value for geometric correction based on one of the average, median, or mode of multiple calculated values ​​including the first calculated value and the second calculated value; and correct the image using the correction value. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an outline of a system used in a correction method according to a first embodiment. [Figure 2] FIG. 1 is a block diagram of a projector according to a first embodiment. [Figure 3] 4 is a flowchart showing the flow of a correction method according to the first embodiment. [Figure 4] FIG. 10 is an explanatory diagram of correction values. [Figure 5] FIG. 10 is an explanatory diagram of calculated values ​​and corrected values. [Figure 6] FIG. 10 is a block diagram of a projector according to a second embodiment. [Figure 7] 10 is a flowchart showing the flow of a correction method according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Preferred embodiments of the present disclosure will be described below with reference to the accompanying drawings. Note that the dimensions and scale of each part in the drawings may differ from the actual dimensions and are shown schematically to facilitate understanding. Furthermore, the scope of the present disclosure is not limited to these embodiments unless otherwise specified in the following description to the effect that the present disclosure is limited. 1. First embodiment 1-1. Overview of the system used for the correction method 1 is a diagram showing an outline of a system 100 used in a correction method according to the first embodiment. As shown in FIG.

[0010] Projector 10 is a display device that projects an image G represented by image information output from a device such as a computer (not shown) onto a projection surface SC. Projection surface SC is the surface of an object such as a screen, and is generally flat. While projection surface SC does not need to be strictly flat, it is preferable that it be a surface that can be considered flat in order to simplify the process of geometrically correcting image G.

[0011] Here, the installation orientation of the projection surface SC may differ depending on the manner of use of the system 100. Therefore, the projector 10 corrects distortion of the image G due to the installation orientation of the projection surface SC by geometric correction such as keystone correction. In geometric correction, for example, if the shape of the image G to be projected is rectangular, the shape of the image G that is actually projected is corrected to be rectangular.

[0012] As will be described in more detail later, the projector 10 is equipped with a sensor 17, and has the function of using the sensor 17 to measure the relative positional relationship between the projector 10 and the projection surface SC, and the function of determining a correction value for geometric correction of the image G based on the results of that measurement.

[0013] The relative positional relationship between the projector 10 and the projection surface SC includes not only the relative position of the projector 10 with respect to the projection surface SC, but also the relative attitude of the projector 10 with respect to the projection surface SC. The relative position of the projector 10 with respect to the projection surface SC changes depending on the installation positions of one or both of the projection surface SC and the projector 10. The relative attitude of the projector 10 with respect to the projection surface SC changes depending on the installation attitude of one or both of the projection surface SC and the projector 10. The installation attitude of the projection surface SC and the projector 10 includes rotations around three orthogonal axes corresponding to roll, pitch, and yaw. The installation position and installation attitude of the projector 10 change depending on, for example, conditions such as the position and inclination of the installation surface on which the projector 10 is installed, adjustments made by an adjustment mechanism provided in the projector 10, and adjustments made by an adjustment mechanism provided on the stand on which the projector 10 is installed. The installation position and installation attitude of the projection surface SC change depending on, for example, the position and inclination of the projection surface SC.

[0014] 1-2.Projector Fig. 2 is a block diagram of a projector 10 according to the first embodiment. As shown in Fig. 2, the projector 10 includes a storage device 11, a processing device 12, a communication device 13, an image processing circuit 14, an optical device 15, an operation device 16, and a sensor 17. These are connected to each other so as to be able to communicate with each other.

[0015] The storage device 11 is a storage device that stores programs executed by the processing device 12 and data processed by the processing device 12. The storage device 11 includes, for example, a hard disk drive or a semiconductor memory. Note that part or all of the storage device 11 may be provided in an external storage device or server outside the projector 10.

[0016] A program PR1, temporary variable information PA-1 to PA-m, temporary correction value information D1-1 to D1-m, and correction value information DC are stored in the storage device 11. m is a natural number equal to or greater than 2 and corresponds to the number of times steps S20 to S40 described below are executed.

[0017] The program PR1 is a program for executing a correction method, which will be described in detail later.

[0018] The temporary variable information PA-1 to PA-m ​​is information indicating variables of an arithmetic expression used for geometric correction of image G at each time from the first time to the mth time, and indicates the degree of geometric correction of image G. The temporary variable information PA-1 to PA-m ​​is calculated based on the detection results of sensor 17 at each time from the first time to the mth time. The detection results of sensor 17 themselves are not included in the temporary variable information PA-1 to PA-m. The variables relate to at least one of the installation angle of projection surface SC, the normal vector of projection surface SC, and the attitude of projector 10. Note that, hereinafter, the temporary variable information PA-1 to PA-m ​​may be referred to as temporary variable information PA without distinction.

[0019] The provisional correction value information D1-1 to D1-m is information indicating correction values ​​for geometric correction of image G at each time from the first time to the m-th time. The correction values ​​are, for example, coordinate values ​​of four corners of image G. The coordinate values ​​are, for example, coordinate values ​​of a display coordinate system set in the light modulator 15b described below or a coordinate system associated therewith.

[0020] In this embodiment, each of the multiple correction values ​​indicated by the provisional correction value information D1-1 to D1-m is an example of a "calculated value." Any of the multiple correction values ​​indicated by the provisional correction value information D1-1 to D1-m is an example of a "first calculated value," and any other correction value is an example of a "second calculated value." Note that, hereinafter, the provisional correction value information D1-1 to D1-m may be referred to as provisional correction value information D1 without distinction.

[0021] The correction value information DC is information indicating the correction value of the geometric correction actually used for the image G. In this embodiment, the correction value information DC is the average value, median, or mode of the correction values ​​indicated by two or more pieces of provisional correction value information D1. Therefore, the correction values ​​indicated by the correction value information DC are, for example, the coordinate values ​​of the four corners of the image G. The coordinate values ​​are, for example, the coordinate values ​​of a display coordinate system set in the optical modulator 15b described below or a coordinate system associated therewith.

[0022] The processing device 12 has the function of controlling each unit of the projector 10 and the function of processing various types of data. The processing device 12 includes, for example, a processor such as a CPU (Central Processing Unit). The processing device 12 may be configured with a single processor or multiple processors. Some or all of the functions of the processing device 12 may be realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The processing device 12 may also be integrated with the image processing circuit 14.

[0023] The communication device 13 is a communication device capable of communicating with various devices and acquires image data IMG from devices not shown. For example, the communication device 13 is a wired communication device such as a wired LAN (Local Area Network), USB (Universal Serial Bus), or HDMI (High Definition Multimedia Interface), or a wireless communication device such as LPWA (Low Power Wide Area), wireless LAN including Wi-Fi, or Bluetooth. "HDMI," "Wi-Fi," and "Bluetooth" are all registered trademarks.

[0024] The image processing circuit 14 performs necessary processing on the image data IMG from the communication device 13 and inputs the image data to the optical device 15. The image processing circuit 14 may include, for example, one or more processors such as a CPU, or may include hardware such as a DSP, ASIC, PLD, or FPGA. The image processing circuit 14 may have, for example, a frame memory (not shown), and may load the image data IMG into the frame memory, appropriately perform various processes such as resolution conversion, resizing, and geometric correction, and input the image data to the optical device 15. Here, correction value information DC is used for the geometric correction. Note that the image processing circuit 14 may also perform processes such as OSD (On Screen Display) processing, which generates image information for menu display or operation guides, etc., and combines the image data IMG with the image data IMG, as needed.

[0025] The optical device 15 is a device that displays an image by projecting image light onto a projection region RP. The optical device 15 includes a light source 15a, a light modulator 15b, and a projection optical system 15c.

[0026] The light source 15a includes a light source such as a halogen lamp, a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode), or a laser light source, and emits red, green, and blue light, respectively. The light modulator 15b includes three light modulation elements corresponding to red, green, and blue. Each light modulation element includes, for example, a transmissive liquid crystal panel, a reflective liquid crystal panel, or a DMD (Digital Micromirror Device), and generates image light of each color by modulating light of the corresponding color. The image light of each color generated by the light modulator 15b is combined by a color combining optical system to become full-color image light. The projection optical system 15c is an optical system including a projection lens and the like that forms and projects the full-color image light from the light modulator 15b onto a projection surface SC.

[0027] The operation device 16 is a device that accepts operations from the user. For example, the operation device 16 includes an operation panel and a remote control receiver, not shown. The operation panel is provided on the exterior housing of the projector 10, and outputs a signal based on an operation from the user. The remote control receiver receives an infrared signal from a remote control, not shown, decodes the infrared signal, and outputs a signal based on the operation of the remote control. The operation device 16 may be provided as needed, or may be omitted.

[0028] Sensor 17 is a sensor for estimating the relative positional relationship between projector 10 and projection surface SC. Sensor 17 includes distance sensor 17a and acceleration sensor 17b. Distance sensor 17a is a time-of-flight (ToF) distance sensor that measures the shape of projection surface SC. Acceleration sensor 17b is a sensor that detects acceleration on three mutually orthogonal axes, and detects the acceleration applied to projector 10.

[0029] 2 as long as it can obtain the detection results necessary to estimate the relative positional relationship between projector 10 and projection surface SC, and may be, for example, an embodiment in which one of distance sensor 17a and acceleration sensor 17b is omitted, or an embodiment in which an inertial sensor such as an angular velocity sensor and a camera are provided instead of one or both of distance sensor 17a and acceleration sensor 17b. However, the detection results necessary to estimate the relative positional relationship between projector 10 and projection surface SC differ depending on the calculation method of geometric correction, etc., and are not particularly limited.

[0030] In the above-described projector 10, the processing device 12 executes the program PR1 stored in the storage device 11, thereby performing various processes required for the correction method described below.

[0031] 1-3. Correction method 3 is a flowchart showing the flow of the correction method according to the first embodiment. The correction method according to the present embodiment is executed by the projector 10 described above.

[0032] The correction method of this embodiment includes steps S10 to S50, as shown in Fig. 3. Steps S20, S30, and S40 are repeatedly executed in this order.

[0033] Here, the program PR1 causes the processing device 12 to execute steps S10 to S50. The processing device 12 and the image processing circuit 14 are examples of an "information processing device" and a "computer," and include at least one processor that executes steps S10 to S50.

[0034] First, in step S10, the processing device 12 starts measurement by the sensor 17. Here, the processing device 12 acquires point cloud data indicating multiple coordinate values ​​on the projection surface SC based on the detection results of the distance sensor 17a, and calculates the direction of the gravitational acceleration acting on the projector 10 based on the detection results of the acceleration sensor 17b.

[0035] After step S10, in step S20, the processing device 12 acquires a calculated value for correcting the shape of the image G at the current time based on the detection result at the current time of the sensor 17. A specific example of the processing of step S20 will be described later with reference to FIG.

[0036] Here, of any two steps S20 in the multiple steps S20, the current time of the earlier step S20 is an example of the “first time,” and the current time of the later step S20 is an example of the “second time.” Furthermore, of the calculated values ​​in the multiple steps S20, the calculated value based on the detection result of sensor 17 at the first time is an example of the “first calculated value,” and the calculated value based on the detection result of sensor 17 at the second time is an example of the “second calculated value.”

[0037] Step S20 includes steps S21 and S22. In step S21, the processing device 12 calculates variables at the current time based on the detection results of the sensor 17 at the current time. Information indicating the calculated variables is stored in the storage device 11 as temporary variable information PA. After step S21, in step S22, the processing device 12 uses the variables at the current time to calculate correction values ​​at the current time as calculation values ​​for correcting the shape of the image G at the current time. Information indicating the calculated correction values ​​is stored in the storage device 11 as temporary correction value information D1.

[0038] After step S20, in step S30, the processing device 12 determines a correction value based on the correction values ​​indicated by the most recent n provisional correction value information D1, where n is a natural number between 2 and m. The correction values ​​indicated by the most recent n provisional correction value information D1 include a first calculated value and a second calculated value.

[0039] Step S30 includes step S31. In step S31, the processing device 12 statistically processes the correction values ​​indicated by the most recent n provisional correction value information D1. As a result, the result of the statistical processing is determined as the final correction value. The statistical processing in step S31 is a process of determining the average value, mode, or median of the most recent n correction values. Information indicating the determined correction value is stored in the storage device 11 as correction value information DC.

[0040] After step S30, in step S40, the processing device 12 and the image processing circuit 14 correct the image G using the correction value indicated by the correction value information DC.

[0041] After step S40, in step S50, the processing device 12 determines whether to terminate. This determination may be made based on, for example, a user's operation on the projector 10, or, if m is a specified value, based on whether step S50 has been executed m times.

[0042] If the process is not to be completed (step S50: NO), the processing device 12 returns to step S20, whereby steps S20, S30, and S40 are repeatedly executed in this order.

[0043] On the other hand, if the process is to be ended (step S50: YES), the processor 12 ends the process.

[0044] In the above correction method, in step S30, the correction value for geometric correction is determined based on the average, median, or mode of multiple calculation values ​​based on the detection results of the sensor 17, so that the change in shape of the corrected image G can be made smoother than in a mode in which the correction value based on the calculation value at the current time is used as is. Therefore, the sense of discomfort felt by the user due to the correction of the image G can be alleviated.

[0045] FIG. 4 is an explanatory diagram of the correction values. In this embodiment, as shown in FIG. 4, image G is rectangular, and in step S20, coordinate values ​​P1(x, y), P2(x, y), P3(x, y), and P4(x, y) of the four corners of image G are calculated as correction values. These coordinate values ​​are coordinate values ​​of a display coordinate system set in the optical device 15 or a coordinate system associated therewith. Therefore, these coordinate values ​​are associated with pixels of the optical device 15. An example of a method for calculating the correction values ​​will be briefly described below. Note that the method for calculating the correction values ​​is merely an example and is not limited thereto, and various known calculation methods may be used, for example.

[0046] In step S21 of step S20, the processing device 12 acquires a depth map of the projection surface SC based on the detection results of the sensor 17, determines a plane equation of the projection surface SC from the depth map, and then calculates the normal vector of the projection surface SC as a variable based on the plane equation.

[0047] The normal vector of the projection surface SC is a variable that represents the relative positional relationship between the projector 10 and the projection surface SC. It can also be said that the plane equation of the projection surface SC and the normal vector of the projection surface SC represent the installation angle of the projection surface SC. Furthermore, the direction of the gravitational acceleration applied to the projector 10 represents the attitude of the projector 10. Therefore, the direction of the gravitational acceleration applied to the projector 10 may be used as a variable that represents the relative positional relationship between the projector 10 and the projection surface SC.

[0048] Then, in step S22, the processing device 12 calculates a rotation matrix representing the rotation of the projection surface SC as seen in the projection direction from the projector 10 based on the normal vector of the projection surface SC and the direction of the gravitational acceleration acting on the projector 10, and then uses the rotation matrix to calculate the coordinate values ​​P1(x,y), P2(x,y), P3(x,y), and P4(x,y) of the four corners of the image G as correction values.

[0049] By using the calculation method described above, the detection results of the sensor 17 can be used to automatically perform geometric correction on the image G projected onto the projection surface SC in real time.

[0050] In this embodiment, the first calculated values ​​are the coordinate values ​​of the four vertices of image G after correction at a first time. The second calculated values ​​are the coordinate values ​​of the four vertices of image G after correction at a second time. In this manner, in an aspect in which the coordinate values ​​of the vertices after correction are used as calculated values, the processing load for determining the correction values ​​can be reduced.

[0051] However, if the correction value for the current time obtained by this calculation method is used as is for geometric correction, the following problems A, B, and C arise.

[0052] Problem A: The frame rate of image G projected by projector 10 is generally between 60 Hz and 240 Hz. On the other hand, the sampling rate of measurement by sensor 17 is generally between 30 Hz and 60 Hz. In particular, distance sensor 17a has a large number of pixel structures, making it difficult to measure at a sampling rate equivalent to the frame rate of image G. For this reason, if the correction value calculated at the timing when the detection result of sensor 17 is sampled is used directly for geometric correction, image G will appear jerky because the correction frequency of geometric correction is lower than the frame rate of image G.

[0053] Problem B: When the attitude of the projector 10 is changed manually, acceleration is applied to the projector 10 due to the person touching the projector 10. When calculating the direction of gravity from the three-axis acceleration of the acceleration sensor 17b, it is difficult to distinguish between acceleration due to the human hand and acceleration due to gravity in the output of the acceleration sensor 17b as is. As a result, the direction of gravity may be calculated incorrectly. Therefore, if the correction value obtained by the calculation method described above is used directly for geometric correction, the quality of the image G will be reduced due to the incorrect correction.

[0054] Problem C: The detection results of the sensor 17 may be affected by noise, etc. For example, the distance accuracy of the distance sensor 17a may vary due to the effects of optical shot noise, etc. This may result in errors in the correction values ​​obtained by the calculation method described above. Therefore, if the correction values ​​obtained by the calculation method described above are used directly for geometric correction, a frame resulting from an incorrect correction may be inserted into the image G, or correction of the image G may be performed even though the positional relationship between the projection surface SC and the projector 10 has not changed.

[0055] To solve at least one of the above problems A, B, and C, step S40 is executed after step S30 is executed. As a result, in step S40, the correction value obtained in step S20 is not used as is, but rather a value obtained by statistically processing the most recent n correction values ​​is used.

[0056] 5 is an explanatory diagram of calculated values ​​and correction values, which shows each calculated value within a predetermined period and a correction value that is the average value of the most recent n calculation values ​​at each timing with a predetermined frequency within the predetermined period.

[0057] 5 can be said to be the result of performing moving average processing on the calculated values ​​at a predetermined frequency. This moving average processing is not particularly limited and may be, for example, a simple moving average processing, a weighted moving average processing, or an exponential moving average processing, but exponential moving average processing is preferable.

[0058] The calculation formula for exponential moving average processing is expressed as, for example, S(t) = α × Y(t) + (1 - α) × S(t-1). Here, Y(t) is the calculated value at a certain time t, S(t) is the exponential moving average value at a certain time t, and α is a constant. t is a natural number. When using such exponential moving average processing, it is only necessary to store the previous exponential moving average value, which has the advantage of not occupying too much storage space in the storage device 11. In addition, the constant α can be changed as a weight between past and present values.

[0059] 5, the processing device 12 and the image processing circuit 14 correct the image G at a frequency higher than the sampling rate of the sensor 17. This allows for smoother changes in the shape of the corrected image G than in a mode in which the image G is corrected at a frequency equal to or lower than the sampling rate of the sensor 17.

[0060] The frequency at which image G is corrected in step S40 is preferably a first frequency that is equal to or lower than the frame rate of image G. This reduces the processing load for correcting image G, while smoothing out changes in the shape of image G after correction. Note that, in order to correct image G in step S40 at the first frequency, for example, steps S20 to S40 may be repeated at the first frequency.

[0061] Furthermore, since the sensor 17 includes the time-of-flight distance sensor 17a as described above, the effect of the present disclosure is significant. Specifically, the effect of being able to solve the problem A described above is significantly achieved.

[0062] As described above, the correction method of this embodiment can smooth the change in shape of the corrected image G compared to a mode in which the correction value based on the calculated value at the current time is used as is. Therefore, it is possible to alleviate the sense of discomfort felt by the user by the correction of the image G.

[0063] 2. Second embodiment A second embodiment of the present disclosure will be described below. In the following exemplary embodiment, for elements whose actions and functions are similar to those of the first embodiment, the reference numerals used in the description of the first embodiment will be used, and detailed descriptions of each element will be omitted as appropriate.

[0064] 6 is a block diagram of a projector 10A in the second embodiment. The projector 10A is similar to the projector 10 in the first embodiment except that it uses a program PR2 instead of the program PR1 in the first embodiment.

[0065] The storage device 11 of the projector 10A stores a program PR2, temporary variable information PA-1 to PA-m, variable information PAx, and correction value information DC.

[0066] In this embodiment, each of the multiple variables indicated by the temporary variable information PA-1 to PA-m ​​is an example of a "calculated value." Any of the multiple variables indicated by the temporary variable information PA-1 to PA-m ​​is an example of a "first calculated value," and any other correction value is an example of a "second calculated value." The first calculated value is a variable at a first time. The second calculated value is a variable at a second time. In an aspect in which such variables are used as calculated values, the processing load for calculating the first calculated value and the second calculated value can be reduced.

[0067] The program PR2 is a program for executing a correction method, which will be described in detail later. The variable information PAx is information indicating the average, median, or mode of the variables indicated by two or more pieces of temporary variable information PA, and indicates the degree of geometric correction of the image G.

[0068] In the projector 10A, the processing device 12 executes the program PR2 stored in the storage device 11, thereby performing various processes required for the correction method described below.

[0069] 7 is a flowchart showing the flow of a correction method according to the second embodiment. The correction method according to the second embodiment is the same as the correction method according to the first embodiment, except that steps S20A and S30A are included instead of steps S20 and S30 of the first embodiment.

[0070] Step S20A is the same as step S20 of the first embodiment except that step S22 of the first embodiment is omitted. That is, in step S21 of step S20A, the processing device 12 calculates a variable at the current time as a calculated value based on the detection result of the sensor 17 at the current time.

[0071] After step S20A, in step S30A, the processing device 12 determines a correction value based on the variables indicated by the most recent n pieces of temporary variable information PA. Step S30A includes steps S32 and S33.

[0072] In step S32, the processing device 12 performs statistical processing on the variables indicated by the temporary variable information PA for the most recent n times. Information indicating the results of the statistical processing is stored as variable information PAx in the storage device 11. The statistical processing in step S32 is processing to obtain the average value, mode, or median of the variables for the most recent n times.

[0073] Thereafter, in step S33, the processing device 12 calculates a correction value using the variable indicated by the variable information PAx. This calculation method is, for example, the same as the method described in the first embodiment. Information indicating the calculated correction value is stored in the storage device 11 as correction value information DC.

[0074] According to the second embodiment described above, the sense of discomfort felt by the user due to the correction of the image G can also be alleviated.

[0075] 3. Variations The above-described embodiments can be modified in various ways. Specific modifications that can be applied to the above-described embodiments are exemplified below. Two or more embodiments arbitrarily selected from the following examples can be combined as appropriate within the scope of not mutually contradictory.

[0076] 3-1. Variation 1 In the above-described embodiment, the sensor 17 includes the distance sensor 17a and the acceleration sensor 17b, but this is not limited to this embodiment, and for example, one of the distance sensor 17a and the acceleration sensor 17b may be omitted.

[0077] For example, if the change in the attitude of the projectors 10, 10A is only a change around the front-to-back axis of the projectors 10, 10A, the distance sensor 17a may be omitted. Also, if there is no change in the attitude of the projectors 10, 10A, the acceleration sensor 17b may be omitted.

[0078] 3-2. Variation 2 In the above embodiment, the statistical processing in steps S30 and 30A is exemplified as calculating an average value, but the present invention is not limited to this, and the statistical processing may be calculated as a mode value or a median value. Furthermore, the statistical processing may be performed by filtering.

[0079] 4. Notes A summary of this disclosure is provided below.

[0080] (Supplementary Note 1) A first aspect, which is a preferred example of the correction method of the present disclosure, includes obtaining a first calculated value for correcting the shape of the image at a first time based on a detection result at a first time of a sensor for estimating a relative positional relationship between a projector that projects an image onto a projection surface and the projection surface; obtaining a second calculated value for correcting the shape of the image at a second time based on a detection result at a second time different from the first time of the sensor; determining a correction value for geometric correction based on one of the average, median, or mode of multiple calculated values ​​including the first calculated value and the second calculated value; and correcting the image using the correction value.

[0081] In the above-described embodiment, the geometric correction value is determined based on the average, median, or mode of multiple calculated values ​​based on the detection results of the sensor, which makes it possible to smooth the change in shape of the corrected image compared to embodiments that use the correction value based on the calculated value at the current time as is, thereby reducing the sense of discomfort felt by the user due to the image correction.

[0082] (Note 2) In the second aspect, which is a preferred example of the first aspect, the image is corrected at a frequency higher than the sampling rate of the sensor. In this aspect, the change in shape of the corrected image can be made smoother than in an aspect in which the image is corrected at a frequency lower than the sampling rate of the sensor.

[0083] (Supplementary Note 3) In a third aspect which is a preferred example of the second aspect, the frequency is a first frequency which is equal to or lower than the frame rate of the image. In the above aspect, it is possible to reduce the processing load of image correction while smoothing the change in shape of the corrected image.

[0084] (Supplementary Note 4) In a fourth aspect which is a preferred example of any of the first to third aspects, the sensor includes a time-of-flight distance sensor. In the above aspect, the effects of the present disclosure become more pronounced.

[0085] (Supplementary Note 5) In a fifth aspect which is a preferred example of any of the first to fourth aspects, the shape of the image is rectangular, the first calculated values ​​are coordinate values ​​of four vertices of the image after correction at the first time, and the second calculated values ​​are coordinate values ​​of four vertices of the image after correction at the second time. In the above aspect, it is possible to reduce the processing load for determining correction values.

[0086] (Supplementary Note 6) In a sixth aspect which is a preferred example of any of the first to fourth aspects, the positional relationship is expressed by a variable related to at least one of an installation angle of the projection surface, a normal vector of the projection surface, and an attitude of the projector, the first calculated value is the variable at the first time, and the second calculated value is the variable at the second time. In the above aspect, it is possible to reduce the processing load of calculating the first calculated value and the second calculated value.

[0087] (Appendix 7) A seventh aspect, which is a preferred example of an information processing device of the present disclosure, includes at least one processor that performs the following operations: acquiring a first calculated value for correcting the shape of the image at a first time based on a detection result at a first time of a sensor for estimating a relative positional relationship between a projector that projects an image onto a projection surface and the projection surface; acquiring a second calculated value for correcting the shape of the image at a second time based on a detection result at a second time different from the first time of the sensor; determining a correction value for geometric correction based on one of the average, median, or mode of multiple calculated values ​​including the first calculated value and the second calculated value; and correcting the image using the correction value.

[0088] In the above-described embodiment, the geometric correction value is determined based on the average, median, or mode of multiple calculated values ​​based on the detection results of the sensor, which makes it possible to smooth the change in shape of the corrected image compared to embodiments that use the correction value based on the calculated value at the current time as is, thereby reducing the sense of discomfort felt by the user due to the image correction.

[0089] (Appendix 8) An eighth aspect, which is a preferred example of the program of the present disclosure, causes a computer to perform the following operations: based on a detection result at a first time of a sensor for estimating a relative positional relationship between a projector that projects an image onto a projection surface and the projection surface, obtain a first calculated value for correcting the shape of the image at the first time; based on a detection result at a second time of the sensor that is different from the first time, obtain a second calculated value for correcting the shape of the image at the second time; based on a detection result at the sensor at the second time, which is different from the first time, determine a correction value for geometric correction based on one of the average, median, or mode of multiple calculated values ​​including the first calculated value and the second calculated value; and correct the image using the correction value.

[0090] In the above-described embodiment, the geometric correction value is determined based on the average, median, or mode of multiple calculated values ​​based on the detection results of the sensor, which makes it possible to smooth the change in shape of the corrected image compared to embodiments that use the correction value based on the calculated value at the current time as is, thereby reducing the sense of discomfort felt by the user due to the image correction. [Explanation of symbols]

[0091] 10...projector, 10A...projector, 11...storage device, 12...processing device, 13...communication device, 14...image processing circuit, 15...optical device, 15a...light source, 15b...light modulator, 15c...projection optical system, 16...operation device, 17...sensor, 17a...distance sensor, 17b...acceleration sensor, 100...system, A...problem, B...problem, C...problem, D1-1 to D1-m...provisional correction value information, DC...correction value information, G...image, IM G...image data, P1 to P4...coordinate values, PA-1 to PA-m...temporary variable information, PAx...variable information, PR1...program, PR2...program, RP...projection area, S10...step, S20...step, S20A...step, S21...step, S22...step, S30...step, S30A...step, S31...step, S32...step, S33...step, S40...step, S50...step, SC...projection surface.

Claims

1. acquiring a first calculated value for correcting a shape of the image at a first time based on a detection result at the first time of a sensor for estimating a relative positional relationship between a projector that projects an image onto a projection surface and the projection surface; acquiring a second calculated value for correcting the shape of the image at a second time, the second time being different from the first time, based on a detection result of the sensor at the second time; determining a correction value for geometric correction based on one of an average value, a median value, or a mode value of a plurality of calculated values ​​including the first calculated value and the second calculated value; correcting the image using the correction value; A correction method comprising:

2. performing the correcting of the image at a frequency greater than the sampling rate of the sensor; The correction method according to claim 1 .

3. the frequency is a first frequency that is equal to or less than a frame rate of the image; The correction method according to claim 2 .

4. the sensor includes a time-of-flight distance sensor; The correction method according to any one of claims 1 to 3.

5. The shape of the image is rectangular, the first calculated values ​​are coordinate values ​​of four vertices of the image after correction at the first time; the second calculated values ​​are coordinate values ​​of four vertices of the image after correction at the second time; The correction method according to claim 1 .

6. the positional relationship is expressed by a variable related to at least one of an installation angle of the projection surface, a normal vector of the projection surface, and an attitude of the projector; the first calculated value is the variable at the first time; the second calculated value is the variable at the second time. The correction method according to claim 1 .

7. acquiring a first calculated value for correcting a shape of the image at a first time based on a detection result at the first time of a sensor for estimating a relative positional relationship between a projector that projects an image onto a projection surface and the projection surface; acquiring a second calculated value for correcting the shape of the image at a second time, the second time being different from the first time, based on a detection result of the sensor at the second time; determining a correction value for geometric correction based on one of an average value, a median value, or a mode value of a plurality of calculated values ​​including the first calculated value and the second calculated value; correcting the image using the correction value; An information processing device comprising at least one processor that executes

8. acquiring a first calculated value for correcting a shape of the image at a first time based on a detection result at the first time of a sensor for estimating a relative positional relationship between a projector that projects an image onto a projection surface and the projection surface; acquiring a second calculated value for correcting the shape of the image at a second time, the second time being different from the first time, based on a detection result of the sensor at the second time; determining a correction value for geometric correction based on one of an average value, a median value, or a mode value of a plurality of calculated values ​​including the first calculated value and the second calculated value; correcting the image using the correction value; A program that causes a computer to execute the following.

Images