Projection method and projection system

Abstract

To display a size related to a projection image being displayed on a projection surface.SOLUTION: A projection method includes: generating, based on information output from a sensor, projection size information including at least one of first size information indicating the distance between two points included in a first projection image displayed on a projection surface and second size information indicating the area of a first image included in the first projection image; and displaying, on the projection surface, a second projection image including an image showing numerical values indicated by the projection size information.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a projection method and a projection system. [Background technology]

[0002] When an image is displayed on a projection surface using a device such as a projector, the size of the image displayed on the projection surface generally changes depending on the distance between the device and the projection surface. Technologies have been developed that focus on the size of the image displayed on the projection surface by projecting light. For example, Patent Document 1 discloses a projector that keeps the size of the displayed image constant by adjusting image data according to the distance even when the distance from an image projection unit that projects an image onto the projection surface to the projection surface changes. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-141411 A Summary of the Invention [Problem to be solved by the invention]

[0004] The projector described in Patent Document 1 was unable to display the size of the image displayed on the projection surface. Therefore, in order for a user to grasp the size of the displayed image, the user had to, for example, actually measure the size of the image on the projection surface, which was time-consuming. [Means for solving the problem]

[0005] One aspect of the projection method of the present invention includes generating projection size information based on information output from a sensor, the projection size information including at least one of first size information indicating the distance between two points included in a first projection image displayed on a projection surface and second size information indicating the area of ​​a first image included in the first projection image, and displaying a second projection image on the projection surface including an image showing a numerical value indicated by the projection size information.

[0006] One aspect of the projection system of the present invention includes a processing device, a sensor, and a projection mechanism, and the processing device generates projection size information based on information output from the sensor, the projection size information including at least one of first size information indicating the distance between two points included in a first projection image displayed on the projection surface, and second size information indicating the area of ​​a first image included in the first projection image, and controls the projection mechanism to display a second projection image on the projection surface, the second projection image including an image representing a numerical value indicated by the projection size information. [Brief description of the drawings]

[0007] [Figure 1] FIG. 13 is a schematic diagram illustrating a state in which a projected image GP1 is displayed. [Diagram 2] FIG. 13 is a schematic diagram illustrating a state in which a projection image GP2 is displayed. [Diagram 3] 1 is a block diagram showing the configuration of a projector 1 according to a first embodiment. [Figure 4] 1 is a block diagram showing a configuration of a storage device 10 according to a first embodiment. [Diagram 5] 1 is a schematic diagram illustrating an example of an image represented by the first projection image information 106. FIG. [Figure 6] 2 is a schematic diagram illustrating an example of an image indicated by captured image information 102. FIG. [Figure 7] 13 is a schematic diagram illustrating an example of an image represented by the second projection image information 107. FIG. [Figure 8] 1 is a flowchart for explaining the operation of the projector 1 according to the first embodiment. [Figure 9]FIG. 11 is a schematic diagram illustrating a state in which a projected image GP3 is displayed. [Figure 10] 13 is a schematic diagram illustrating a state in which a projected image GP4 is displayed. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. Note that the dimensions and scale of each part in the drawings may differ from the actual dimensions, and some parts are shown diagrammatically to facilitate understanding. Furthermore, the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited thereto.

[0009] 1. First embodiment In the first embodiment, a projection method and a projection system according to the present invention will be described by taking as an example a projector that calculates the size of a projection image displayed on a projection surface based on captured image information output from a camera that captures the projection image, and displays a number representing the calculated size on the projection surface. Here, the number representing the calculated size is an example of an image representing a numerical value indicated by the projection size information. In this embodiment, numbers are used, but are not limited to this. For example, the size of the projection image may be expressed using letters such as the alphabet.

[0010] 1.1.Projector Overview Hereinafter, an overview of a projector 1 according to a first embodiment will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is a schematic diagram illustrating how a projected image GP1 is displayed. Fig. 2 is a schematic diagram illustrating how a projected image GP2 is displayed.

[0011] The projector 1 includes a housing 18. The housing 18 includes a projection mechanism 16 that displays an image on a projection surface by projecting projection light, and a camera 14 that captures the image displayed on the projection surface by the projection mechanism 16. The camera 14 includes an imaging lens 140 and an imaging element 142.

[0012] In the following description, the images displayed on the projection surface by the projection light projected from the projector 1 will be collectively referred to as "projected image GP."

[0013] The projector 1 controls the projection mechanism 16 to display a projection image GP1 on a wall surface W1.

[0014] The projected image GP1 includes an image GP11 and an image GP12. The image GP11 includes an image object OP1, an image object OP2, an image object OP3, and an image object OP4. The image objects OP1 to OP4 are images having a geometric pattern. Furthermore, the image objects OP1 to OP4 each have a plurality of vertices. Furthermore, the image GP11 has a plurality of vertices. The image GP12 is an image having arrow-shaped ends, and its longitudinal direction extends along one of the two diagonals of the projected image GP1. The image GP12 indicates the upper left vertex as seen from the user and the lower right vertex as seen from the user, among the plurality of vertices of the projected image GP1. Furthermore, the image GP12 has a plurality of vertices, similar to the image GP11.

[0015] Moreover, the projected image GP1 includes points DP1, DP2, DP3, DP4, DP5, and DP6. Point DP1 is located at one of the vertices of the image object OP1. Point DP2 is located at one of the vertices of the image object OP2. Point DP3 is located at one of the vertices of the image object OP3. Point DP4 is located at one of the vertices of the image object OP4. Point DP5 is located at the upper left vertex as seen by the user among the vertices of the projected image GP1 indicated by the image GP12. Point DP6 is located at the lower right vertex as seen by the user among the vertices of the projected image GP1 indicated by the image GP12.

[0016] In FIG. 1 and FIG. 2, an axis parallel to the vertical direction is called the Y axis. Among the directions parallel to the Y axis, the vertical direction is defined as the -Y direction, and the direction opposite to the -Y direction is defined as the Y direction. An axis perpendicular to the Y axis and parallel to the optical axis of the projection mechanism 16 is called the Z axis. Among the directions parallel to the Z axis, the direction from the projector 1 to the wall surface W1 is defined as the Z direction, and the direction opposite to the Z direction is defined as the -Z direction. An axis perpendicular to the Y axis and the Z axis is called the X axis. Among the directions parallel to the X axis, the direction from the image object OP4 to the image object OP2 is defined as the X direction, and the direction opposite to the X direction is defined as the -X direction. The origin of the XYZ coordinate system coincides with the optical center of the projection mechanism 16. That is, the XYZ coordinate system is a coordinate system that indicates the positional relationship of each point with the optical center of the projection mechanism 16 as a reference. Moreover, the coordinates of a point in the XYZ coordinate system are expressed as (x, y, z). Here, the value x represents the X coordinate of the point in the XYZ coordinate system, the value y represents the Y coordinate of the point in the XYZ coordinate system, and the value z represents the Z coordinate of the point in the XYZ coordinate system.

[0017] The camera 14 captures an image of the imaging area E1 including the projection image GP1 to obtain the captured image. The projector 1 calculates the size of the projection image GP1 displayed on the wall surface W1, for example, the distance between the points DP5 and DP6, based on captured image information representing the captured image obtained by the camera 14. In this embodiment, the distance between the points DP5 and DP6 is equal to the length of the diagonal line of the projection image GP1.

[0018] The projector 1 may calculate the distance between any two points included in the projected image GP1 as the size of the projected image GP1. The projector 1 may also calculate the area of ​​an image included in the projected image GP1, for example, the area of ​​the image GP11 or the area of ​​the image GP12, as the size of the projected image GP1. In this case, the image GP11 or the image GP12 is an example of the "first image". On the other hand, the image included in the projected image GP1 may match the projected image GP1. That is, the projector 1 may calculate the area of ​​the projected image GP1 as the size of the projected image GP1. In this case, the image that matches the projected image GP1 is an example of the "first image". In addition, the size of the projected image GP1 is not limited to the above form. The size of an arbitrary image set in the projected image GP1 may be calculated as the size of the projected image GP1. The arbitrary image may be any image whose area can be calculated, and may be, for example, any polygon surrounded by three or more points. In this case, the arbitrary image set in the projected image GP1 is an example of the "first image".

[0019] When the distance between the points DP5 and DP6 is calculated as the size of the projection image GP1, the projector 1 controls the projection mechanism 16 to display the projection image GP2 on the wall surface W1.

[0020] The projected image GP2 is an image in which the character string Q1 is superimposed on the projected image GP1. That is, various dimensions of the projected image GP2 are the same as those of the projected image GP1. The character string Q1 includes a number Q11 and a character string Q12. The number Q11 is an image that represents the size of the projected image GP1. Specifically, the number Q11 is an image that includes the number "120" that represents the distance between the points DP5 and DP6, that is, the length of the diagonal of the projected image GP1 and the projected image GP2. More specifically, the number Q11 is an image that includes a number that displays the length of the diagonal of the projected image GP1 and the projected image GP2 in inches. The character string Q12 is a character string that represents the unit of the size of the projected image GP1. Specifically, the character string Q12 is a character string "inch" that represents that the length of the diagonal of the projected image GP1 and the projected image GP2 is displayed in inches.

[0021] In addition, when the area of ​​the image contained in the projected image GP1, for example the area of ​​the image GP11, is calculated as the size of the projected image GP1, the projector 1 controls the projection mechanism 16 to display the projected image GP including an image representing the area of ​​the image GP11 on the wall surface W1.

[0022] By checking the character string Q1, the user can ascertain that the diagonal length of the projected image GP1 (projected image GP2) is 120 inches. In addition, the user can easily adjust the size of the projected image GP to a desired size by changing the position of the projector 1 while checking the image showing the size of the projected image GP displayed on the wall surface W1.

[0023] 1.2.Projector configuration and functions The configuration and functions of the projector 1 according to the first embodiment will be described below with reference to FIGS.

[0024] 3 is a block diagram showing the configuration of the projector 1 according to the first embodiment. The projector 1 includes a storage device 10 that stores various information, a processing device 12 that controls the operation of the projector 1, a camera 14 that captures a projection image GP displayed on a projection surface, a projection mechanism 16 that displays the projection image GP on the projection surface by projecting projection light, a housing 18 that stores various components that constitute the projector 1, an operation device 20 that accepts input operations from a user, and a communication device 22 that executes communication with an external storage device or an external server. The processing device 12 has functions as an acquisition unit 120, a projection control unit 121, an input management unit 122, a detection unit 123, a size information generation unit 124, and an image generation unit 125.

[0025] The storage device 10 includes, for example, a volatile memory such as a RAM and a non-volatile memory such as a ROM. Here, RAM is an abbreviation for Random Access Memory, and ROM is an abbreviation for Read Only Memory.

[0026] 4 is a block diagram showing the configuration of the storage device 10 according to the first embodiment. The non-volatile memory of the storage device 10 stores a program 100 that specifies the operation of the projector 1, size setting information 101 that indicates the setting contents related to the display of the size of the projection image GP displayed on the projection surface, captured image information 102 that indicates the result of capturing the projection image GP displayed on the projection surface, position information 104 that indicates the positional relationship between the camera 14 and the projection mechanism 16, projection image information 105 for forming the projection light projected when displaying the projection image GP on the projection surface, coordinate information 108 that indicates the coordinates of points included in various images, projection size information 111 that indicates the size of the projection image GP displayed on the projection surface, internal parameters 114 that indicate various variables unique to the device, and calculation parameters 103 that indicate various variables calculated based on the captured image information 102, the projection image information 105, and the internal parameters 114. The projection image information 105 includes first projection image information 106 and second projection image information 107. The coordinate information 108 includes first coordinate information 109 and second coordinate information 110. The projection size information 111 includes first size information 112 indicating the distance between any two points included in the projection image GP displayed on the projection surface, and second size information 113 indicating the area of ​​the image included in the projection image GP. The internal parameters 114 include projection parameters 115 representing variables derived from the projection mechanism 16, and camera parameters 116 representing variables derived from the camera 14.

[0027] The projector 1 displays the numerical value indicated by the projection size information 111 on the projection surface. The projection size information 111 may include at least one of the first size information 112 and the second size information 113. In this embodiment, the projection size information 111 includes the first size information 112. Specifically, the first size information 112 indicates the distance between the point DP5 and the point DP6. The number Q11 is an image for notifying the user of the numerical value indicated by the first size information 112. In addition, when the projection size information 111 includes the second size information 113, the second size information 113 indicates, for example, the area of ​​the image GP11 or the area of ​​the image GP12.

[0028] The size setting information 101 includes information indicating what size the numerical value displayed by the projector 1 on the projection surface represents. That is, the size setting information 101 includes information specifying whether the numerical value displayed by the projector 1 on the projection surface is based on the first size information 112 or the second size information 113. When the numerical value displayed on the projection surface is based on the first size information 112, the size setting information 101 includes information indicating the coordinates of two points included in the image indicated by the projection image information 105, which correspond one-to-one with two points that are the reference of the distance to be displayed. When the numerical value displayed on the projection surface is based on the second size information 113, the size setting information 101 includes information regarding an image having an area to be displayed. When the numerical value displayed on the projection surface is based on the second size information 113, the size setting information 101 includes, for example, information indicating the ratio of the area of ​​an image included in the image indicated by the projection image information 105, which corresponds to the image having an area to be displayed, to the entire image indicated by the projection image information 105.

[0029] The position information 104 is, for example, information indicating a position vector T' that represents the position of the camera 14 with respect to the projection mechanism 16. The magnitude of the vector T' coincides with the distance between the optical center of the camera 14 and the optical center of the projection mechanism 16.

[0030] Furthermore, the volatile memory of the storage device 10 is used by the processing device 12 as a work area when the program 100 is executed.

[0031] A part or all of the storage device 10 may be provided in an external storage device or an external server. A part or all of the various information stored in the storage device 10 may be stored in advance in the storage device 10, or may be acquired from an external storage device or an external server. In this embodiment, the storage device 10 is provided in the housing 18. The location information 104 is stored in advance in the storage device 10.

[0032] 5 is a schematic diagram illustrating an example of an image indicated by the first projection image information 106. In this embodiment, the first projection image information 106 indicates an image GF1. The projector 1 causes the projection mechanism 16 to project projection light based on the first projection image information 106, thereby displaying a projection image GP1 on the wall surface W1. In other words, the first projection image information 106 indicates an image that does not include an image indicating the size of the projection image GP.

[0033] The image GF1 is an image corresponding to the projected image GP1. The image GF1 includes an image GF11 and an image GF12.

[0034] Image GF11 is an image corresponding to image GP11. Image GF11 includes image objects OF1, OF2, OF3, and OF4. Image object OF1 corresponds to image object OP1. Image object OF2 corresponds to image object OP2. Image object OF3 corresponds to image object OP3. Image object OF4 corresponds to image object OP4. Image objects OF1 to OF4 are images having a geometric pattern in which a plurality of rectangles are combined. That is, image objects OF1 to OF4 each have a plurality of vertices. Furthermore, image GF11 has a plurality of vertices.

[0035] Image GF12 is an image corresponding to image GP12. Image GF12 is an image with both ends shaped like an arrow, and its longitudinal direction extends along one of the two diagonals of image GF1. Image GF12 indicates the upper left vertex and the lower right vertex of the multiple vertices of image GF1. Image GF12 also has multiple vertices, similar to image GF11.

[0036] Moreover, the image GF1 includes points DF1, DF2, DF3, DF4, DF5, and DF6. The point DF1 corresponds to the point DP1. The point DF1 is located at one of the vertices of the image object OF1. The point DF2 corresponds to the point DP2. The point DF2 is located at one of the vertices of the image object OF2. The point DF3 corresponds to the point DP3. The point DF3 is located at one of the vertices of the image object OF3. The point DF4 corresponds to the point DP4. The point DF4 is located at one of the vertices of the image object OF4. The point DF5 corresponds to the point DP5. The point DF5 is located at the upper left vertex of the vertices of the image GF1 indicated by the image GF12. The point DF6 corresponds to the point DP6. Point DF6 is located at the lower right vertex of image GF1 pointed to by image GF12.

[0037] In this embodiment, size setting information 101 includes information specifying that the numerical values ​​to be displayed on the projection surface by projector 1 are based on first size information 112. That is, size setting information 101 includes information indicating the coordinates of two points included in the image indicated by projection image information 105, which correspond one-to-one to points DP5 and DP6 that are references for the distance between points DP5 and DP6 to be displayed. Specifically, size setting information 101 includes information indicating the coordinates of point DF5 corresponding to point DP5, and the coordinates of point DF6 corresponding to point DP6.

[0038] In image GF1, the top left vertex of the image is defined as the origin, the direction parallel to the longitudinal direction of image GF1 and going right from the origin as one faces the image as the J direction, and the direction parallel to the lateral direction of image GF1 and going downward from the origin as one faces the image as the K direction. The axis extending from the origin in the J direction is defined as the J axis, and the axis extending from the origin in the K direction is defined as the K axis. The coordinates of a point in image GF1 are expressed as (j, k). Here, the value j represents the J coordinate of the point in image GF1. The value k represents the K coordinate of the point in image GF1.

[0039] 6 is a schematic diagram illustrating an example of an image indicated by the captured image information 102. In this embodiment, the captured image information 102 represents the result of capturing a projection image GP displayed on a wall surface W1. The captured image information 102 indicates, for example, a captured image GS1. The captured image GS1 includes an image GS11. The image GS11 is an image indicating the wall surface W1. The image GS11 includes an image GV1. The image GV1 is an image indicating the projection image GP1.

[0040] Image GV1 includes image objects OV1, OV2, OV3, and OV4. Image object OV1 corresponds to image object OP1. That is, image object OV1 corresponds to image object OF1. Image object OV2 corresponds to image object OP2. That is, image object OV2 corresponds to image object OF2. Image object OV3 corresponds to image object OP3. That is, image object OV3 corresponds to image object OF3. Image object OV4 corresponds to image object OP4. That is, image object OV4 corresponds to image object OF4. Image objects OV1 to OV4 are images having geometric patterns. Furthermore, each of image objects OV1 to OV4 has a plurality of vertices.

[0041] Moreover, the image GV1 includes an image GV12. The image GV12 is an image corresponding to the image GP12. That is, the image GV12 is an image corresponding to the image GF12. The image GV12 is an image with both ends shaped like an arrow, and its longitudinal direction extends along one of the two diagonals of the image GV1. The image GV12 indicates the upper left vertex and the lower right vertex of the multiple vertices of the image GV1. The image GV12 also has multiple vertices.

[0042] Moreover, the image GV1 includes points DV1, DV2, DV3, DV4, DV5, and DV6. The point DV1 is a point corresponding to the point DP1. That is, the point DV1 is a point corresponding to the point DF1. The point DV1 is located at one of the multiple vertices of the image object OV1. The point DV2 is a point corresponding to the point DP2. That is, the point DV2 is a point corresponding to the point DF2. The point DV2 is located at one of the multiple vertices of the image object OV2. The point DV3 is a point corresponding to the point DP3. That is, the point DV3 is a point corresponding to the point DF3. The point DV3 is located at one of the multiple vertices of the image object OV3. The point DV4 is a point corresponding to the point DP4. That is, the point DV4 is a point corresponding to the point DF4. Point DV4 is located at one of a plurality of vertices of image object OV4, and therefore points DV1 to DV4 correspond one-to-one to points DF1 to DF4.

[0043] Point DV5 corresponds to point DP5. That is, point DV5 corresponds to point DF5. Point DV5 is located at the upper left vertex of the vertices of image GV1 pointed to by image GV12. Point DV6 corresponds to point DP6. That is, point DV6 corresponds to point DF6. Point DV6 is located at the lower right vertex of the vertices of image GV1 pointed to by image GV12.

[0044] In the captured image GS1, the top left vertex of the image is defined as the origin, the direction parallel to the longitudinal direction of the captured image GS1 and extending from the origin to the right as viewed in the image as the M direction, and the direction parallel to the lateral direction of the captured image GS1 and extending downward from the origin as viewed in the image as the N direction. The axis extending from the origin in the M direction is defined as the M axis, and the axis extending from the origin in the N direction is defined as the N axis. The coordinates of a point in the captured image GS1 are expressed as (m, n). Here, the value m represents the M coordinate of the point in the captured image GS1. The value n represents the N coordinate of the point in the captured image GS1.

[0045] 3, the processing device 12 includes one or more CPUs. However, the processing device 12 may include a programmable logic device such as an FPGA instead of or in addition to a CPU. Here, CPU is an abbreviation for Central Processing Unit, and FPGA is an abbreviation for Field-Programmable Gate Array.

[0046] The processing device 12 functions as an acquisition unit 120, a projection control unit 121, an input management unit 122, a detection unit 123, a size information generation unit 124, and an image generation unit 125 shown in FIG. 3 by the CPU or the like of the processing device 12 executing the program 100.

[0047] The acquisition unit 120 controls the communication device 22 to acquire various information from a terminal device, an external storage device, an external server, or the like that is communicatively connected to the projector 1. The acquisition unit 120 also stores the acquired various information in the storage device 10. In this embodiment, the acquisition unit 120 acquires the first projection image information 106 and the first coordinate information 109 from the external server that is communicatively connected to the projector 1. The acquisition unit 120 also stores the acquired first projection image information 106 and the first coordinate information 109 in the storage device 10. The acquisition unit 120 also acquires the captured image information 102 output from the image sensor 142 included in the camera 14. The acquisition unit 120 also stores the acquired captured image information 102 in the storage device 10.

[0048] The projection control unit 121 controls the projection mechanism 16 to project the projection light for displaying the projection image GP onto the projection surface. In this embodiment, the projection control unit 121 controls the projection mechanism 16 to project the projection light for displaying the projection image GP onto the wall surface W1. Specifically, the projection control unit 121 causes the projection mechanism 16 to project the projection light based on the projection image information 105, thereby displaying the projection image GP on the wall surface W1. More specifically, the projection control unit 121 causes the projection mechanism 16 to project the projection light based on the first projection image information 106, thereby displaying the projection image GP1 on the wall surface W1. In addition, the projection control unit 121 causes the projection mechanism 16 to project the projection light based on the second projection image information 107, thereby displaying the projection image GP2 on the wall surface W1.

[0049] The input management unit 122 acquires operation data indicating the content of an operation accepted from a user by controlling the operation device 20. Furthermore, the input management unit 122 generates various types of information based on the acquired operation data.

[0050] In this embodiment, the input management unit 122 acquires operation data representing the content of an operation for setting the display of the size of the projection image GP displayed on the projection surface. The input management unit 122 also generates size setting information 101 based on the operation data. The input management unit 122 also stores the generated size setting information 101 in the storage device 10. Note that, hereinafter, an operation for setting the display of the size of the projection image GP displayed on the projection surface may be referred to as a "size setting operation."

[0051] The detection unit 123 detects points included in an image by executing image processing on the image indicated by various image information. That is, the detection unit 123 acquires coordinate information 108 indicating the coordinates of the detected points. The detection unit 123 also stores the acquired coordinate information 108 in the storage device 10.

[0052] In this embodiment, the detection unit 123 detects a plurality of points included in the image represented by the captured image information 102 by executing image processing on the image represented by the captured image information 102. That is, the detection unit 123 acquires second coordinate information 110 indicating the coordinates of a plurality of points included in the image represented by the captured image information 102. Furthermore, the detection unit 123 stores the acquired second coordinate information 110 in the storage device 10. Specifically, the detection unit 123 detects a plurality of points including points DV1 to DV4 as a plurality of points included in the captured image GS1 by executing image processing on the captured image GS1. The second coordinate information 110 indicates the coordinates of a plurality of points included in the captured image GS1. In other words, the second coordinate information 110 indicates the coordinates of a plurality of points including points DV1 to DV4.

[0053] The detection unit 123 may detect a plurality of points included in the image represented by the projection image information 105 by executing image processing on the image represented by the projection image information 105. When the detection unit 123 detects a plurality of points included in the image represented by the projection image information 105, the detection unit 123 acquires first coordinate information 109 indicating the coordinates of the plurality of points included in the image represented by the projection image information 105. Furthermore, the detection unit 123 stores the acquired first coordinate information 109 in the storage device 10. Note that in this embodiment, the first coordinate information 109 indicates the coordinates of a plurality of points included in the image represented by the first projection image information 106. Specifically, the first coordinate information 109 indicates the coordinates of a plurality of points included in the image GF1. More specifically, the first coordinate information 109 indicates the coordinates of a plurality of points including the points DF1 to DF4.

[0054] A known image processing technique may be used in the function related to point detection. Examples of known image processing techniques related to point detection include template matching and an algorithm called "AKAZE". In this specification, detailed technical explanations related to point detection are omitted.

[0055] The size information generation unit 124 generates the projection size information 111 based on the information output from the sensor. In this embodiment, the size information generation unit 124 generates the projection size information 111 based on the captured image information 102 output from the image sensor 142, the position information 104, and the projection image information 105.

[0056] Specifically, the size information generating unit 124 generates the calculation parameters 103 based on the second coordinate information 110 indicating the coordinates of a point included in the image indicated by the captured image information 102, the position information 104, the first coordinate information 109 indicating the coordinates of a point included in the image indicated by the projection image information 105, and the internal parameters 114. More specifically, the size information generating unit 124 generates the calculation parameters 103 based on the second coordinate information 110 indicating the coordinates of a point included in the captured image GS1, the position information 104, the first coordinate information 109 indicating the coordinates of a point included in the image GF1, and the internal parameters 114. In addition, the size information generating unit 124 generates the projection size information 111 based on the size setting information 101, the calculation parameters 103, and the internal parameters 114. In this embodiment, the size information generating unit 124 generates the first size information 112 based on the size setting information 101, the calculation parameters 103, and the internal parameters 114.

[0057] The projection parameters 115 included in the internal parameters 114 include information indicating the focal length of the projection mechanism 16, and information indicating the coordinates of the point where the center of the projection lens included in the projection mechanism 16 overlaps with the image GF1 in the JK coordinate system, in other words, the coordinates of the point where the optical axes of the image GF1 and the projection lens included in the projection mechanism 16 virtually intersect. The camera parameters 116 included in the internal parameters 114 include information indicating the focal length of the imaging lens 140, and information indicating the coordinates of the point where the center of the imaging lens 140 overlaps with the captured image GS1 in the MN coordinate system, in other words, the coordinates of the point where the optical axes of the captured image GS1 and the imaging lens 140 virtually intersect.

[0058] Various variables indicated by the calculation parameters 103 are derived from a projection transformation matrix generated based on at least four pairs of correspondences between the coordinates of a plurality of points indicated by the first coordinate information 109 normalized based on the projection parameters 115 and the coordinates of a plurality of points indicated by the second coordinate information 110 normalized based on the camera parameters 116. Specifically, the calculation parameters 103 include information indicating values ​​a, b, and c when the equation representing the projection surface in the XYZ coordinate system is ax+by+cz=1. The calculation parameters 103 also include information indicating a vector T representing the relative position of the camera 14 with respect to the projection mechanism 16 in the XYZ coordinate system. The calculation parameters 103 also include information indicating a rotation matrix R representing the attitude of the camera 14 with respect to the projection mechanism 16. The calculation parameters 103 also include information indicating a value s representing the ratio between the magnitude of the vector T' indicated by the position information 104 and the magnitude of the vector T.

[0059] When the values ​​a, b, and c indicated by the calculation parameters 103 are grasped, the coordinates of the points included in the projection image GP in the XYZ coordinate system are calculated from the correspondence between the points included in the image indicated by the projection image information 105 and the points included in the projection image GP. In addition, the size of the projection image GP is calculated based on the coordinates of the points included in the projection image GP in the XYZ coordinate system and the value s. In this embodiment, the coordinates of the points DP5 and DP6 are calculated based on the coordinates of the points DF5 and DF6 indicated by the size setting information 101 normalized based on the projection parameters 115 and the values ​​a, b, and c indicated by the calculation parameters 103. In addition, the distance between the points DP5 and DP6 is calculated based on the coordinates of the points DP5 and DP6 and the value s indicated by the calculation parameters 103. That is, based on the size setting information 101, the calculation parameters 103, and the internal parameters 114, first size information 112 indicating the distance between the points DP5 and DP6 is generated.

[0060] Note that, when the size setting information 101 includes information specifying that the numerical values ​​to be displayed on the projection surface by the projector 1 are based on the second size information 113, for example, the area of ​​the projection image GP is calculated based on the coordinates of multiple vertices of the projection image GP and the value s. Also, the area of ​​the image included in the projection image GP is calculated based on the area of ​​the projection image GP and the ratio of the area indicated by the size setting information 101. That is, the second size information 113 indicating the area of ​​the image included in the projection image GP is generated based on the size setting information 101, the calculation parameters 103, and the internal parameters 114.

[0061] The image generation section 125 generates the second projection image information 107 based on the first projection image information 106 and the projection size information 111 .

[0062] 7 is a schematic diagram illustrating an example of an image indicated by the second projection image information 107. The second projection image information 107 indicates an image including an image indicating a size related to the projection image GP. In other words, the second projection image information 107 indicates an image including an image indicating a numerical value indicated by the projection size information 111. In this embodiment, the second projection image information 107 indicates an image GF2. The projector 1 causes the projection mechanism 16 to project projection light based on the second projection image information 107, thereby displaying the projection image GP2 on the wall surface W1.

[0063] The image GF2 is an image corresponding to the projected image GP2. The image GF2 is an image in which the character string Q2 is superimposed on the image GF1. In other words, the various dimensions of the image GF2 are the same as those of the image GF1.

[0064] The character string Q2 corresponds to the character string Q1. The character string Q2 includes the number Q21 and the character string Q22.

[0065] The number Q21 corresponds to the number Q11. The number Q21 is an image that represents a numerical value indicated by the first size information 112. That is, the number Q21 is an image that includes the number "120" to indicate the length of the diagonal of the projection image GP1 and the projection image GP2 in inches.

[0066] The character string Q22 corresponds to the character string Q12. The character string Q22 is a character string indicating the unit of the numerical values ​​indicated by the first size information 112. In other words, the character string Q22 is the character string "inch" indicating that the diagonal lengths of the projection images GP1 and GP2 are displayed in inches.

[0067] In the following, generating the second projection image information 107 based on the first projection image information 106 and the projection size information 111 may be referred to as "updating the projection image information."

[0068] The camera 14 includes an imaging lens 140 for collecting light, and an imaging element 142 for converting the collected light into an electrical signal. The imaging element 142 is an image sensor, for example, a CCD or a CMOS. Here, CCD is an abbreviation for Charge Coupled Device, and CMOS is an abbreviation for Complementary Metal Oxide Semiconductor. The camera 14 captures an image of an imaging area E1 on the wall surface W1 on which the projection image GP is displayed. The imaging element 142 outputs captured image information 102 representing the result of capturing an image of the imaging area E1 on the wall surface W1 on which the projection image GP is displayed, to the processing device 12. In this embodiment, the camera 14 is provided in the housing 18.

[0069] The projection mechanism 16 includes a light source, an optical modulator that forms projection light based on a signal input from the processing device 12, and a projection optical system that projects the projection light onto a projection surface. The light source includes, for example, a halogen lamp, a xenon lamp, an ultra-high pressure mercury lamp, an LED, or a laser light source. The optical modulator includes, for example, a DMD or a liquid crystal panel. The projection optical system includes a projection lens group having a plurality of lenses. Here, LED is an abbreviation for Light Emitting Diode, and DMD is an abbreviation for Digital Mirror Device. The projection mechanism 16 projects projection light for displaying a projection image GP on a projection surface under the control of the projection control unit 121. In this embodiment, the projection mechanism 16 projects projection light for displaying the projection image GP on a wall surface W1. Specifically, the projection mechanism 16 displays the projection image GP1 by projecting an image GF1 onto the wall surface W1. The projection mechanism 16 projects the image GF2 onto the wall surface W1 to display a projection image GP2. The projection mechanism 16 is provided in a housing .

[0070] The operation device 20 accepts an input operation on the projector 1 from a user of the projector 1. The operation device 20 includes, for example, a touch panel or operation buttons provided on the housing 18. When the operation device 20 includes a touch panel, the operation device 20 outputs data indicating a detected touch position to the processing device 12. When the operation device 20 includes operation buttons, the operation device 20 outputs data identifying the pressed button to the processing device 12. In this manner, the content of the input operation on the projector 1 is transmitted to the processing device 12. In this embodiment, the operation device 20 accepts a size setting operation from the user. Furthermore, the operation device 20 outputs operation data indicating the content of the size setting operation to the processing device 12.

[0071] The communication device 22 includes, for example, an interface board having a connector and an interface circuit, and has a function of receiving various information from a terminal device, an external storage device, an external server, etc., and a function of transmitting various information to a terminal device, an external storage device, an external server, etc. The communication device 22 may transmit and receive various information using wired communication, or may transmit and receive various information using wireless communication. When wireless communication is used, the communication device 22 is configured to include an antenna compatible with wireless communication that complies with a predetermined communication standard. In this embodiment, the communication device 22 is communicably connected to an external server (not shown), and transmits and receives various information to and from the external server.

[0072] 1.3.Projector Operation 8 is a flowchart for explaining the operation of the projector 1 according to the first embodiment. A series of operations shown in the flowchart is started, for example, when the projector 1 is turned on and the operation device 20 receives an input operation for starting the operation from the user of the projector 1.

[0073] In step S101, the acquisition unit 120 controls the communication device 22 to acquire various pieces of information from an external server communicatively connected to the projector 1. Specifically, the acquisition unit 120 acquires the first projection image information 106 and the first coordinate information 109 from the external server. The acquisition unit 120 also stores the acquired first projection image information 106 and first coordinate information 109 in the storage device 10.

[0074] In step S102, the projection control unit 121 controls the projection mechanism 16 to project projection light for displaying the projection image GP1 onto the wall surface W1. In other words, the projection control unit 121 causes the projection mechanism 16 to project projection light based on the first projection image information 106, thereby displaying the projection image GP1 on the wall surface W1.

[0075] The operation device 20 accepts a size setting operation from a user of the projector 1. The operation device 20 also outputs operation data representing the content of the size setting operation to the processing device 12. In step S103, the input management unit 122 acquires the operation data representing the content of the size setting operation accepted from the user. The input management unit 122 also generates size setting information 101 based on the operation data. The input management unit 122 also stores the generated size setting information 101 in the storage device 10.

[0076] In step S104, the acquisition unit 120 acquires the captured image information 102 output from the image sensor 142. The acquisition unit 120 also causes the storage device 10 to store the acquired captured image information 102.

[0077] In step S105, the detection unit 123 performs image processing on the image indicated by the captured image information 102, and detects multiple points included in the image indicated by the captured image information 102. Specifically, the detection unit 123 performs image processing on the captured image GS1 indicated by the captured image information 102, and detects multiple points included in the captured image GS1. That is, the detection unit 123 acquires second coordinate information 110 indicating the coordinates of multiple points included in the image indicated by the captured image information 102.

[0078] In step S106, the detection unit 123 determines whether or not the detection of multiple points included in the image indicated by the captured image information 102 has been successful. Specifically, the detection unit 123 determines whether or not the detection of multiple points included in the captured image GS1 has been successful. If the detection of multiple points included in the image indicated by the captured image information 102 has been successful, that is, if YES in step S106, the detection unit 123 advances the process to step S107. If the detection of multiple points included in the image indicated by the captured image information 102 has failed, that is, if NO in step S106, the detection unit 123 advances the process to step S104.

[0079] For example, the detection unit 123 may compare the number of coordinates indicated by the first coordinate information 109 with the number of coordinates indicated by the second coordinate information 110. When the number of coordinates indicated by the first coordinate information 109 matches the number of coordinates indicated by the second coordinate information 110, the detection unit 123 may determine that detection of a plurality of points included in the image indicated by the captured image information 102 has been successful.

[0080] Also, for example, the image represented by the captured image information 102 may be blurred due to unintended vibrations or the like that occur when the camera 14 captures the projection image GP displayed on the wall surface W1. In such a case, the detection unit 123 may not be able to detect multiple points included in the image represented by the captured image information 102. In other words, when multiple points included in the image represented by the captured image information 102 are not detected, the detection unit 123 may determine that it has failed to detect multiple points included in the image represented by the captured image information 102.

[0081] If it is determined that the detection of the multiple points included in the image indicated by the captured image information 102 has failed, the processing device 12 reacquires the captured image information 102 in step S104. Then, in step S105, the processing device 12 executes image processing on the image indicated by the captured image information 102 to redetect the multiple points included in the image indicated by the captured image information 102.

[0082] In step S107 , the size information generating unit 124 generates the calculation parameters 103 based on the second coordinate information 110 , the position information 104 , the first coordinate information 109 , and the internal parameters 114 .

[0083] In step S108, the size information generation unit 124 generates projection size information 111 based on the size setting information 101, the calculation parameters 103, and the internal parameters 114. In this embodiment, the size information generation unit 124 generates first size information 112 based on the size setting information 101, the calculation parameters 103, and the internal parameters 114.

[0084] In step S109, the image generation unit 125 updates the projection image information. That is, the image generation unit 125 generates the second projection image information 107 based on the first projection image information 106 and the projection size information 111.

[0085] In step S110, the projection control unit 121 controls the projection mechanism 16 to project projection light for displaying the projection image GP onto the wall surface W1. In other words, the projection control unit 121 causes the projection mechanism 16 to project projection light based on the second projection image information 107, thereby displaying the projection image GP on the wall surface W1. Specifically, the projection control unit 121 causes the projection mechanism 16 to project projection light based on the second projection image information 107, thereby displaying the projection image GP2 on the wall surface W1.

[0086] In step S111, the input management unit 122 determines whether or not an operation to end the operation has been received from the user based on the operation data. If an operation to end the operation has been received from the user, that is, if YES in step S111, the processing device 12 including the input management unit 122 ends the series of operations shown in the flowchart of Fig. 8. If an operation to end the operation has not been received from the user, that is, if NO in step S111, the input management unit 122 advances the process to step S104.

[0087] As described above, according to the first embodiment, the projector 1 displays the size of the projection image GP displayed on the projection surface. That is, the user can easily grasp the size of the projection image GP displayed on the projection surface.

[0088] Furthermore, according to the first embodiment, the projector 1 displays an image having a plurality of vertices, for example, the image GP11 and the image GP12. That is, the image indicated by the captured image information 102 includes an image having a plurality of vertices, for example, the image objects OV1 to OV4 and the image GV12.

[0089] When detecting points from an image by image processing, it is generally preferable that the image contains points whose positions can be accurately determined. Specifically, it is preferable that the image contains points where the amount of change in luminance value, etc. between adjacent pixels is large in both the vertical direction and the horizontal direction of the image. For example, a corner-shaped portion included in the image is an example of a portion where the amount of change in luminance value, etc. between adjacent pixels is large in both the vertical direction and the horizontal direction of the image. Since the image represented by the captured image information 102 contains multiple vertices, i.e., multiple corner-shaped portions, the projector 1 can accurately detect points from the image represented by the captured image information 102.

[0090] As described above, the projection method according to the first embodiment includes generating projection size information 111 including at least one of first size information 112 indicating the distance between two points included in the projection image GP1 displayed on the wall surface W1 and second size information 113 indicating the area of ​​the first image included in the projection image GP1, based on the captured image information 102 output from the imaging element 142, and displaying the projection image GP2 including the number Q11 representing the numerical value indicated by the projection size information 111 on the wall surface W1.

[0091] Furthermore, the projector 1 according to the first embodiment includes a processing device 12, an image sensor 142, and a projection mechanism 16. The processing device 12 generates projection size information 111 based on the captured image information 102 output from the image sensor 142, the projection size information 111 including at least one of first size information 112 indicating the distance between two points included in the projection image GP1 displayed on the wall surface W1 and second size information 113 indicating the area of ​​the first image included in the projection image GP1, and controls the projection mechanism 16 to display the projection image GP2 including a number Q11 representing the numerical value indicated by the projection size information 111 on the wall surface W1.

[0092] That is, the projector 1 displays a number Q11 indicating the size of the projected image GP1 displayed on the wall surface W1, which serves as the projection surface, on the wall surface W1. This allows the user to easily grasp the size of the projected image GP1 displayed on the wall surface W1. Therefore, the user can easily adjust the size of the projected image GP to a desired size by changing the position of the projector 1 while checking the number Q11 displayed on the wall surface W1.

[0093] In the first embodiment, the projector 1 is an example of a "projection system", the image sensor 142 is an example of a "sensor", the captured image information 102 is an example of "information", the wall surface W1 is an example of a "projection surface", the projected image GP1 is an example of a "first projected image", the first size information 112 is an example of a "first size information", the second size information 113 is an example of a "second size information", the projected size information 111 is an example of a "projection size information", the number Q11 is an example of an "image representing a numerical value indicated by the projection size information", the projected image GP2 is an example of a "second projected image", the processing device 12 is an example of a "processing device", and the projection mechanism 16 is an example of a "projection mechanism". In addition, the "distance between two points" is an example of the distance between the points DP5 and DP6. In addition, the "first image" is an example of the image GP11 and the image GP12.

[0094] Furthermore, in the projection method according to the first embodiment, the projection size information 111 includes first size information 112, which indicates the length of the diagonal of the projection image GP1, and the projection image GP2 includes a character string Q12 indicating that the numerical value indicated by the projection size information 111 is displayed in inches.

[0095] For example, the size of an image display area in a display device such as a television is compared based on the length of the diagonal. That is, by displaying the length of the diagonal, the user can easily compare the size of the projected image GP with the size of the display device. Furthermore, the length of the diagonal of the display device is generally displayed in inches. That is, by notifying the user that the unit used to compare the size of the image display area of ​​the display device matches the unit used to express the size of the projected image GP, the user can more easily compare the size of the projected image GP with the size of the display device.

[0096] In the first embodiment, the inch unit is an example of a "first unit", and the character string Q12 is an example of a "character string".

[0097] Furthermore, in the projection method of the first embodiment, the projection image GP1 is displayed by the projection mechanism 16 projecting the image GF1 onto the wall surface W1, the imaging element 142 is an image sensor provided in the camera 14 that captures the projection image GP1, and generating the projection size information 111 includes obtaining captured image information 102 output from the imaging element 142 by capturing the projection image GP1, and generating the projection size information 111 based on position information 104 indicating the positional relationship between the camera 14 and the projection mechanism 16, first projection image information 106 representing the image GF1, and the captured image information 102.

[0098] That is, the projector 1 can calculate the size of the projection image GP1 without using a special device such as a ToF sensor. This allows the projector 1 to display an image that represents the size of the projection image GP at a lower cost than when a special device such as a ToF sensor is used. Here, ToF is an abbreviation for Time of Flight.

[0099] In the first embodiment, image GF1 is an example of a "second image," camera 14 is an example of a "camera," position information 104 is an example of "position information," and first projection image information 106 is an example of "second image information."

[0100] In the projection method according to the first embodiment, the camera 14 and the projection mechanism 16 are provided in a housing 18, and the housing 18 includes a storage device 10 that stores position information 104.

[0101] That is, the position information 104 is information specific to the device and is constant. In addition, the position information 104 is known in advance. This allows the user to omit the task of acquiring the position information 104 that is necessary when calculating the size of the projection image GP.

[0102] In the first embodiment, the housing 18 is an example of a "housing", and the storage device 10 is an example of a "storage device".

[0103] Furthermore, in the projection method of the first embodiment, generating projection size information 111 includes acquiring first coordinate information 109 indicating the coordinates of points DF1 to DF4 included in image GF1, and acquiring second coordinate information 110 indicating the coordinates of points DV1 to DV4 which correspond one-to-one to points DF1 to DF4 and are included in the captured image GS1 indicated by the captured image information 102, and generating the projection size information 111 based on the position information 104, the first projection image information 106, and the captured image information 102 includes generating the projection size information 111 based on the position information 104, the first coordinate information 109, and the second coordinate information 110.

[0104] That is, the projector 1 calculates the size of the projected image GP based on the correspondence between a plurality of points included in the image GF1 and a plurality of points included in the captured image GS1. This allows the projector 1 to accurately calculate the size of the projected image GP.

[0105] In the first embodiment, points DF1 to DF4 are an example of "plurality of first points", first coordinate information 109 is an example of "first coordinate information", captured image GS1 is an example of an "captured image", points DV1 to DV4 are an example of "plurality of second points", and second coordinate information 110 is an example of "second coordinate information".

[0106] In the projection method according to the first embodiment, the image GF1 includes a third image having a plurality of vertices.

[0107] That is, the projector 1 projects the image GF1 to display an image having multiple vertices, for example, the image GP11 and the image GP12, on the wall surface W1. As a result, the captured image GS1 includes images having multiple vertices with high detection accuracy in image processing, for example, the image objects OV1 to OV4 and the image GV12. Therefore, the projector 1 can accurately detect points located at the multiple vertices included in the captured image GS1. As a result, the projector 1 can more accurately calculate the size of the projected image GP.

[0108] In the first embodiment, the "third image" is exemplified by image GF11 and image GF12.

[0109] 2. Variations The above embodiment may be modified in various ways. Specific modified aspects are exemplified below. Two or more aspects arbitrarily selected from the following examples may be appropriately combined within a range that does not contradict each other. In the modified examples exemplified below, the elements whose actions and functions are equivalent to those of the above embodiment will be appropriately omitted by using the symbols used in the above explanation.

[0110] 2.1. Variation 1 In the above embodiment, a case where one set of character strings representing the size of the projection image GP is displayed is illustrated, but the present invention is not limited to this embodiment. Two or more sets of character strings representing the size of the projection image GP may be displayed at the same time. For example, a character string including numbers representing the numerical value indicated by the first size information 112 and a character string including numbers representing the numerical value indicated by the second size information 113 may be displayed at the same time.

[0111] Fig. 9 is a schematic diagram showing how the projected image GP3 is displayed, and Fig. 10 is a schematic diagram showing how the projected image GP4 is displayed.

[0112] The projector 1 controls the projection mechanism 16 to display a projection image GP3 on the wall surface W1.

[0113] The projected image GP3 includes an image GP31. The image GP31 includes an area having a pattern in which a plurality of black squares and a plurality of white squares are alternately arranged, and a plain area located near the center of the image.

[0114] Furthermore, the projected image GP3 includes points DP7, DP8, DP9, and DP10. Point DP7 is located at the top right vertex as seen by the user among the multiple vertices of image GP31. Point DP8 is located at the bottom right vertex as seen by the user among the multiple vertices of image GP31. Point DP9 is located at the bottom left vertex as seen by the user among the multiple vertices of image GP31. Point DP10 is located at the top left vertex as seen by the user among the multiple vertices of image GP31.

[0115] The camera 14 captures an image of the imaging area E1 including the projection image GP3 to obtain a captured image. The projector 1 calculates the size of the projection image GP3 displayed on the wall surface W1 based on the captured image obtained by the camera 14. In this modification, the projector 1 calculates the distance between the points DP7 and DP8 as the size of the projection image GP3. The projector 1 also calculates the distance between the points DP7 and DP10 as the size of the projection image GP3. The projector 1 also calculates the area of ​​the image GP31 as the size of the projection image GP3.

[0116] When the distance between points DP7 and DP8, the distance between points DP7 and DP10, and the area of ​​image GP31 are calculated as the sizes of projected image GP3, the projector 1 controls the projection mechanism 16 to display projected image GP4 on wall surface W1.

[0117] The projected image GP4 is an image in which the character strings Q3, Q4, and Q5 are superimposed on the projected image GP3. Various dimensions of the projected image GP4 are the same as those of the projected image GP3.

[0118] The character string Q3 includes a number Q31 and a character string Q32. The number Q31 is an image that represents the size of the projected image GP3. Specifically, the number Q31 is an image that includes the number "2.0" that represents the distance between the points DP7 and DP10. More specifically, the number Q31 is an image that includes a number that displays the distance between the points DP7 and DP10 in meters. The character string Q32 is a character string that represents the unit of the size of the projected image GP3. Specifically, the character string Q32 is a character string "m" that represents that the distance between the points DP7 and DP10 is displayed in meters.

[0119] The character string Q4 includes a number Q41 and a character string Q42. The number Q41 is an image that represents the size of the projected image GP3. Specifically, the number Q41 is an image that includes the number "1.5" that represents the distance between the points DP7 and DP8. More specifically, the number Q41 is an image that includes a number that displays the distance between the points DP7 and DP8 in meters. The character string Q42 is a character string that represents the unit of the size of the projected image GP3. Specifically, the character string Q42 is the character string "m" that represents that the distance between the points DP7 and DP8 is displayed in meters.

[0120] The character string Q5 includes a number Q51 and a character string Q52. The number Q51 is an image that indicates the size of the projected image GP3. Specifically, the number Q51 is an image that includes the number "3.0" that indicates the area of ​​the image GP31. More specifically, the number Q51 is an image that includes a number that displays the area of ​​the image GP31 in square meters. The character string Q52 is a character string that indicates the unit of the size of the projected image GP3. Specifically, the character string Q52 includes "m 2 " is the string.

[0121] By checking the character string Q3, the user can ascertain that the distance between points DP7 and DP10 is 2.0 meters. Furthermore, by checking the character string Q4, the user can ascertain that the distance between points DP7 and DP8 is 1.5 meters. Furthermore, by checking the character string Q5, the user can ascertain that the area of ​​the image GP31 is 3.0 square meters. Furthermore, the user can easily adjust the size of the projected image GP to a desired size by changing the position of the projector 1 while checking the numbers indicating the size of the projected image GP displayed on the wall W1.

[0122] The character string representing the size of the projection image GP is not limited to one including Arabic numerals and Roman letters. For example, Chinese numerals or Roman numerals may be displayed as the numerals representing the size of the projection image GP. Also, a character string including Chinese characters or Hangul may be displayed as the character string representing the unit of the size of the projection image GP.

[0123] 2.2. Variation 2 In the above-described embodiment and modified example, the case where camera 14 is provided in housing 18 has been exemplified, but the present invention is not limited to this aspect. For example, camera 14 may be fixed to a location other than housing 18. When the positional relationship between camera 14 and projection mechanism 16 is changed, position information 104 needs to be updated according to the situation.

[0124] 2.3. Variation 3 In the above-described embodiment and modified example, the projection size information 111 is generated based on the captured image information 102 output from the image sensor 142 included in the camera 14, but the present invention is not limited to such an embodiment. For example, the projection size information 111 may be generated using a ToF sensor instead of the camera 14. The ToF sensor outputs information on the distance from the ToF sensor to an object for each pixel. That is, the coordinates of multiple points included in the projection image GP are derived based on the information output from the ToF sensor. Furthermore, the projection size information 111 is generated based on the coordinates of the multiple points. In this case, the ToF sensor is an example of a "sensor".

[0125] 3. Notes The present invention will be summarized below as an appendix.

[0126] Appendix 1 A projection method comprising: generating projection size information including at least one of first size information indicating a distance between two points included in a first projection image displayed on a projection surface, based on information output from a sensor, and second size information indicating an area of ​​a first image included in the first projection image; and displaying a second projection image on the projection surface, the second projection image including an image showing a numerical value indicated by the projection size information.

[0127] That is, the projection system realizing the projection method described in Supplementary Note 1 displays an image showing a numerical value indicated by the projection size information on the projection surface. This allows the user to easily grasp the size of the first projection image displayed on the projection surface. Therefore, the user can easily adjust the size of the projection image displayed on the projection surface to a desired size by changing the position of the projection device included in the projection system realizing the projection method described in Supplementary Note 1 while checking the image displayed on the projection surface showing the numerical value indicated by the projection size information.

[0128] Appendix 2 The projection method described in Appendix 1, wherein the projection size information includes the first size information, the first size information indicating a diagonal length of the first projection image, and the second projection image includes a character string indicating that the numerical value indicated by the projection size information is displayed in a first unit.

[0129] For example, the size of an image display area in a display device such as a television is compared based on the length of the diagonal. That is, by displaying the length of the diagonal, the user can easily compare the size of the first projected image with the size of the display device. Also, the length of the diagonal of the display device is generally displayed in inches. That is, by notifying the user that the unit used to compare the size of the image display area of ​​the display device matches the unit used to express the size of the first projected image, the user can more easily compare the size of the first projected image with the size of the display device.

[0130] Appendix 3 The projection method described in Appendix 1 or 2, wherein the first projection image is displayed by a projection mechanism projecting a second image onto the projection surface, the sensor is an image sensor provided in a camera that captures the first projection image, and generating the projection size information includes obtaining captured image information output from the sensor by capturing the first projection image, and generating the projection size information based on position information indicating a positional relationship between the camera and the projection mechanism, second image information representing the second image, and the captured image information.

[0131] That is, the projection system that realizes the projection method described in Supplementary Note 3 can generate projection size information without using a special device such as a ToF sensor. As a result, the projection system that realizes the projection method described in Supplementary Note 3 can display an image that represents a numerical value indicated by the projection size information at a lower cost than when a special device such as a ToF sensor is used.

[0132] Appendix 4 The projection method according to claim 3, wherein the camera and the projection mechanism are provided in a housing, and the housing includes a storage device that stores the position information.

[0133] That is, the position information 104 is information specific to the device and is constant. Also, the position information 104 is known in advance. This allows the user to omit the task of acquiring the position information 104 that is required when generating projection size information.

[0134] Appendix 5 The projection method described in Appendix 3 or 4, wherein generating the projection size information includes acquiring first coordinate information indicating coordinates of a plurality of first points included in the second image, and acquiring second coordinate information indicating coordinates of a plurality of second points included in the captured image indicated by the captured image information and having one-to-one correspondence with the plurality of first points, and generating the projection size information based on the position information, the second image information, and the captured image information includes generating the projection size information based on the position information, the first coordinate information, and the second coordinate information.

[0135] That is, the projection system realizing the projection method described in Supplementary Note 5 generates projection size information based on the correspondence between a plurality of points included in the second image and a plurality of points included in the captured image. This allows the projection system realizing the projection method described in Supplementary Note 5 to generate highly accurate projection size information.

[0136] Appendix 6 the second image includes a third image having a plurality of vertices; 6. The projection method according to claim 5.

[0137] That is, the projection system realizing the projection method described in Supplementary Note 6 projects the second image to display an image having multiple vertices, i.e., an image corresponding to the third image, on the projection surface. As a result, the captured image includes an image having multiple vertices. Therefore, the projection system realizing the projection method described in Supplementary Note 6 can accurately detect points located at the multiple vertices included in the captured image. As a result, the projection system realizing the projection method described in Supplementary Note 6 can generate more accurate projection size information.

[0138] Appendix 7 A projection system comprising a processing device, a sensor, and a projection mechanism, wherein the processing device generates projection size information based on information output from the sensor, the projection size information including at least one of first size information indicating a distance between two points included in a first projection image displayed on a projection surface, and second size information indicating an area of ​​a first image included in the first projection image, and controls the projection mechanism to display a second projection image on the projection surface, the second projection image including an image representing a numerical value indicated by the projection size information.

[0139] That is, the projection system described in Supplementary Note 7 displays an image showing a numerical value indicated by the projection size information on the projection surface. This allows the user to easily grasp the size of the first projection image displayed on the projection surface. Therefore, the user can easily adjust the size of the projection image displayed on the projection surface to a desired size by changing the position of the projection device included in the projection system described in Supplementary Note 7 while checking the image displayed on the projection surface showing the numerical value indicated by the projection size information. [Explanation of symbols]

[0140] 1...projector, 10...storage device, 12...processing device, 14...camera, 16...projection mechanism, 18...housing, 20...operation device, 22...communication device, 100...program, 101...size setting information, 102...captured image information, 103...calculated parameters, 104...position information, 105...projected image information, 106...first projected image information, 107...second projected image information, 108...coordinate information, 109...first coordinate information, 110...second coordinate information, 111...projection size information, 112...first size information, 113...second size information, 114...internal parameters, 115...projection parameters, 116...camera parameters, 120...acquisition unit, 121...projection control unit, 122...input management unit, 123...detection unit, 124...size information generation unit, 125...image generation unit, 140...imaging lens, 142...imaging element, GP1...projected image, GF1...image, GS1...captured image, DP1...point, OP1...image object, Q1...character string, Q11...number, Q12...character string, E1...imaging area, W1...wall surface.

Claims

1. generating projection size information including at least one of first size information indicating a distance between two points included in a first projection image displayed on a projection surface and second size information indicating an area of ​​a first image included in the first projection image based on information output from the sensor; displaying a second projection image on the projection surface, the second projection image including an image showing a numerical value indicated by the projection size information; Including, Projection method.

2. the projection size information includes the first size information, the first size information indicating a length of a diagonal of the first projected image, the second projection image includes a character string indicating that a numerical value indicated by the projection size information is displayed in a first unit; The projection method according to claim 1 .

3. the first projection image is displayed by a projection mechanism projecting a second image onto the projection surface; the sensor is an image sensor included in a camera that captures the first projection image, Generating the projection size information includes: acquiring captured image information output from the sensor by capturing the first projection image; generating the projection size information based on position information indicating a positional relationship between the camera and the projection mechanism, second image information representing the second image, and the captured image information; Including, The projection method according to claim 1 or 2.

4. the camera and the projection mechanism are provided in a housing, The housing includes a storage device that stores the position information. The projection method according to claim 3 .

5. Generating the projection size information includes: acquiring first coordinate information indicating coordinates of a plurality of first points included in the second image; acquiring second coordinate information indicating coordinates of a plurality of second points that correspond one-to-one to the plurality of first points and are included in a captured image indicated by the captured image information; Including, generating the projection size information based on the position information, the second image information, and the captured image information includes generating the projection size information based on the position information, the first coordinate information, and the second coordinate information; The projection method according to claim 3 .

6. the second image includes a third image having a plurality of vertices; The projection method according to claim 5 .

7. A processing device; A sensor, A projection mechanism; Equipped with The processing device includes: generating projection size information including at least one of first size information indicating a distance between two points included in a first projection image displayed on a projection surface and second size information indicating an area of ​​a first image included in the first projection image based on information output from the sensor; displaying a second projection image on the projection surface, the second projection image including an image representing a numerical value indicated by the projection size information, by controlling the projection mechanism; Execute Projection system.

Images