Information processing device, information processing method, and information processing program
By generating image data that can be set in the range of display projection devices, the user can help select an appropriate projection device model, solve the problem of inconvenient setting of projection devices in the prior art, and improve the user experience.
Patent Information
- Application Number
- CN202380029966.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-03-25
- Filing Date
- 2023-03-03
- Publication Date
- 2024-11-08
AI Technical Summary
The prior art is difficult to effectively assist in the selection of the model of the projection device, resulting in users facing difficulties and inconveniences when setting up the projection device.
By acquiring the image data captured by the imaging device and the data of the imaginary projection surface and the projection device, image data that can be set in the display projection device can be generated to help the user visually compare and select an appropriate projection device model.
It improves the convenience of the user when selecting the projection device model, simplifies the setting process of the projection device, and reduces the complexity of user operations.
Smart Images

Figure CN118923105A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an information processing device, an information processing method and an information processing program. Background Art
[0002] Patent document 1 describes a projection image adjustment system, the subject of which is to facilitate the installation and adjustment of a projection type display device. Patent document 2 describes a projection device, the subject of which is to reduce the trouble of setting up an image projection device and projecting an image. Patent document 3 describes an information processing device for assisting the installation of a projector, the subject of which is to determine the installation conditions of a projector for installing a depicted object and projecting a projection image related to the depicted object based on projection image data.
[0003] Previous technical literature
[0004] Patent Literature
[0005] Patent Document 1: Japanese Patent Application Publication No. 2018-005115
[0006] Patent Document 2: Japanese Patent Application Publication No. 2020-036120
[0007] Patent Document 3: Japanese Patent Application Publication No. 2019-071567 Summary of the invention
[0008] One embodiment of the technology according to the present invention provides an information processing device, an information processing method, and an information processing program capable of improving the convenience of a user in selecting a model of a projector to be installed.
[0009] Means for solving technical problems
[0010] An information processing device of one aspect of the present invention comprises a processor, wherein the processor performs the following processing: obtaining first image data representing a first image captured by an imaging device, first imaginary projection surface data related to a first imaginary projection surface, first imaginary projection device data related to a first imaginary projection device corresponding to the first imaginary projection surface, and second imaginary projection device data related to a second imaginary projection device; generating second image data representing a second image in which the first imaginary projection surface, the first imaginary projection device or the settable range of the first imaginary projection device, and the second imaginary projection device or the settable range of the second imaginary projection device are displayed on the first image based on the first imaginary projection surface data, the first imaginary projection device data, and the second imaginary projection device data; and outputting the second image data to an output destination.
[0011] In an information processing method of one aspect of the present invention, a processor of an information processing device performs the following processing: acquiring first image data representing a first image captured by an imaging device, first imaginary projection surface data related to a first imaginary projection surface, first imaginary projection device data related to a first imaginary projection device corresponding to the first imaginary projection surface, and second imaginary projection device data related to a second imaginary projection device; generating second image data representing a second image in which the first imaginary projection surface, the first imaginary projection device or the settable range of the first imaginary projection device, and the second imaginary projection device or the settable range of the second imaginary projection device are displayed on the first image based on the first imaginary projection surface data, the first imaginary projection device data, and the second imaginary projection device data; and outputting the second image data to an output destination.
[0012] An information processing program of one aspect of the present invention is used to cause a processor of an information processing device to perform the following processing: obtaining first image data representing a first image captured by a camera device, first imaginary projection surface data related to a first imaginary projection surface, first imaginary projection device data related to a first imaginary projection device corresponding to the first imaginary projection surface, and second imaginary projection device data related to a second imaginary projection device; generating second image data representing a second image in which the first imaginary projection surface, the first imaginary projection device or the settable range of the first imaginary projection device, and the second imaginary projection device or the settable range of the second imaginary projection device are displayed on the first image based on the first imaginary projection surface data, the first imaginary projection device data, and the second imaginary projection device data; and outputting the second image data to an output destination.
[0013] Effects of the Invention
[0014] According to the present invention, it is possible to provide an information processing device, an information processing method, and an information processing program that can improve the convenience of a user in selecting a model of a projector to be installed. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 1 is a schematic diagram showing an example of a projection device 10 that is a setting assistance object of the information processing device according to the embodiment.
[0016] Figure 2 Yes means Figure 1 Schematic diagram of an example of the internal structure of the projection unit 1 shown.
[0017] Figure 3 1 is a schematic diagram showing the external appearance of the projection device 10 .
[0018] Figure 4 yes Figure 3 FIG. 1 is a schematic cross-sectional view of the optical unit 106 of the projection device 10 shown.
[0019] Figure 5 This is a diagram showing an example of an information processing device 50 according to the embodiment.
[0020] Figure 6 This is a diagram showing an example of the hardware configuration of the information processing device 50 according to the embodiment.
[0021] Figure 7 This is a flowchart showing an example of processing of the information processing device 50 in the screen priority mode.
[0022] Figure 8 is Figure 7 An example (part 1) of an image displayed by the information processing device 50 during the process shown.
[0023] Fig. 9 is Figure 7 An example of an image displayed by the information processing device 50 during the process shown (part 2).
[0024] Fig.10 is Figure 7 An example of an image displayed by the information processing device 50 during the process shown (part 3).
[0025] Fig.11 is Figure 7 An example of an image displayed by the information processing device 50 during the process shown (part 4).
[0026] Fig.12 is Figure 7 An example of an image displayed by the information processing device 50 during the process shown (part 5).
[0027] Fig.13 is Figure 7 An example of an image displayed by the information processing device 50 during the process shown (part 6).
[0028] Fig.14 This is a flowchart showing an example of the processing of the information processing device 50 in the projector priority first mode.
[0029] Fig.15 is Fig.14 An example (part 1) of an image displayed by the information processing device 50 during the process shown.
[0030] Fig.16 is Fig.14 An example of an image displayed by the information processing device 50 during the process shown (part 2).
[0031] Fig.17 is Fig.14 An example of an image displayed by the information processing device 50 during the process shown (part 3).
[0032] Fig.18 is Fig.14 An example of an image displayed by the information processing device 50 during the process shown (part 4).
[0033] Fig.19 is Fig.14 An example of an image displayed by the information processing device 50 during the process shown (part 5).
[0034] Fig. 20 This is a flowchart (part 1) showing an example of the processing of the information processing device 50 in the projector priority second mode.
[0035] Fig.21 This is a flowchart (part 2) showing an example of the processing of the information processing device 50 in the projector priority second mode.
[0036] Fig. 22 is Fig. 20 , Fig.21 An example (part 1) of an image displayed by the information processing device 50 during the process shown.
[0037] Fig.23 is Fig. 20 , Fig.21 An example of an image displayed by the information processing device 50 during the process shown (part 2).
[0038] Fig.24 is Fig. 20 , Fig.21 An example of an image displayed by the information processing device 50 during the process shown (part 3).
[0039] Fig.25 is Fig. 20 , Fig.21 An example of an image displayed by the information processing device 50 during the process shown (part 4).
[0040] Fig.26 is Fig. 20 , Fig.21 An example of an image displayed by the information processing device 50 during the process shown (part 5).
[0041] Fig. 27 This is a flowchart showing an example of the processing of the information processing device 50 in the projector priority third mode.
[0042] Fig.28 Yes means Fig. 27 Flowchart of an example of calculation in step S277.
[0043] Fig.29 is Fig.28 In the example Fig. 27 An example of an image displayed by the information processing device 50 during processing.
[0044] Fig.30 Yes means Fig. 27 Flowchart of another example of the calculation in step S277.
[0045] Fig.31 is Fig.30 In the example Fig. 27 An example of an image displayed by the information processing device 50 during processing (part 1).
[0046] Fig.32 is Fig.30 In the example Fig. 27 An example of an image displayed by the information processing device 50 during processing (part 2).
[0047] Fig.33 This is a flowchart showing an example of processing of the information processing device 50 in the model number search mode.
[0048] Fig.34 is Fig.33 An example (part 1) of an image displayed by the information processing device 50 during the process shown.
[0049] Fig.35 is Fig.33 An example of an image displayed by the information processing device 50 during the process shown (part 2).
[0050] Fig.36 is Fig.33 An example of an image displayed by the information processing device 50 during the process shown (part 3).
[0051] Fig.37 is Fig.33 An example of an image displayed by the information processing device 50 during the process shown (part 4).
[0052] Fig.38 is Fig.33 An example of an image displayed by the information processing device 50 during the process shown (part 5). DETAILED DESCRIPTION
[0053] Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.
[0054] <An example of the projection device 10 to be changed in the configuration of the information processing device according to the embodiment>
[0055] Figure 11 is a schematic diagram showing an example of a projection device 10 that is a setting assistance object of the information processing device according to the embodiment.
[0056] The information processing device of the embodiment can be used, for example, to assist in the configuration of a projection device 10. The projection device 10 includes a projection unit 1, a control device 4, and an operation receiving unit 2. The projection unit 1 is composed of, for example, a liquid crystal projector or a projector using LCOS (Liquid Crystal On Silicon). In the following, the projection unit 1 is described as a liquid crystal projector.
[0057] The control device 4 is a control device for controlling the projection of the projection device 10. The control device 4 includes a control unit composed of various processors, a communication interface (not shown) for communicating with each unit, and a memory 4a such as a hard disk, SSD (Solid State Drive) or ROM (Read Only Memory), and centrally controls the projection unit 1.
[0058] Various processors serving as the control unit of the control device 4 include a general-purpose processor that executes programs to perform various processes, namely a CPU (Central Processing Unit), a processor whose circuit structure can be changed after manufacturing, such as an FPGA (Field Programmable Gate Array), a programmable logic device (PLD), or an ASIC (Application Specific Integrated Circuit), a processor with a circuit structure specially designed to perform specific processing, such as a dedicated circuit.
[0059] More specifically, the structure of these various processors is a circuit formed by combining circuit elements such as semiconductor elements. The control unit of the control device 4 can be composed of one of the various processors, or a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA).
[0060] The operation receiving unit 2 detects instructions from the user by receiving various operations from the user. The operation receiving unit 2 can be a button, a key, a joystick, etc. provided on the control device 4, or a receiving unit for receiving signals from a remote controller for remotely operating the control device 4.
[0061] The projection object 6 is an object such as a screen or a wall having a projection surface on which the projection image is displayed by the projection unit 1. Figure 1In the example shown, regarding the projection object 6 , the projection surface of the projection object 6 is a rectangular plane. Figure 1 The up, down, left, and right directions of the object to be projected 6 in FIG. 6 are set to be the actual up, down, left, and right directions of the object to be projected 6 .
[0062] The projection range 11 indicated by the single-dot chain line is a region of the projection object 6 to which the projection light is irradiated by the projection unit 1. Figure 1 In the example shown, the projection range 11 is a rectangle. The projection range 11 is a part or the whole of the projectable range that can be projected by the projection unit 1.
[0063] In addition, the projection unit 1, the control device 4 and the operation receiving unit 2 are implemented by, for example, one device (for example, refer to Figure 3 , Figure 4 ). Alternatively, the projection unit 1, the control device 4, and the operation receiving unit 2 may be different devices that cooperate by communicating with each other.
[0064] < Figure 1 The internal structure of the projection unit 1 shown>
[0065] Figure 2 Yes means Figure 1 Schematic diagram of an example of the internal structure of the projection unit 1 shown.
[0066] like Figure 2 As shown, the projection unit 1 includes a light source 21 , a light modulation unit 22 , a projection optical system 23 , and a control circuit 24 .
[0067] The light source 21 includes a light emitting element such as a laser or an LED (Light Emitting Diode), and emits, for example, white light.
[0068] The light modulator 22 is composed of three liquid crystal panels, and the liquid crystal panels modulate the light emitted from the light source 21 and separated into three colors of red, blue, and green by a color separation mechanism (not shown) according to image information to emit images of each color. Red, blue, and green filters can be mounted on the three liquid crystal panels, respectively, and each liquid crystal panel modulates the white light emitted from the light source 21 to emit images of each color.
[0069] The projection optical system 23 receives light from the light source 21 and the light modulator 22 and is composed of, for example, a relay optical system including at least one lens. The light passing through the projection optical system 23 is projected onto the projection object 6 .
[0070] In the projection object 6, the area irradiated by the light of the entire range of the light modulating unit 22 becomes the projectable range that can be projected by the projection unit 1. In the projectable range, the area irradiated by the light actually transmitted from the light modulating unit 22 becomes the projection range 11. For example, by controlling the size, position and shape of the area through which the light is transmitted in the light modulating unit 22, the size, position and shape of the projection range 11 are changed within the projectable range.
[0071] The control circuit 24 controls the light source 21, the light modulator 22 and the projection optical system 23 according to the display data input from the control device 4, thereby projecting an image based on the display data onto the projection object 6. The display data input to the control circuit 24 is composed of three: red display data, blue display data and green display data.
[0072] Furthermore, the control circuit 24 changes the projection optical system 23 according to the command input from the control device 4, thereby controlling the projection range 11 (reference Figure 1 Furthermore, the control device 4 can change the projection optical system 23 according to the operation from the user received by the operation receiving unit 2, thereby moving the projection range 11 of the projection unit 1.
[0073] Furthermore, the projection device 10 has a displacement mechanism that mechanically or optically moves the projection range 11 while maintaining the image circle of the projection optical system 23. The image circle of the projection optical system 23 is a region where the projection light incident on the projection optical system 23 appropriately passes through the projection optical system 23 from points such as light quantity reduction, color separation, and peripheral bending.
[0074] The shift mechanism is realized by at least one of an optical system shift mechanism that performs optical system shift and an electronic shift mechanism that performs electronic shift.
[0075] The optical system displacement mechanism is, for example, a mechanism for moving the projection optical system 23 in a direction perpendicular to the optical axis (for example, see Figure 3 , Figure 4 ), or a mechanism that moves the light modulator 22 in a direction perpendicular to the optical axis instead of moving the projection optical system 23. Furthermore, the optical system displacement mechanism may combine the movement of the projection optical system 23 and the movement of the light modulator 22.
[0076] The electronic displacement mechanism is a mechanism for performing a displacement of the simulated projection range 11 by changing the light transmission range in the light modulation unit 22 .
[0077] Furthermore, the projection device 10 may include a projection direction changing mechanism for moving the projection range 11 together with the image circle of the projection optical system 23. The projection direction changing mechanism is a mechanism for changing the projection direction of the projection unit 1 by mechanically rotating and changing the orientation of the projection unit 1 (for example, see Figure 3 , Figure 4 ).
[0078] <Mechanical Structure of Projection Device 10>
[0079] Figure 3 1 is a schematic diagram showing the external appearance of the projection device 10 . Figure 4 yes Figure 3 FIG. 1 is a schematic cross-sectional view of the optical unit 106 of the projection device 10 shown. Figure 4 Indicates that along Figure 3 The cross section on the plane of the optical path of light emitted from the main body 101 is shown.
[0080] like Figure 3 As shown in FIG. 1 , the projection device 10 includes a main body 101 and an optical unit 106 protruding from the main body 101. Figure 3 In the illustrated structure, the operation receiving unit 2, the control device 4, the light source 21, the light modulation unit 22 and the control circuit 24 of the projection unit 1 are provided in the main body 101. The projection optical system 23 of the projection unit 1 is provided in the optical unit 106.
[0081] The optical unit 106 includes a first member 102 supported by the main body 101 and a second member 103 supported by the first member 102 .
[0082] In addition, the first member 102 and the second member 103 may be an integrated member. The optical unit 106 may be configured to be detachable from the main body 101 (in other words, configured to be replaceable).
[0083] The main body 101 has an opening 15a (see FIG. 15A ) for passing light formed in a portion connected to the optical unit 106. Figure 4 ) frame 15 (reference Figure 4 ).
[0084] like Figure 3 As shown, a light source 21 and a light modulation unit 12 are provided inside the frame 15 of the main body 101. The light modulation unit 12 includes a light modulation unit 22 (see FIG. 1 ) for spatially modulating light emitted from the light source 21 according to input image data to generate an image. Figure 2 ).
[0085] The light emitted from the light source 21 enters the light modulation section 22 of the light modulation unit 12 , is spatially modulated by the light modulation section 22 , and is emitted.
[0086] like Figure 4 As shown, an image formed by the light spatially modulated by the light modulation unit 12 enters the optical unit 106 through the opening 15 a of the frame 15 and is projected onto the projection object 6 as the projection target, so that the observer visually recognizes the image G1 .
[0087] like Figure 4 As shown, the optical unit 106 comprises: a first component 102 having a hollow portion 2A connected to the interior of a main body 101; a second component 103 having a hollow portion 3A connected to the hollow portion 2A; a first optical system 121 and a reflecting component 122 arranged in the hollow portion 2A; a second optical system 31, a reflecting component 32, a third optical system 33 and a lens 34 arranged in the hollow portion 3A; a displacement mechanism 105; and a projection direction changing mechanism 104.
[0088] The first member 102 is a member having a rectangular cross-sectional shape, and the opening 2a and the opening 2b are formed on mutually perpendicular planes. The first member 102 is supported by the main body 101 in a state where the opening 2a is arranged at a position of the main body 101 opposite to the opening 15a. Light emitted from the light modulation unit 22 of the light modulation unit 12 of the main body 101 passes through the opening 15a and the opening 2a and enters the hollow portion 2A of the first member 102.
[0089] The incident direction of light entering the hollow portion 2A from the main body 101 is recorded as direction X1, the opposite direction of direction X1 is recorded as direction X2, and the directions X1 and X2 are collectively recorded as direction X. Figure 4 In the diagram, a direction from the front of the paper toward the inside and the opposite direction are described as directions Z. In the directions Z, a direction from the front of the paper toward the inside is described as direction Z1, and a direction from the inside of the paper toward the front is described as direction Z2.
[0090] Furthermore, the direction perpendicular to the direction X and the direction Z is recorded as the direction Y. In the direction Y, Figure 4 The upward direction is recorded as direction Y1, and Figure 4 The downward direction is recorded as direction Y2. Figure 4 In the example of FIG. 1 , the projection device 10 is arranged so that the direction Y2 becomes a vertical direction.
[0091] Figure 2 The projection optical system 23 shown is composed of a first optical system 121 , a reflecting member 122 , a second optical system 31 , a reflecting member 32 , a third optical system 33 , and a lens 34 . Figure 42 shows the optical axis K of the projection optical system 23. The first optical system 121, the reflecting member 122, the second optical system 31, the reflecting member 32, the third optical system 33 and the lens 34 are arranged along the optical axis K in this order from the light modulating unit 22 side.
[0092] The first optical system 121 includes at least one lens, and guides light that is incident from the main body 101 to the first member 102 and travels in the direction X1 to the reflective member 122 .
[0093] The reflecting member 122 reflects the light incident from the first optical system 121 in the direction Y1. The reflecting member 122 is composed of, for example, a reflecting mirror. An opening 2b is formed in the first member 102 on the optical path of the light reflected by the reflecting member 122, and the reflected light passes through the opening 2b toward the hollow portion 3A of the second member 103.
[0094] The second member 103 is a member having a substantially T-shaped cross-sectional shape, and has an opening 3a formed at a position opposite to the opening 2b of the first member 102. Light from the main body 101 that has passed through the opening 2b of the first member 102 enters the hollow portion 3A of the second member 103 through the opening 3a. The cross-sectional shape of the first member 102 or the second member 103 is arbitrary and is not limited to the cross-sectional shape.
[0095] The second optical system 31 includes at least one lens, and guides the light incident from the first member 102 to the reflecting member 32 .
[0096] The reflecting member 32 reflects the light incident from the second optical system 31 in the direction X2 and guides the light to the third optical system 33. The reflecting member 32 is constituted by, for example, a reflecting mirror or the like.
[0097] The third optical system 33 includes at least one lens, and guides the light reflected by the reflecting member 32 to the lens 34 .
[0098] The lens 34 is disposed at the end portion so as to close the opening 3c formed at the end portion on the side in the direction X2 of the second member 103. The lens 34 projects the light incident from the third optical system 33 onto the projection object 6.
[0099] The projection direction changing mechanism 104 is a rotation mechanism that rotatably connects the second member 103 to the first member 102. The projection direction changing mechanism 104 allows the second member 103 to be rotatable about a rotation axis (specifically, the optical axis K) extending along the direction Y. The projection direction changing mechanism 104 is not limited to any other mechanism as long as it can rotate the optical system. Figure 4 Furthermore, the number of the rotating mechanisms is not limited to one, and a plurality of rotating mechanisms may be provided.
[0100] The displacement mechanism 105 is used to move the optical axis K of the projection optical system (in other words, the optical unit 106) along a direction perpendicular to the optical axis K thereof ( Figure 4 Specifically, the displacement mechanism 105 is configured to be able to change the position of the first component 102 relative to the main body 101 in the direction Y. In addition to manually moving the first component 102, the displacement mechanism 105 can also electrically move the first component 102.
[0101] Figure 4 This shows a state where the first component 102 is moved to the maximum extent in the direction Y1 by the displacement mechanism 105. Figure 4 In the state shown, the first component 102 is moved in the direction Y2 through the displacement mechanism 105, thereby changing the relative position of the center of the image formed by the light modulator 22 (in other words, the center of the display surface) and the optical axis K, so that the image G1 projected onto the projection object 6 can be displaced in the direction Y2 (parallel movement).
[0102] Alternatively, the displacement mechanism 105 may be a mechanism that moves the light modulator 22 in the direction Y instead of moving the optical unit 106 in the direction Y. In this case, the image G1 projected onto the object 6 can also be moved in the direction Y2.
[0103] <Information Processing Device 50 According to Embodiment>
[0104] Figure 5 : is a diagram showing an example of an information processing device 50 of an embodiment. The information processing device 50 of the embodiment is a tablet terminal having a touch panel 51. The touch panel 51 is a display that can be operated by touch. For example, a user of the information processing device 50 brings the information processing device 50 into a space (e.g., a room) where the projection device 10 is installed and projection is performed. The information processing device 50 displays a setting auxiliary image for assisting the installation of the projection device 10 in the space through the touch panel 51.
[0105] Specifically, information processing device 50 can overlap the first imaginary projection surface as imaginary projection surface, the first imaginary projection device as the first imaginary projection device or the first imaginary projection device, the second imaginary projection device as the second imaginary projection device or the second imaginary projection device, and the second imaginary projection device as the second imaginary projection device. The first imaginary projection device and the second imaginary projection device represent the candidate of projection device 10 virtually. The user can specify the projection device of different models from each other as the first imaginary projection device and the second imaginary projection device. And the user can also specify the projection device of different manufacturers from each other as the first imaginary projection device and the second imaginary projection device.
[0106] <Hardware Configuration of Information Processing Device 50>
[0107] Figure 6 is a diagram showing an example of the hardware configuration of the information processing device 50 according to the embodiment. Figure 6 As shown, Figure 5 The information processing device 50 shown includes a processor 61, a memory 62, a communication interface 63, a user interface 64, and a sensor 65. The processor 61, the memory 62, the communication interface 63, the user interface 64, and the sensor 65 are connected via a bus 69, for example.
[0108] The processor 61 is a circuit that performs signal processing, and is, for example, a CPU that manages the overall control of the information processing device 50. The processor 61 may be implemented by other digital circuits such as an FPGA or a DSP (Digital Signal Processor). Furthermore, the processor 61 may be implemented by combining a plurality of digital circuits.
[0109] The memory 62 includes, for example, a main memory and an auxiliary memory. The main memory is, for example, a RAM (Random Access Memory). The main memory is used as a work area for the processor 61.
[0110] The auxiliary memory is, for example, a nonvolatile memory such as a magnetic disk or a flash memory. Various programs for operating the information processing device 50 are stored in the auxiliary memory. The programs stored in the auxiliary memory are loaded into the main memory and executed by the processor 61.
[0111] Furthermore, the auxiliary memory may include a portable memory that can be attached to and detached from the information processing device 50. The portable memory includes a memory card such as a USB (Universal Serial Bus) flash drive or an SD (Secure Digital) memory card, or an external hard disk drive.
[0112] The communication interface 63 is a communication interface for communicating with an external device of the information processing device 50 . The communication interface 63 includes at least one of a wired communication interface for communicating by wire and a wireless communication interface for communicating by wireless. The communication interface 63 is controlled by the processor 61 .
[0113] The user interface 64 includes, for example, an input device for receiving operation input from the user or an output device for outputting information to the user. The input device can be implemented, for example, by a key (such as a keyboard) or a remote controller. The output device can be implemented, for example, by a display or a speaker. Figure 5 In the information processing device 50 shown, the input device and the output device are realized by the touch panel 51. The user interface 64 is controlled by the processor 61.
[0114] The sensor 65 includes an imaging device having an imaging optical system and an imaging element and capable of capturing images, or a space recognition sensor capable of three-dimensionally recognizing the surrounding space of the information processing device 50. The imaging device includes, for example, a device provided at Figure 5 The camera device is shown on the back side of the information processing device 50.
[0115] An example of a space recognition sensor is LIDAR (Light Detection and Ranging), which measures the distance or direction to an object by irradiating a laser and measuring the time it takes for the irradiated laser to hit the object and bounce back. However, the space recognition sensor is not limited to this, and can be a variety of sensors such as a radar that emits radio waves or an ultrasonic sensor that emits ultrasonic waves.
[0116] The information processing device 50 executes at least one of the following processes: a screen priority mode, a first projector priority mode, a second projector priority mode, a third projector priority mode, and a model search mode, for example.
[0117] (Screen priority mode)
[0118] Figure 7 This is a flowchart showing an example of processing of the information processing device 50 in the screen priority mode. Figure 8 to Figure 13 is Figure 7 An example of an image displayed by the information processing device 50 during the process shown in FIG. Figure 7When the process shown starts, the information processing device 50 displays Figure 8 The physical space image 80 is a live view image (live view image) showing in real time an image captured by an imaging device as the sensor 65 provided in the information processing device 50. The physical space image 80 is an example of the first image of the present invention.
[0119] The information processing device 50 three-dimensionally recognizes the physical space represented by the physical space image 80. In this space recognition, for example, a space recognition sensor included in the sensor 65 of the information processing device 50 is used. In the physical space represented by the physical space image 80, there are a floor 81 and a wall 82. Here, it is assumed that the projection device 10 is installed on the floor 81 and projects onto the wall 82.
[0120] First, the information processing device 50 receives the designation of the position and size of the first virtual projection plane from the user (step S71). The first virtual projection plane, for example, virtually represents the above-mentioned projection range 11. The position of the first virtual projection plane is, for example, the center position of the first virtual projection plane (the position of the projection center). For example, by the user indicating a point on the physical space image 80 as the center position of the first virtual projection plane, the position of the first virtual projection plane is designated as the two-dimensional coordinates of the physical space image 80. The information processing device 50 receives the three-dimensional position of the first virtual projection plane by converting the designated two-dimensional coordinates into three-dimensional coordinates according to the recognition result of the physical space represented by the physical space image 80. The size of the first virtual projection plane is designated using, for example, the actual distance in the physical space such as the length of the diagonal line = x [inch (inch)].
[0121] Next, the information processing device 50 displays the first virtual projection plane superimposed on the physical space image 80 according to the designation received in step S71 (step S72). Specifically, the information processing device 50 calculates the three-dimensional area of the first virtual projection plane in the physical space represented by the physical space image 80, converts the three-dimensional area into a two-dimensional area corresponding to the physical space image 80 according to the recognition result of the physical space represented by the physical space image 80, and superimposes the image representing the converted area as the first virtual projection plane on the physical space image 80. For example, Fig. 9 As shown, the information processing device 50 displays an image in which the first virtual projection plane 91 is superimposed on the physical space image 80 .
[0122] Next, the information processing device 50 receives the designation of the model of the first virtual projection device from the user from among the multiple selection items of the model of the projection device 10 (step S73). Next, the information processing device 50 calculates the position and size of the first settable range represented by a polyhedron, for example, based on the current position and size of the first virtual projection plane 91 and the projection ratio and lens shift amount that can be set using the model designated as the model of the first virtual projection device received in step S73, and displays the calculated first settable range in the position and size superimposed on the physical space image 80 (step S74).
[0123] Specifically, the information processing device 50 calculates the three-dimensional area of the first settable range in the physical space represented by the physical space image 80, converts the three-dimensional area into a two-dimensional area corresponding to the physical space image 80 according to the recognition result of the physical space represented by the physical space image 80, and overlays the image representing the converted area on the physical space image 80 as the first settable range. For example, Fig.10 As shown, the information processing device 50 displays an image in which the first virtual projection plane 91 and the first installable range 92 are superimposed on the physical space image 80 .
[0124] Next, the information processing device 50 receives the designation of the model of the second virtual projection device from the user from among the multiple selection items of the model of the projection device 10 (step S75). Next, the information processing device 50 calculates the position and size of the second settable range represented by the polyhedron, for example, based on the current position and size of the first virtual projection plane 91 and the projection ratio and lens shift amount that can be set using the model designated as the model of the second virtual projection device received in step S75, and displays the calculated second settable range in the position and size superimposed on the physical space image 80 (step S76).
[0125] Specifically, the information processing device 50 calculates the three-dimensional area of the second settable range in the physical space represented by the physical space image 80, converts the three-dimensional area into a two-dimensional area corresponding to the physical space image 80 based on the recognition result of the physical space represented by the physical space image 80, and overlays the image representing the converted area on the physical space image 80 as the second settable range. For example, Fig.11 As shown, the information processing device 50 displays an image in which the first virtual projection plane 91 , the first installable range 92 , and the second installable range 93 are superimposed on the physical space image 80 .
[0126] Thus, for projection devices having different setting ranges for parameters such as projection ratio or lens shift amount, the user can visually compare the settable ranges (e.g., the first settable range 92 or the second settable range 93) relative to a desired projection surface (e.g., the first imaginary projection surface 91), thereby easily selecting the model of an appropriate projection device.
[0127] Next, the information processing device 50 determines whether an instruction to change the size of the first virtual projection plane 91 is received from the user (step S77). If an instruction to change the size of the first virtual projection plane 91 is received (step S77: yes), the information processing device 50 changes the size of the first virtual projection plane 91 and displays the changed first virtual projection plane 91 superimposed on the physical space image 80 (step S78).
[0128] Furthermore, the information processing device 50 recalculates the position and size of the first settable range 92 based on the current position and size of the first virtual projection plane 91 and the projection ratio and lens shift amount that can be set using the model designated as the model of the first virtual projection device received in step S73. Furthermore, the information processing device 50 recalculates the position and size of the second settable range 93 based on the current position and size of the first virtual projection plane 91 and the projection ratio and lens shift amount that can be set using the model designated as the model of the second virtual projection device received in step S75. Then, the information processing device 50 superimposes the first settable range 92 and the second settable range 93 of the recalculated position and size on the physical space image 80 and displays them (step S79), and returns to step S77.
[0129] Thus, the user can visually grasp the change in the position or size of the settable range of the first settable range 92 and the second settable range 93 relative to the change in the size of the first virtual projection plane 91, and can easily select an appropriate model. For example, assume that a change instruction is given to change the size of the first virtual projection plane 91 from x [inch] to y [inch] (x>y). In this case, for example, Fig.12 As shown, the first settable range 92 and the second settable range 93 are reduced in conjunction with the reduction of the first virtual projection plane 91, and the distances between the first settable range 92 and the second settable range 93 and the first virtual projection plane 91 are reduced.
[0130] In step S77, when no instruction to change the size of the first virtual projection plane 91 is received (step S77: No), the information processing device 50 determines whether an instruction to change the position of the first virtual projection plane 91 is received (step S80). When an instruction to change the position of the first virtual projection plane 91 is received (step S80: Yes), the information processing device 50 changes the position of the first virtual projection plane 91 and displays the changed first virtual projection plane 91 superimposed on the physical space image (step S81).
[0131] And, similar to step S79, the information processing device 50 recalculates the respective positions and sizes of the first settable range 92 and the second settable range 93, displays the first settable range 92 and the second settable range 93 of the recalculated positions and sizes overlapping on the physical space image 80 (step S82), and returns to step S77.
[0132] Thus, the user can visually grasp the change in the position or size of the settable ranges of the first settable range 92 and the second settable range 93 relative to the position of the first virtual projection plane 91, and can easily select an appropriate model. For example, assume that a change instruction is given to move the position of the first virtual projection plane 91 to the left of the first virtual projection plane 91. In this case, for example, Fig.13 As shown, the first settable range 92 and the second settable range 93 move to the left in conjunction with the movement of the first virtual projection plane 91 to the left.
[0133] In step S80 , when the instruction to change the position of the first virtual projection plane 91 is not received (step S80 : No), the information processing device 50 returns to step S77 .
[0134] Thus, in the screen priority mode, the information processing device 50 displays the second image including the first virtual projection plane 91, the first settable range 92 of the first virtual projection device, and the second settable range 93 of the second virtual projection device. Furthermore, the information processing device 50 sets the first settable range 92 of the first virtual projection device and the second settable range 93 of the second virtual projection device according to the position and size of the first virtual projection plane 91. Furthermore, the information processing device 50 may also set the first settable range 92 of the first virtual projection device and the second settable range 93 of the second virtual projection device according to any one of the position and size of the first virtual projection plane 91.
[0135] Furthermore, in the screen priority mode, when at least one of the position and the size of the first virtual projection plane 91 is changed, the information processing device 50 resets the first installable range 92 of the first virtual projection device and the second installable range 93 of the second virtual projection device.
[0136] (Projector priority 1st mode)
[0137] Fig.14 This is a flowchart showing an example of the processing of the information processing device 50 in the projector priority first mode. Figure 15 to Figure 19 is Fig.14 An example of an image displayed by the information processing device 50 during the process shown in FIG. Fig.14 When the process shown starts, the information processing device 50 displays Figure 8 The physical space image 80 is shown. Then, the information processing device 50 recognizes the physical space represented by the physical space image 80 in three dimensions.
[0138] First, the information processing device 50 receives the designation of the position of the first virtual projection device and the position of the first virtual projection plane 91 (the position of the projection center) from the user (step S141). The position of the first virtual projection device is, for example, the position of the front lens of the projection lens of the first virtual projection device. For example, the information processing device 50 receives Fig.15 The position P1 of the first virtual projection device and the position P2 of the first virtual projection plane 91 are designated. The information processing device 50 calculates the projection distance D1, which is the distance between the positions P1 and P2, based on the designated positions P1 and P2.
[0139] Next, the information processing device 50 receives the designation of the model of the first virtual projection device from the user from a plurality of options of the model of the projection device 10 (step S142). Next, the information processing device 50 calculates the size of the first virtual projection surface 91 based on the projection distance D1 and the projection ratio corresponding to the model of the first virtual projection device designated in step S142 (step S143).
[0140] Then, the information processing device 50 displays the three-dimensional model of the first virtual projection device designated in step S142, namely, the first virtual projection device 151 and the first virtual projection plane 91, superimposed on the physical space image 80 (step S144). Fig.16 As shown, the information processing device 50 displays an image in which the first virtual projection device 151 and the first virtual projection plane 91 are superimposed on the physical space image 80 .
[0141] The first virtual projection device 151 is arranged so that the front lens of the projection lens of the first virtual projection device 151 becomes Fig.15 The first virtual projection plane 91 is arranged so that the center position of the first virtual projection plane 91 becomes Fig.15 The position P2 shown. Furthermore, the size of the first virtual projection plane 91 is the size calculated in step S143.
[0142] Next, the information processing device 50 receives the designation of the model of the second virtual projection device from the multiple selection items of the model of the projection device 10 (step S145). Next, the information processing device 50 calculates the position and size of the second settable range 93 in which the second virtual projection device can be set based on the position and size of the first virtual projection plane 91 and the projection ratio and lens shift amount that can be set using the model designated as the model of the second virtual projection device received in step S145, and displays the second settable range 93 with the calculated position and size superimposed on the physical space image 80 (step S146). For example, Fig.17 As shown, the information processing device 50 displays an image in which the first virtual projection device 151 , the first virtual projection plane 91 , and the second installable range 93 are superimposed on the physical space image 80 .
[0143] Thus, the user can visually grasp the second installable range 93 of the second virtual projection device as a comparison object with respect to the first virtual projection device 151 temporarily virtually arranged, and can easily select an appropriate model.
[0144] Next, the information processing device 50 determines whether a change instruction of the position of the first virtual projection device 151 is received from the user (step S147). When a change instruction of the position of the first virtual projection device 151 is received (step S147: Yes), the information processing device 50 changes the position of the first virtual projection device 151, and displays the changed first virtual projection device 151 superimposed on the physical space image 80 (step S148).
[0145] Furthermore, the information processing device 50 recalculates the position and size of the first virtual projection plane 91 based on the current position of the first virtual projection device 151 and the projection ratio and lens shift amount that can be set using the model designated as the model of the first virtual projection device received in step S142. Then, the information processing device 50 overlays the recalculated first virtual projection plane 91 on the physical space image 80 and displays it (step S149).
[0146] Next, the information processing device 50 recalculates the position and size of the second settable range 93 based on the recalculated position and size of the first virtual projection plane 91 and the projection ratio and lens shift amount that can be set using the model designated as the model of the second virtual projection device received in step S145. Then, the information processing device 50 overlays the recalculated second settable range 93 on the physical space image 80 and displays it (step S150), and returns to step S147.
[0147] Thus, the user can visually grasp the change in the position and size of the second settable range 93 of the model (second virtual projection device) as the comparison object relative to the size of the first virtual projection plane 91, thereby easily selecting an appropriate model. For example, assume that a change instruction is given to instruct to bring the first virtual projection device 151 closer to the first virtual projection plane 91. In this case, for example, Fig.18 As shown, the first virtual projection plane 91 is reduced in conjunction with the movement of the first virtual projection device 151 toward the first virtual projection plane 91 , and the second settable range 93 moves toward the first virtual projection plane 91 , and the second settable range 93 is reduced.
[0148] In step S147, when the change instruction of the position of the first virtual projection device 151 is not received (step S147: No), the information processing device 50 determines whether the change instruction of the position of the first virtual projection plane 91 is received from the user (step S151). When the change instruction of the position of the first virtual projection plane 91 is received (step S151: Yes), the information processing device 50 changes the position of the first virtual projection plane 91, and displays the changed first virtual projection plane 91 superimposed on the physical space image 80 (step S152).
[0149] Next, the information processing device 50 recalculates the position and size of the second settable range 93 based on the current position and size of the first virtual projection plane 91 and the projection ratio and lens shift amount that can be set using the model designated as the model of the second virtual projection device received in step S145. Then, the information processing device 50 overlays the recalculated second settable range 93 of the position and size on the physical space image 80 for display (step S153), and returns to step S147.
[0150] Thus, the user can visually grasp the change in the position of the second settable range 93 of the model (second virtual projection device) as the comparison object relative to the position of the first virtual projection plane 91, thereby easily selecting an appropriate model. For example, assume that a change instruction is given to move the position of the first virtual projection plane 91 to the left toward the first virtual projection plane 91. In this case, for example, Fig.19 As shown, the second settable range 93 moves to the left direction in conjunction with the movement of the first virtual projection plane 91 to the left direction.
[0151] In step S151 , when the instruction to change the position of the first virtual projection plane 91 is not received (step S151 : No), the information processing device 50 returns to step S147 .
[0152] In addition, the information processing device 50 may also receive a change instruction of the projection ratio of the first virtual projection device 151 from the user after step S147. For example, when receiving a change instruction to reduce the projection ratio of the first virtual projection device 151, the information processing device 50 reduces the first virtual projection surface 91 while maintaining the position of the first virtual projection device 151. In this case, the information processing device 50 moves the second settable range 93 to the first virtual projection surface 91, thereby reducing the second settable range 93.
[0153] Thus, in the projector priority first mode, the information processing device 50 displays the second image including the first virtual projection plane 91, the first virtual projection device 151, and the second settable range 93 of the second virtual projection device. Furthermore, the information processing device 50 sets at least one of the position and size of the first virtual projection plane 91 according to the position of the first virtual projection device 151, and sets the second settable range 93 of the second virtual projection device according to at least one of the set position and size of the first virtual projection plane 91.
[0154] And, in the projector priority 1st mode, when the position or parameter of the 1st imaginary projection device 151 is changed, the information processing device 50 resets at least any one of the position and size of the 1st imaginary projection surface 91 and the 2nd settable range 93 of the 2nd imaginary projection device. And, when the position of the 1st imaginary projection surface 91 is changed, the information processing device 50 resets the 2nd settable range 93 of the 2nd imaginary projection device.
[0155] (Projector priority 2nd mode)
[0156] Fig. 20 and Fig.21 This is a flowchart showing an example of the processing of the information processing device 50 in the projector priority second mode. Figure 22 to Figure 26 is Fig. 20 , Fig.21 An example of an image displayed by the information processing device 50 during the process shown in FIG. Fig. 20 When the process shown starts, the information processing device 50 displays Figure 8 The physical space image 80 is shown. Then, the information processing device 50 recognizes the physical space represented by the physical space image 80 in three dimensions.
[0157] First, with Fig.14 Similarly to step S141 shown in FIG. 1 , the information processing device 50 receives from the user the designation of the position of the first virtual projection device 151 and the position of the first virtual projection plane 91 (position of the projection center) (step S201). Fig.15The position P1 of the first virtual projection device and the position P2 of the first virtual projection plane 91 are designated.
[0158] Then, with Fig.14 Similarly to step S142 shown, the information processing device 50 receives the designation of the model of the first virtual projection device from the user from among the plurality of options for the model of the projection device 10 (step S202 ).
[0159] Next, the information processing device 50 calculates the position of the projection center when the lens shift is not performed in the first virtual projection device 151 based on the position of the first virtual projection device 151 designated in step S201 (step S203). Fig. 22 As shown, the information processing device 50 calculates the position P3 of the projection center when no lens shift is performed (when the lens shift amount = 0) in the first virtual projection device 151. The information processing device 50 also calculates the projection distance D2 which is the distance between the positions P1 and P3.
[0160] Next, the information processing device 50 calculates the size of the first virtual projection plane 91 based on the projection distance D2 and the projection ratio corresponding to the model of the first virtual projection device designated by step S202 (step S204). For example, the information processing device 50 calculates the size of the projection plane when the first virtual projection device 151 projects to the position P3 without lens shift as the size of the first virtual projection plane 91.
[0161] Next, the information processing device 50 projects the first virtual projection device 151 and the first virtual projection plane 91 onto the physical space image 80 for display (step S205). Fig.23 As shown, the information processing device 50 displays an image in which the first virtual projection device 151 and the first virtual projection plane 91 are superimposed on the physical space image 80. The first virtual projection device 151 is a three-dimensional model of the model designated as the first virtual projection device model received in step S202.
[0162] The first virtual projection device 151 is arranged so that the front lens of the projection lens of the first virtual projection device 151 is located at Fig. 22 The first virtual projection plane 91 is arranged so that the center position of the first virtual projection plane 91 becomes Fig. 22 The position P2 shown in FIG. 1 is a position P2 shown in FIG.
[0163] And, if Fig.24As shown, the information processing device 50 sets a coordinate system in which the axis perpendicular to the bottom surface of the first virtual projection device 151 is set as the Y axis, the projection direction of the first virtual projection device 151 is set as the Z axis, and the axis orthogonal to the Y axis and the Z axis is set as the X axis.
[0164] Next, the information processing device 50 receives the designation of the model of the second virtual projection device from a plurality of options of the model of the projection device 10 (step S206). Next, the information processing device 50 sets the position of the second virtual projection device to the position where the evaluation value of the distance between the first virtual projection device 151 and the second virtual projection device is the minimum, based on the specification of the model designated as the model of the second virtual projection device received by step S206 (step S207).
[0165] This evaluation value is, for example, an evaluation value that becomes smaller as the distance between the first virtual projection device 151 and the second virtual projection device becomes shorter. The distance between the first virtual projection device 151 and the second virtual projection device can be the distance between the lens positions (for example, the position of the front lens) of the first virtual projection device 151 and the second virtual projection device, or the distance between the centers of the bottom surfaces between the first virtual projection device 151 and the second virtual projection device. Here, it is assumed that the above evaluation value becomes minimum at a position where the Y coordinates of the bottom surfaces of the first virtual projection device 151 and the second virtual projection device are consistent and the X coordinates and Z coordinates of the lens positions of the first virtual projection device 151 and the second virtual projection device are consistent. In addition, it can also be simply assumed that the evaluation value becomes minimum when the X coordinates, Y coordinates, and Z coordinates of the lenses of the first virtual projection device 151 and the second virtual projection device are consistent.
[0166] Next, the information processing device 50 calculates the position of the projection center when the lens shift is not performed in the second virtual projection device, similarly to the position P3, based on the specification of the model designated as the model of the second virtual projection device received in step S206 and the position of the second virtual projection device set in step S207 (step S208). For example, the information processing device 50 calculates the position P4 of the projection center when the lens shift is not performed in the second virtual projection device (when the lens shift amount = 0). In addition, the information processing device 50 calculates the distance between the position of the second virtual projection device set in step S207 and the position P4, that is, the projection distance D3.
[0167] Next, the information processing device 50 calculates the size of the 2nd imaginary projection surface based on the 2nd imaginary projection device similarly to the calculation of the size of the 1st imaginary projection surface 91 based on step S204 (step S209). For example, the information processing device 50 calculates the size of the projection surface when the 2nd imaginary projection device is projected to position P4 without lens shifting as the size of the 2nd imaginary projection surface.
[0168] Furthermore, when the second virtual projection device has a zoom function, the information processing device 50 sets the projection ratio of the second virtual projection device in such a manner that the size of the second virtual projection plane is closest to the size of the first virtual projection plane 91. Furthermore, the information processing device 50 sets the lens shift amount of the second virtual projection device in such a manner that the distance between the center position of the first virtual projection plane 91 and the center position of the second virtual projection plane is minimized based on the specification of the model designated as the model of the second virtual projection device received in step S206 (step S210).
[0169] Next, the information processing device 50 overlaps the second virtual projection device and the second virtual projection plane on the physical space image 80 and displays them (step S211). Fig.25 As shown, the information processing device 50 displays an image in which the first virtual projection device 151, the first virtual projection plane 91, the second virtual projection device 251, and the second virtual projection plane 252 are superimposed on the physical space image 80. The second virtual projection device 251 is a three-dimensional model of the model designated as the model of the second virtual projection device received in step S206.
[0170] The second virtual projection device 251 is arranged so that the front lens of the projection lens of the second virtual projection device 251 is at the position of the second virtual projection device set in step S207. The second virtual projection plane 252 is arranged so that the center position of the second virtual projection plane 252 is displaced by the lens shift amount set in step S210 relative to the center position of the second virtual projection plane calculated in step S208. Furthermore, the size of the second virtual projection plane 252 is the size calculated in step S209.
[0171] Thus, even if the user changes the model, there is no need to reset the position of the virtual projection device or the virtual projection plane, and the user can visually grasp the projection state, thereby easily selecting an appropriate model.
[0172] The information processing device 50 can display the range in which the position and size of the first virtual projection plane 91 or the second virtual projection plane 252 can be adjusted by changing the parameters of the first virtual projection device 151 or the second virtual projection device 251. Fig.26As shown, the information processing device 50 displays the lens shift range 261 of the second virtual projection device 251 when the size of the first virtual projection plane 91 is closest to the size of the second virtual projection plane 252. The lens shift range 261 is a range centered on the center position of the second virtual projection plane calculated in step S208.
[0173] This allows the user to visually grasp the settable range of the virtual projection plane when changing the model, thereby making it easier for the user to select an appropriate model.
[0174] Next, the information processing device 50 determines whether a change instruction of the position of the first virtual projection device 151 is received from the user (step S212). When a change instruction of the position of the first virtual projection device 151 is received (step S212: Yes), the information processing device 50 changes the position of the first virtual projection device 151, and displays the changed first virtual projection device 151 superimposed on the physical space image 80 (step S213).
[0175] Furthermore, the information processing device 50 recalculates the position and size of the first virtual projection plane 91 based on the current position of the first virtual projection device 151 and the projection ratio and lens shift amount that can be set using the model designated as the model of the first virtual projection device received in step S142. Then, the information processing device 50 overlays the recalculated first virtual projection plane 91 on the physical space image 80 and displays it (step S214).
[0176] Next, the information processing device 50 recalculates the position of the second virtual projection device 251 so that the distance from the first virtual projection device 151 becomes the minimum. In addition, the information processing device 50 recalculates the parameters (for example, lens shift amount or zoom amount) of the second virtual projection device 251 in such a way that the difference between the position and size of the second virtual projection plane 252 and the position and size of the first virtual projection plane 91 recalculated in step S214 becomes the minimum. Then, the information processing device 50 superimposes the second virtual projection device 251 and the second virtual projection plane 252 based on these recalculated results on the physical space image 80 for display (step S215), and returns to step S212.
[0177] Thus, the user can visually grasp the projection state of the model to be compared (the second virtual projection device 251 ) without performing any operation other than changing the position of the first virtual projection device 151 , and can easily select an appropriate model.
[0178] In step S212, when the change instruction of the position of the first virtual projection device 151 is not received (step S212: No), the information processing device 50 determines whether the change instruction of the zoom amount of the first virtual projection device 151 is received from the user (step S216). When the change instruction of the zoom amount of the first virtual projection device 151 is received (step S212: Yes), the information processing device 50 recalculates the size of the first virtual projection plane 91 based on the zoom amount of the changed first virtual projection device 151, that is, the projection ratio, and displays the first virtual projection plane 91 of the recalculated size superimposed on the physical space image 80 (step S217).
[0179] Next, the information processing device 50 recalculates the parameters (e.g., zoom amount) of the second virtual projection device 251 in such a way that the difference between the size of the second virtual projection plane 252 and the size of the first virtual projection plane 91 recalculated in step S217 is minimized. Then, the information processing device 50 superimposes the second virtual projection device 251 and the second virtual projection plane 252 based on the recalculated result on the physical space image 80 and displays them (step S218), and returns to step S212.
[0180] Thus, the user can visually grasp the projection state of the model to be compared (second virtual projection device 251 ) without performing any operation other than changing the zoom amount (projection ratio) with respect to the first virtual projection device 151 , thereby easily selecting an appropriate model.
[0181] In step S216, when the information processing device 50 has not received an instruction to change the position of the first virtual projection device 151 (step S216: No), the information processing device 50 determines whether an instruction to change the lens shift amount of the first virtual projection device 151 (the position of the first virtual projection plane 91) has been received from the user (step S219). When the information processing device 50 has received an instruction to change the lens shift amount of the first virtual projection device 151 (step S219: Yes), the information processing device 50 recalculates the position of the first virtual projection plane 91 based on the lens shift amount of the first virtual projection device 151 after the change, and displays the recalculated first virtual projection plane 91 superimposed on the physical space image 80 (step S220).
[0182] Next, the information processing device 50 sets the lens shift amount of the second virtual projection device 251 in such a way that the distance between the center position of the first virtual projection plane 91 and the center position of the second virtual projection plane 252 is minimized based on the specification of the model designated as the model of the second virtual projection device received in step S206. And, the information processing device 50 recalculates the position of the second virtual projection plane 252 based on the set lens shift amount of the second virtual projection device 251. Then, the information processing device 50 displays the second virtual projection plane 252 at the recalculated position by overlapping it on the physical space image 80 (step S221), and returns to step S212.
[0183] Thus, the user can visually grasp the projection state of the model (second virtual projection device 251) as the comparison object without performing any operation other than changing the lens shift amount (position of the first virtual projection plane 91) of the first virtual projection device 151, thereby easily selecting an appropriate model.
[0184] In step S219 , when the information processing device 50 has not received the instruction to change the lens shift amount of the first virtual projection device 151 (step S219 : No), the information processing device 50 returns to step S212 .
[0185] Thus, in the projector priority second mode, the information processing device 50 displays the second image including the first virtual projection plane 91, the first virtual projection device 151, the second virtual projection device 251, and the second virtual projection plane 252 corresponding to the second virtual projection device 251. Furthermore, the information processing device 50 sets the position of the second virtual projection plane 252 and the position of the second virtual projection device 251 according to the position of the first virtual projection device 151 and the position of the first virtual projection plane 91.
[0186] Furthermore, in the projector priority second mode, the information processing device 50 sets the size of the first virtual projection plane 91 according to the position of the first virtual projection device 151 and the position of the first virtual projection plane 91, and sets the position of the second virtual projection device according to the position of the first virtual projection device 151 (for example, set it to a position as close as possible). Then, the information processing device 50 sets the parameters (lens shift amount or zoom amount, etc.) of the second virtual projection device 251 related to the position and size of the second virtual projection plane 252 according to the position and size of the first virtual projection plane 91 (for example, set it so that the position and size of the first virtual projection plane 91 and the second virtual projection plane 252 are as close as possible).
[0187] Furthermore, the information processing device 50 may also perform the following processing: an image representing a range in which at least any one of the position and size of the first virtual projection plane 91 or the second virtual projection plane 252 can be adjusted by changing parameters of the first virtual projection device 151 or the second virtual projection device 251 (for example, the lens shift range 261) is included in the second image.
[0188] (Projector priority 3rd mode)
[0189] Fig. 27 FIG. 4 is a flowchart showing an example of processing of the information processing device 50 in the projector priority third mode. Fig. 27 When the process shown starts, the information processing device 50 displays Figure 8 The physical space image 80 is shown. Then, the information processing device 50 recognizes the physical space represented by the physical space image 80 in three dimensions.
[0190] Fig. 27 Steps S271 to S276 shown in FIG. Fig. 20 After step S276, the information processing device 50 calculates the position of the second virtual projection device 251 and the position and size of the second virtual projection surface 252 according to the specifications of the model designated as the model of the second virtual projection device received by step S276 (step S277). Regarding the calculation in step S277, Fig.28 , Fig.30 The following will be described later.
[0191] Next, the information processing device 50 superimposes the second virtual projection device 251 and the second virtual projection plane 252 on the physical space image 80 and displays them (step S278), and ends a series of processing. Fig.21 In steps S212 to S221 shown, processing based on various change instructions is performed in the same manner.
[0192] Fig.28 Yes means Fig. 27 Flowchart of an example of calculation in step S277. Fig.29 is Fig.28 In the example Fig. 27 An example of an image displayed by the information processing device 50 during the processing of Fig. 27 In step S277, the information processing device 50 performs, for example, Fig.28 Processing shown.
[0193] First, the information processing device 50 determines whether the first virtual projection device 151 and the second virtual projection device 251 can be set to the same aspect ratio (step S281). Fig. 27 The process is performed based on the models of the first virtual projection device 151 and the second virtual projection device 251 specified in steps S272 and S276.
[0194] In step S281, when the first virtual projection device 151 and the second virtual projection device 251 can be set to the same aspect ratio (step S281: Yes), the information processing device 50 sets the size of the second virtual projection plane 252 to the same size as that of the second virtual projection plane 252. Fig. 27 The size of the first virtual projection plane 91 calculated in step S274 is the same as the size (step S282).
[0195] In step S281, when the first virtual projection device 151 and the second virtual projection device 251 cannot be set to the same aspect ratio (step S281: No), the information processing device 50 sets the size of the second virtual projection plane 252 such that the upper and lower width of the second virtual projection plane 252 is greater than the upper and lower width of the first virtual projection device 151 and the left and right width of the second virtual projection plane 252 is greater than the upper and lower width of the first virtual projection device 151 (step S283).
[0196] Next, the information processing device 50 calculates the possible range of the projection distance of the second imaginary projection device 251 (step S284) according to the projection ratio of the model designated as the model of the second imaginary projection device received by step S206. And, the information processing device 50 sets the position of the second imaginary projection surface 252 at the same position as the position of the first imaginary projection surface 91 (step S285).
[0197] Next, the information processing device 50 sets the position of the second virtual projection device 251 (step S286), and ends. Fig. 27 Specifically, in step S286, the information processing device 50 sets the position of the second virtual projection device 251 in a manner that the distance between the first virtual projection device 151 and the second virtual projection device 251 is minimized in the setting candidate range of the second virtual projection device 251 obtained from the position of the second virtual projection surface 252 set in step S285 and the possible range of the projection distance of the second virtual projection device 251 calculated in step S284.
[0198] exist Fig.28 In the example, Fig. 27 In step S278, for example, Fig.29As shown in FIG. 1 , the information processing device 50 displays the first virtual projection plane 91 and the second virtual projection plane 252 whose center positions coincide with each other and the first virtual projection device 151 and the second virtual projection plane 252 corresponding thereto. Fig.29 In the example of , the bottom surface of the first virtual projection device 151 and the bottom surface of the second virtual projection device 251 are not on the same plane.
[0199] Fig.30 Yes means Fig. 27 Flowchart of another example of the calculation in step S277. Fig.31 and Fig.32 is Fig.30 In the example Fig. 27 An example of an image displayed by the information processing device 50 during the processing. When the constraint condition related to the position of the second virtual projection device 251 is set, Fig. 27 In step S277, the information processing device 50 performs, for example, Fig.30 Processing shown.
[0200] Fig.30 Steps S301 to S305 shown in FIG. Fig.28 After step S305, the information processing device 50 determines whether there is a position satisfying the constraint condition within the settable range of the second virtual projection device 251 (step S306). If there is a position satisfying the constraint condition within the settable range of the second virtual projection device 251 (step S306: yes), the information processing device 50 sets the settable range of the second virtual projection device 251 and the position of the second virtual projection device 251 satisfying the constraint condition (step S307), and ends. Fig. 27 Step S277.
[0201] In step S306, when there is no position satisfying the constraint condition within the settable range of the second virtual projection device 251 (step S306: No), the information processing device 50 changes the position of the second virtual projection device 251 in a manner satisfying the constraint condition and within the range of the projection distance calculated by step S304 (step S308).
[0202] Next, the information processing device 50 sets the position (center position) of the second virtual projection plane 252 to the position (center position) closest to the position of the first virtual projection plane 91 (step S309), and ends. Fig. 27 Step S277.
[0203] For example, in Fig.30In the example of , the constraint condition is set to be that the bottom surface of the first virtual projection device 151 and the bottom surface of the second virtual projection device 251 are located on the same plane. Furthermore, in the second virtual projection device 251, if the position (center position) of the second virtual projection plane 252 and the position (center position) of the first virtual projection plane 91 are set at the same position, there is no position that satisfies the constraint condition within the settable range of the second virtual projection device 251. In this case, Fig. 27 In step S278, for example, Fig.31 As shown, the information processing device 50 displays the first virtual projection device 151, the second virtual projection device 251 having a bottom surface located on the same plane as the bottom surface of the first virtual projection device 151, the first virtual projection surface 91, and the second virtual projection surface 252 that can be projected from the second virtual projection device 251 and is close to the first virtual projection surface 91.
[0204] And, in Fig.30 In the example, the constraint condition is set such that the bottom surface of the second virtual projection device 251 is located on the physical plane. Furthermore, the position of the first virtual projection device 151 is set to a position where the bottom surface of the first virtual projection device 151 is a plane different from the floor 81 (physical plane). Furthermore, if the lens displacement range of the second virtual projection device 251 is sufficiently large, the position (center position) of the second virtual projection plane 252 and the position (center position) of the first virtual projection plane 91 are set at the same position, and the settable range of the second virtual projection device 251 overlaps with the floor 81. In this case, Fig. 27 In step S278, for example, Fig.32 As shown, the information processing device 50 displays the first virtual projection device 151 , the second virtual projection device 251 whose bottom surface is located on the floor 81 , and the first virtual projection plane 91 and the second virtual projection plane 252 whose positions coincide with each other.
[0205] Thus, the user can visually grasp the positional differences of the virtual projection devices when virtual projection surfaces having sizes as close as possible are arranged among different models of virtual projection devices, thereby facilitating selection of an appropriate model.
[0206] Thus, in the projector priority third mode, similarly to the projector priority second mode, the information processing device 50 displays the second image including the first virtual projection plane 91, the first virtual projection device 151, the second virtual projection device 251, and the second virtual projection plane 252 corresponding to the second virtual projection device 251. Furthermore, the information processing device 50 sets the position of the second virtual projection plane 252 and the position of the second virtual projection device 251 based on the position of the first virtual projection device 151 and the position of the first virtual projection plane 91.
[0207] Furthermore, in the projector priority third mode, the information processing device 50 sets the size of the first virtual projection plane 91 according to the position of the first virtual projection device 151 and the position of the first virtual projection plane 91, and sets the position and size of the second virtual projection plane 252 according to the position and size of the first virtual projection plane 91 (for example, setting it to a position and size as close as possible). Then, the information processing device 50 sets the position of the second virtual projection plane 252 according to the position of the first virtual projection device 151 (for example, setting it so that the positions of the first virtual projection plane and the second virtual projection plane are as close as possible).
[0208] Furthermore, in the projector priority third mode, after determining the positions or sizes of the first virtual projection device 151 and the first virtual projection plane 91, the second virtual projection device 251 or the second virtual projection plane 252 is set with these positions as constraints, but the information processing device 50 may also release the constraints according to a predetermined condition such as an instruction from a user. When the constraints are released, the information processing device 50 sets the positions or sizes of the second virtual projection device 251 or the second virtual projection plane 252 regardless of the positions or sizes of the first virtual projection device 151 and the first virtual projection plane 91 according to the instructions from the user. That is, the information processing device 50 may also be able to switch between a state in which the second virtual projection device 251 and the second virtual projection plane 252 are set with the constraints based on the settings of the first virtual projection device 151 and the first virtual projection plane 91 and a state in which the second virtual projection device 251 and the second virtual projection plane 252 are set without the constraints.
[0209] As described above, the information processing device 50 can improve the convenience of the user related to the model selection of the projection device to be set by displaying the second image displayed on the physical space image 80 by displaying the first virtual projection plane 91, the first virtual projection device 151 or the first settable range 92 and the second virtual projection device 251 or the second settable range 93.
[0210] (Model search mode)
[0211] Fig.33 This is a flowchart showing an example of processing of the information processing device 50 in the model number search mode. Figure 34 to Figure 38 is Fig.33 An example of an image displayed by the information processing device 50 during the process shown. Figure 7 Similarly, the information processing device 50 Fig.33 When the process shown starts, the display Figure 8 The physical space image 80 is shown. Then, the information processing device 50 recognizes the physical space represented by the physical space image 80 in three dimensions.
[0212] First, the information processing device 50 receives the designation of the position and size of the virtual projection plane from the user (step S331). Next, the information processing device 50 displays the virtual projection plane superimposed on the physical space image 80 according to the designation received in step S331 (step S332). The processing of displaying the image superimposed on the virtual projection plane on the physical space image 80 is similar to the processing of displaying the virtual projection plane on the physical space image 80. Fig. 9 The processing of superimposing the image of the virtual projection surface on the physical space image 80 shown is the same. Fig.34 As shown, the information processing device 50 displays an image in which the virtual projection plane 341 is superimposed on the physical space image 80 .
[0213] Next, the information processing device 50 receives the designation of the position and size of the range where the virtual projection device is desired to be set, that is, the setting candidate range, from the user (step S333). Next, the information processing device 50 displays the designated setting candidate range received in step S333 by overlapping it on the physical space image 80 (step S334). Fig.35 As shown, the information processing device 50 displays an image in which the virtual projection plane 341 and the setting candidate range 351 are superimposed on the physical space image 80. In this example, the setting candidate range 351 is a rectangular parallelepiped.
[0214] Next, the information processing device 50 extracts the model of the virtual projection surface 341 that can realize the position and size specified in step S331 from the registered projection device models (step S335). For example, the information processing device 50 configures each of the registered projection device models within the setting candidate range 351 specified in step S333, and determines whether the virtual projection surface 341 can be realized by setting the parameters of the model (for example, the range displacement or the zoom amount), thereby performing the extraction in step S335.
[0215] Next, the information processing device 50 displays a list of the models extracted in step S335 (step S336). For example, the information processing device 50 displays Fig.36 In this example, the suitable model list 361 shows "Projector 1", "Projector 2" and "Projector 5" as models (suitable models) that can realize the virtual projection surface 341. In addition, the information processing device 50 can replace Fig.34 The image shown shows a list of suitable models 361, which can also be used with Fig.34 The image shown also shows a list 361 of suitable models.
[0216] This allows the user to narrow down the models that are suitable for the installation candidate range 351 where the user wants to install the projection device, thereby making it easier for the user to select an appropriate model.
[0217] Next, the information processing device 50 receives the selection of a model of a virtual projection device from the user from the list displayed in step S336 (step S337 ).
[0218] Next, the information processing device 50 calculates the position and size of the settable range based on the position and size of the virtual projection plane 341 received in step S331 and the projection ratio and lens shift that can be set using the model designated as the model of the virtual projection device received in step S337. Then, the information processing device 50 displays the settable range whose position and size are calculated by overlapping it on the physical space image 80 (step S338). For example, Fig.37 As shown, the information processing device 50 displays an image in which the virtual projection plane 341, the setting candidate range 351, and the settable range 371 are superimposed on the physical space image 80. In this example, the setting candidate range 351 is a rectangular parallelepiped. In addition, at this time, the information processing device 50 can also limit the settable range 371 to only the overlapping area with the settable range 351 specified by the user.
[0219] This allows the user to visually grasp the settable range limited by the model being considered for setting, and thus easily select an appropriate model.
[0220] And, in Fig.33 In step S335, the information processing device 50 may also configure each model of the registered projection device within the setting candidate range 351 received in step S333, and determine whether the virtual projection surface 341 can be realized by setting the parameters of its model according to whether there is clipping. Then, in step S336, the information processing device 50 may distinguish and display the candidates of the model that can realize the virtual projection surface 341 from the candidates of the model that can realize the virtual projection surface 341 without clipping and the candidates of the model that can realize the virtual projection surface 341 with clipping. For example, the information processing device 50 displays Fig.38 The list of compatible models 361 is shown. In this example, the list of compatible models 361 indicates "Projector 1", "Projector 2", "Projector 5", and "Projector 6" as models (compatible models) that can realize the virtual projection plane 341. Among these, "Projector 1", "Projector 2", and "Projector 5" are models that can realize the virtual projection plane 341 without cutting, and "Projector 6" is a model that can realize the virtual projection plane 341 with cutting. In this case, in the list of compatible models 361, a cutting mark 381 indicating that it is a model that can realize the virtual projection plane 341 with cutting is attached to "Projector 6".
[0221] Thus, the user can understand the models that need to be adjusted and the models that do not need to be adjusted among the models that are suitable for the range in which the projection device is desired to be installed, thereby making it easy for the user to select an appropriate model.
[0222] In this way, in the model search mode, the information processing device 50 outputs the candidate models of the projection device, that is, the suitable model list 361, which are the candidates of the model of the projection device that can project onto the range of the virtual projection plane 341 by setting the projection device within the setting candidate range 351, based on the position and size (range) of the virtual projection plane 341 (projection plane) received from the user through the physical space image 80 (third image) captured by the camera device of the sensor 65 and the designation of the setting candidate range 351 of the projection device.
[0223] Furthermore, in the model search mode, the information processing device 50 may limit the projector's settable range 371 to an overlapping area with the projector's setting candidate range 351 and display it on the physical space image 80. Furthermore, in the model search mode, the information processing device 50 may output the suitable model list 361 together with the use conditions of the projector's candidate (e.g., whether cropping is required).
[0224] Furthermore, the information processing device 50 may perform the processing of the model search mode in combination with the processing of the above-mentioned screen priority mode, projector priority first mode, projector priority second mode, or projector priority third mode. For example, the information processing device 50 may perform the processing of receiving the designation of the model of the projection device from the candidates output in the model search mode instead of the processing of receiving the designation of the model of the first virtual projection device and the second virtual projection device in the above-mentioned screen priority mode, projector priority first mode, projector priority second mode, or projector priority third mode.
[0225] As described above, the information processing device 50 outputs candidates for models of projection devices that can project onto the range of the virtual projection plane 341 by setting the projection device within the setting candidate range 351, i.e., a list of suitable models 361, based on the position and size (range) of the virtual projection plane 341 (projection plane) received from the user based on the physical space image 80 (third image) and the designation of the setting candidate range 351 of the projection device. This can improve the convenience of the user in selecting the model of the projection device to be set.
[0226] (Variation Example)
[0227] Modifications of the above-described embodiments will be described.
[0228] <Variation 1>
[0229] The physical space image 80 representing the physical space is described as a live view image that displays an image captured by the camera device provided in the information processing device 50 in real time, but the physical space image 80 is not limited to this. For example, the physical space image 80 may be a still image captured in the past. Furthermore, the space image 80 may not be an image captured, but may be an image generated based on three-dimensional data representing the structure of the physical space.
[0230] <Variation 2>
[0231] Although the information processing device 50 is described as a tablet terminal having a touch panel 51, the information processing device 50 is not limited to this configuration. For example, the information processing device 50 may be an information terminal such as a smartphone or a personal computer.
[0232] <Variation 3>
[0233] The configuration in which the information processing device 50 displays the second image has been described, but the information processing device 50 may also perform control such that the second image is displayed on another device by sending the generated second image to the other device. In this case, the information processing device 50 may be a server device or the like that does not have a display device.
[0234] <Variation 4>
[0235] Although the physical space image 80 is an image obtained by photographing by the camera of the information processing device 50, the physical space image 80 may be an image obtained by photographing by a device different from the information processing device 50 and received by the information processing device 50 from the device. In this case, the information processing device 50 may be a device without a camera.
[0236] <Variant 5>
[0237] In the projector priority second mode and the projector priority third mode, after displaying the second image, the information processing device 50 can reset at least any one of the setting state of the second virtual projection device 251, the parameters of the second virtual projection device 251 (for example, the lens shift amount or the zoom amount), and the position of the second virtual projection plane 252 when the setting state of the first virtual projection device 151 (for example, whether it is placed on the ground or hung from the ceiling, etc.) or the position of the second virtual projection device 251 is changed by user designation.
[0238] <Variant 6>
[0239] The information processing device 50 may perform control as follows: when displaying the second image, the visual recognition of some objects (the first virtual projection plane 91, the first virtual projection device 151 or the first settable range 92, the second virtual projection device 251 or the second settable range 93) superimposed on the second image is made lower than the visual recognition of other objects. Fig.17 The opacity of the first virtual projection plane 91, the first virtual projection device 151, and the first virtual projection device 151 in the second settable range 93 included in the second image shown is set to be higher than the opacity of the first virtual projection plane 91 and the second settable range 93, and control is performed to reduce the visibility of the first virtual projection device 151. The object whose visibility is reduced among the objects can be selected by, for example, a user operation, or can be automatically selected based on various information that can be acquired by the information processing device 50 (for example, parameters of the projection device, the position of the camera device of the information processing device 50, the order of operations, etc.).
[0240] <Variant 7>
[0241] The case where the second image is a still image including the first imaginary projection plane 91, the first imaginary projection device 151 or the first settable range 92 and the second imaginary projection device 251 or the second settable range 93 has been described, but the second image may also be a dynamic image that alternately displays the first imaginary projection plane 91, the first imaginary projection device 151 or the first settable range 92 and the second imaginary projection device 251 or the second settable range 93.
[0242] In addition, the information processing method described in the above embodiment can be implemented by executing a pre-prepared information processing program by a computer. This information processing program is recorded in a computer-readable storage medium and is executed by reading it from the storage medium. Moreover, this information processing program can be provided in the form of being stored in a non-temporary storage medium such as a flash memory, or can be provided via a network such as the Internet. The computer executing this information processing program can be a computer included in an information processing device, or a computer included in an electronic device such as a smart phone, a tablet terminal or a personal computer that can communicate with the information processing device, or a computer included in a server device that can communicate with these information processing devices and electronic devices.
[0243] The above-described embodiments and modifications may be implemented in combination.
[0244] This manual describes at least the following matters. (1)
[0246] An information processing device comprising a processor, wherein:
[0247] The above processor performs the following processing:
[0248] acquiring first image data representing a first image captured by an imaging device, and
[0249] First virtual projection surface data related to the first virtual projection surface, first virtual projection device data related to the first virtual projection device corresponding to the first virtual projection surface, and second virtual projection device data related to the second virtual projection device;
[0250] Generate, based on the first virtual projection plane data, the first virtual projection device data and the second virtual projection device data, second image data representing a second image in which the first virtual projection plane, the first virtual projection device or the settable range of the first virtual projection device, and the second virtual projection device or the settable range of the second virtual projection device are displayed on the first image; and
[0251] The second image data is output to an output destination. (2)
[0253] The information processing device according to (1), wherein:
[0254] The second image includes the first virtual projection plane, the settable range of the first virtual projection device, and the settable range of the second virtual projection device.
[0255] The above processor performs the following processing:
[0256] The installable range of the first virtual projection device and the installable range of the second virtual projection device are set based on at least one of the position and the size of the first virtual projection plane. (3)
[0258] The information processing device according to (2), wherein:
[0259] The above processor performs the following processing:
[0260] When at least one of the position and the size of the first virtual projection plane is changed, the installable range of the first virtual projection device and the installable range of the second virtual projection device are reset. (4)
[0262] The information processing device according to (1), wherein:
[0263] The second image includes the first virtual projection plane, the first virtual projection device, and the settable range of the second virtual projection device.
[0264] The processor performs the following processing: setting at least one of the position and the size of the first virtual projection surface according to the position of the first virtual projection device,
[0265] The installable range of the second virtual projection device is set based on at least one of the position and the size of the first virtual projection plane. (5)
[0267] The information processing device according to (4), wherein:
[0268] The processor performs the following processing: when the position or parameter of the first virtual projection device is changed, resetting at least one of the position and size of the first virtual projection plane and the installable range of the second virtual projection device. (6)
[0270] The information processing device according to (4) or (5), wherein:
[0271] The above processor performs the following processing:
[0272] When the position of the first virtual projection plane is changed, the installable range of the second virtual projection device is reset. (7)
[0274] The information processing device according to (1), wherein:
[0275] The second image includes the first virtual projection surface, the first virtual projection device, the second virtual projection device, and a second virtual projection surface corresponding to the second virtual projection device.
[0276] The above processor performs the following processing:
[0277] The position of the second virtual projection plane and the position of the second virtual projection device are set according to the position of the first virtual projection device and the position of the first virtual projection plane. (8)
[0279] The information processing device according to (7), wherein:
[0280] The above processor performs the following processing:
[0281] Setting the size of the first imaginary projection plane according to the position of the first imaginary projection device and the position of the first imaginary projection plane;
[0282] Setting the position of the second virtual projection device according to the position of the first virtual projection device; and
[0283] The parameters of the second virtual projection plane related to the position and size of the second virtual projection plane are set according to the position and size of the first virtual projection plane. (9)
[0285] The information processing device according to (7), wherein:
[0286] The above processor performs the following processing:
[0287] Setting the size of the first imaginary projection plane according to the position of the first imaginary projection device and the position of the first imaginary projection plane;
[0288] Setting the position and size of the second imaginary projection plane according to the position and size of the first imaginary projection plane; and
[0289] The parameters of the second virtual projection plane related to the position of the second virtual projection plane are set according to the position of the first virtual projection device. (10)
[0291] The information processing device according to any one of (7) to (9), wherein:
[0292] The above processor performs the following processing:
[0293] An image indicating a range in which at least one of a position and a size of the first virtual projection plane or the second virtual projection plane can be adjusted by changing a parameter of the first virtual projection device or the second virtual projection device is included in the second image. (11)
[0295] The information processing device according to any one of (7) to (10), wherein:
[0296] The above processor performs the following processing:
[0297] When the setting state of the above-mentioned first imaginary projection device or the position of the above-mentioned second imaginary projection device are changed, reset at least any one of the setting state of the above-mentioned second imaginary projection device, the parameter of the above-mentioned second imaginary projection device and the position of the above-mentioned second imaginary projection surface. (12)
[0299] The information processing device according to any one of (7) to (11), wherein:
[0300] The above processor performs the following processing:
[0301] A state in which the second virtual projection device and the second virtual projection surface are set by constraints based on the settings of the first virtual projection device and the first virtual projection surface and a state in which the second virtual projection device and the second virtual projection surface are set without the constraints can be switched. (13)
[0303] The information processing device according to any one of (1) to (12), wherein:
[0304] The above processor performs the following processing:
[0305] outputting, based on the range of the projection surface and the designation of the installation range of the projection device received from the user based on the third image captured by the imaging device, a candidate projection device that can project onto the range of the projection surface by installing the projection device within the installation range,
[0306] The first virtual projection device and the second virtual projection device are virtual projection devices representing the respective projection devices selected by the user from among the projection device candidates. (14)
[0308] The information processing device according to (13), wherein:
[0309] The above processor performs the following processing:
[0310] The installable range of at least one of the first virtual projection device and the second virtual projection device is limited to an overlapping area with the installation range of the projection device and displayed on the first image. (15)
[0312] The information processing device according to (13) or (14), wherein:
[0313] The above processor performs the following processing:
[0314] The projection device candidates are output together with the usage conditions of the projection device candidates. (16)
[0316] The information processing device according to any one of (1) to (15), wherein:
[0317] The above processor performs the following control:
[0318] In the second image, the visual recognizability of some objects in the first virtual projection plane, the first virtual projection device or the settable range of the first virtual projection device, and the second virtual projection device or the settable range of the second virtual projection device is lower than the visual recognizability of other objects. (17)
[0320] An information processing method, wherein:
[0321] The processor of the information processing device performs the following processing:
[0322] acquiring first image data representing a first image captured by an imaging device, and
[0323] First virtual projection surface data related to the first virtual projection surface, first virtual projection device data related to the first virtual projection device corresponding to the first virtual projection surface, and second virtual projection device data related to the second virtual projection device;
[0324] Generate, based on the first virtual projection plane data, the first virtual projection device data and the second virtual projection device data, second image data representing a second image in which the first virtual projection plane, the first virtual projection device or the settable range of the first virtual projection device, and the second virtual projection device or the settable range of the second virtual projection device are displayed on the first image; and
[0325] The second image data is output to an output destination. (18)
[0327] An information processing program is used to cause a processor of an information processing device to perform the following processing:
[0328] acquiring first image data representing a first image captured by an imaging device, and
[0329] First virtual projection surface data related to the first virtual projection surface, first virtual projection device data related to the first virtual projection device corresponding to the first virtual projection surface, and second virtual projection device data related to the second virtual projection device;
[0330] Generate, based on the first virtual projection plane data, the first virtual projection device data and the second virtual projection device data, second image data representing a second image in which the first virtual projection plane, the first virtual projection device or the settable range of the first virtual projection device, and the second virtual projection device or the settable range of the second virtual projection device are displayed on the first image; and
[0331] The second image data is output to an output destination. (19)
[0333] An information processing device comprising a processor, wherein:
[0334] The above processor performs the following processing:
[0335] Based on the range of the projection surface and the designation of the installation range of the projection device received from the user based on the image captured by the imaging device, a projection device candidate that can project onto the range of the projection surface by installing the projection device in the installation range is output. (20)
[0337] An information processing method, wherein:
[0338] The processor of the information processing device performs the following processing:
[0339] Based on the range of the projection surface and the designation of the installation range of the projection device received from the user based on the image captured by the imaging device, a projection device candidate that can project onto the range of the projection surface by installing the projection device in the installation range is output. (twenty one)
[0341] An information processing program is used to cause a processor of an information processing device to perform the following processing:
[0342] Based on the range of the projection surface and the designation of the installation range of the projection device received from the user based on the image captured by the imaging device, a projection device candidate that can project onto the range of the projection surface by installing the projection device in the installation range is output.
[0343] Various embodiments have been described above, but the present invention is certainly not limited to these examples. Obviously, those skilled in the art can think of various variations or modifications within the scope described in the technical solution, and it should be understood that these variations or modifications naturally also fall within the technical scope of the present invention. Furthermore, the constituent elements in the above embodiments can be arbitrarily combined within the scope of the purpose of the invention.
[0344] In addition, this application is based on the Japanese patent application (Japanese Patent Application No. 2022-049515) filed on March 25, 2022, and the contents are incorporated herein by reference.
[0345] Explanation of symbols
[0346] 1-projection unit, 2-operation receiving unit, 2A, 3A-hollow part, 2a, 2b, 3a, 3c, 15a-opening, 4-control device, 4a, 62-memory, 6-projected object, 10-projection device, 11-projection range, 12-light modulation unit, 15-frame, 21-light source, 22-light modulation unit, 23-projection optical system, 24-control circuit, 31-second optical system, 32, 122-reflection component, 33-third optical system, 34-lens, 50-information processing device, 51-touch panel, 61-processor, 63-communication interface, 64-user interface, 65-sensor, 69-bus, 80-physical space image, 81- Floor, 82-wall, 91-1st imaginary projection plane, 92-1st settable range, 93-2nd settable range, 101-main body, 102-1st component, 103-2nd component, 104-projection direction changing mechanism, 105-displacement mechanism, 106-optical unit, 121-1st optical system, 151-1st imaginary projection device, 251-2nd imaginary projection device, 252-2nd imaginary projection plane, 261-lens displacement range, 341-imaginary projection plane, 351-setting candidate range, 361-suitable model list, 371-settable range, 381-cropping mark, G1-image, P1~P4-position, D1~D3-projection distance.
Claims
1. An information processing device comprising a processor, wherein: The processor performs the following processing: Acquire first image data representing a first image captured by an imaging device, first imaginary projection surface data related to a first imaginary projection surface, first imaginary projection device data related to a first imaginary projection device corresponding to the first imaginary projection surface, and second imaginary projection device data related to a second imaginary projection device; generating, based on the first virtual projection plane data, the first virtual projection device data, and the second virtual projection device data, second image data representing a second image obtained by displaying the first virtual projection plane, the first virtual projection device or the settable range of the first virtual projection device, and the second virtual projection device or the settable range of the second virtual projection device on the first image; and The second image data is output to an output destination.
2. The information processing device according to claim 1, wherein: The second image includes the first virtual projection plane, the settable range of the first virtual projection device, and the settable range of the second virtual projection device. The processor performs the following processing: The installable range of the first virtual projection device and the installable range of the second virtual projection device are set based on at least one of the position and the size of the first virtual projection plane.
3. The information processing device according to claim 2, wherein: The processor performs the following processing: When at least one of the position and the size of the first virtual projection plane is changed, the installable range of the first virtual projection device and the installable range of the second virtual projection device are reset.
4. The information processing device according to claim 1, wherein: The second image includes the first virtual projection plane, the first virtual projection device, and an installable range of the second virtual projection device. The processor performs the following processing: at least one of the position and the size of the first virtual projection plane is set according to the position of the first virtual projection device, The installable range of the second virtual projection device is set according to at least one of the position and the size of the first virtual projection plane.
5. The information processing device according to claim 4, wherein: The processor performs the following processing: When the position or parameter of the first virtual projection device is changed, at least one of the position and the size of the first virtual projection plane and the installable range of the second virtual projection device are reset.
6. The information processing device according to claim 4 or 5, wherein: The processor performs the following processing: When the position of the first virtual projection plane is changed, the installable range of the second virtual projection device is reset.
7. The information processing device according to claim 1, wherein: The second image includes the first virtual projection plane, the first virtual projection device, the second virtual projection device, and a second virtual projection plane corresponding to the second virtual projection device. The processor performs the following processing: The position of the second virtual projection plane and the position of the second virtual projection device are set according to the position of the first virtual projection device and the position of the first virtual projection plane.
8. The information processing device according to claim 7, wherein: The processor performs the following processing: Setting the size of the first virtual projection plane according to the position of the first virtual projection device and the position of the first virtual projection plane; Setting the position of the second virtual projection device according to the position of the first virtual projection device; and The parameters of the second virtual projection plane related to the position and size of the second virtual projection plane are set according to the position and size of the first virtual projection plane.
9. The information processing device according to claim 7, wherein: The processor performs the following processing: Setting the size of the first virtual projection plane according to the position of the first virtual projection device and the position of the first virtual projection plane; Setting the position and size of the second imaginary projection plane according to the position and size of the first imaginary projection plane; and The parameters of the second virtual projection plane related to the position of the second virtual projection plane are set according to the position of the first virtual projection device.
10. The information processing device according to any one of claims 7 to 9, wherein: The processor performs the following processing: An image showing a range of at least one of a position and a size of the first virtual projection plane or the second virtual projection plane that can be adjusted by changing parameters of the first virtual projection device or the second virtual projection device is included in the second image.
11. The information processing device according to any one of claims 7 to 10, wherein: The processor performs the following processing: When the setting state of the first virtual projection device or the position of the second virtual projection device is changed, at least one of the setting state of the second virtual projection device, the parameters of the second virtual projection device and the position of the second virtual projection plane is reset.
12. The information processing device according to any one of claims 7 to 11, wherein: The processor is capable of switching between the following states: A state in which the second virtual projection device and the second virtual projection plane are set by constraints based on the settings of the first virtual projection device and the first virtual projection plane, and a state in which the second virtual projection device and the second virtual projection plane are set without the constraints.
13. The information processing device according to any one of claims 1 to 12, wherein: The processor performs the following processing: outputting, based on the range of the projection surface and the designation of the setting range of the projection device received from the user based on the third image captured by the imaging device, a candidate projection device that can project onto the range of the projection surface by setting the setting range of the projection device, The first virtual projection device and the second virtual projection device are virtual projection devices representing respective projection devices selected by a user from among the projection device candidates.
14. The information processing device according to claim 13, wherein: The processor performs the following processing: The installable range of at least one of the first virtual projection device and the second virtual projection device is limited to an overlapping area with the installation range of the projection device and is displayed on the first image.
15. The information processing device according to claim 13 or 14, wherein: The processor performs the following processing: The projection device candidates are output together with the usage conditions of the projection device candidates.
16. The information processing device according to any one of claims 1 to 15, wherein: The processor performs the following control: In the second image, visual recognizability of some objects in the first virtual projection plane, the first virtual projection device or the settable range of the first virtual projection device, and the second virtual projection device or the settable range of the second virtual projection device is lower than that of other objects.
17. An information processing method, wherein: The processor of the information processing device performs the following processing: Acquire first image data representing a first image captured by an imaging device, first imaginary projection surface data related to a first imaginary projection surface, first imaginary projection device data related to a first imaginary projection device corresponding to the first imaginary projection surface, and second imaginary projection device data related to a second imaginary projection device; generating, based on the first virtual projection plane data, the first virtual projection device data, and the second virtual projection device data, second image data representing a second image obtained by displaying the first virtual projection plane, the first virtual projection device or the settable range of the first virtual projection device, and the second virtual projection device or the settable range of the second virtual projection device on the first image; and The second image data is output to an output destination.
18. An information processing program for causing a processor of an information processing device to execute the following processing: Acquire first image data representing a first image captured by an imaging device, first imaginary projection surface data related to a first imaginary projection surface, first imaginary projection device data related to a first imaginary projection device corresponding to the first imaginary projection surface, and second imaginary projection device data related to a second imaginary projection device; Generating, based on the first virtual projection surface data, the first virtual projection device data, and the second virtual projection device data, second image data representing a second image obtained by displaying the first virtual projection surface, the first virtual projection device or the settable range of the first virtual projection device, and the second virtual projection device or the settable range of the second virtual projection device on the first image; and The second image data is output to an output destination.
