Information processing device, information processing method, and information processing program

The information processing apparatus enhances user convenience in selecting and installing projection devices by generating images that display virtual projection devices and their installable ranges, simplifying the installation process.

JP7954194B2Active Publication Date: 2026-11-19FUJIFILM CORP
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2024509917
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2022-03-25
Filing Date
2023-03-03
Publication Date
2026-11-19
Estimated Expiration
2043-03-03

AI Technical Summary

Technical Problem

Existing technologies lack user-friendly solutions for selecting the appropriate model of a projection apparatus during installation, which complicates the process and reduces convenience.

Method used

An information processing apparatus and method that acquires image data, virtual projection plane and device data to generate an image displaying virtual projection devices and their installable ranges, aiding in model selection and installation.

Benefits of technology

Improves user convenience by visually assisting in the selection and placement of projection devices, making the installation process more efficient.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007954194000001
    Figure 0007954194000001
  • Figure 0007954194000002
    Figure 0007954194000002
  • Figure 0007954194000003
    Figure 0007954194000003
Patent Text Reader

Abstract

Provided are an information processing device, an information processing method and an information processing program wherein user's convenience related to selection of the type of a projection device that is to be installed can be improved. On the basis of first virtual projection plane data, the first virtual projection device data and the second virtual projection device data, this information processing device (50) generates second image data that is representative of a second image in which there are displayed in a physical space image (80): a first virtual projection plane (91); a first virtual projection device (151) or a first possible installation range (92) of the first virtual projection device (151); and a second virtual projection device (251) or a second possible installation range (93) of the second virtual projection device (251). The information processing device (50) then outputs the second image data to an output destination.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] Patent Document 1 describes a projected image adjustment system that aims to facilitate the installation and adjustment of a projection display apparatus. Patent Document 2 describes a projection apparatus that aims to reduce the effort required when installing a video projection apparatus to project video. Patent Document 3 describes an information processing apparatus that assists with projector installation, which aims to determine installation conditions for installing a drawn object and a projector that projects a projection image related to the drawn object in accordance with projection image data.

Prior Art Literature

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

[0004] One embodiment according to the technology of the present disclosure provides an information processing apparatus, an information processing method, and an information processing program that can improve user convenience related to selecting the model of a projection apparatus to be installed.

Means for Solving the Problems

[0005] An information processing apparatus according to one aspect of the present invention is an information processing apparatus comprising a processor, wherein the processor acquires first image data representing a first image captured by an imaging device, first virtual projection plane data relating to a first virtual projection plane, first virtual projection device data relating to a first virtual projection device corresponding to the first virtual projection plane, and second virtual projection device data relating to a second virtual projection device, and generates second image data representing a second image in which the first virtual projection plane, the first virtual projection device or the installable range of the first virtual projection device, and the second virtual projection device or the installable range of the second virtual projection device are displayed on the first image, based on the first virtual projection plane data, the first virtual projection device data, and the second virtual projection device data, and outputs the second image data to an output destination.

[0006] An information processing method according to one aspect of the present invention involves a processor of an information processing device acquiring first image data representing a first image captured by an imaging device, first virtual projection plane data relating to a first virtual projection plane, first virtual projection device data relating to a first virtual projection device corresponding to the first virtual projection plane, and second virtual projection device data relating to a second virtual projection device. Based on the first virtual projection plane data, the first virtual projection device data, and the second virtual projection device data, the processor generates second image data representing a second image in which the first virtual projection plane, the first virtual projection device or the installable range of the first virtual projection device, and the second virtual projection device or the installable range of the second virtual projection device are displayed on the first image, and outputs the second image data to an output destination.

[0007] An information processing program according to one aspect of the present invention causes the processor of an information processing device to acquire first image data representing a first image captured by an imaging device, first virtual projection surface data relating to a first virtual projection surface, first virtual projection device data relating to a first virtual projection device corresponding to the first virtual projection surface, and second virtual projection device data relating to a second virtual projection device, and based on the first virtual projection surface data, the first virtual projection device data, and the second virtual projection device data, to generate second image data representing a second image in which the first virtual projection surface, the first virtual projection device or the installable range of the first virtual projection device, and the second virtual projection device or the installable range of the second virtual projection device are displayed on the first image, and to output the second image data to an output destination. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide an information processing device, an information processing method, and an information processing program that can improve user convenience regarding the selection of the model of projection device to be installed. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic diagram showing an example of a projection device 10 that is subject to installation support by the information processing device of the embodiment. [Figure 2] Figure 1 is a schematic diagram showing an example of the internal structure of the projection unit 1. [Figure 3] This is a schematic diagram showing the external configuration of the projection device 10. [Figure 4] Figure 3 is a schematic cross-sectional view of the optical unit 106 of the projection device 10 shown. [Figure 5] This figure shows an example of an information processing device 50 of the embodiment. [Figure 6] This figure shows an example of the hardware configuration of the information processing device 50 of the embodiment. [Figure 7] This flowchart shows an example of processing by the information processing device 50 in screen priority mode. [Figure 8]This is an example (part 1) of an image displayed by the information processing device 50 in the processing shown in Figure 7. [Figure 9] This is an example (part 2) of an image displayed by the information processing device 50 in the processing shown in Figure 7. [Figure 10] This is an example (part 3) of an image displayed by the information processing device 50 in the processing shown in Figure 7. [Figure 11] This is an example (part 4) of an image displayed by the information processing device 50 in the processing shown in Figure 7. [Figure 12] This is an example (part 5) of an image displayed by the information processing device 50 in the processing shown in Figure 7. [Figure 13] This is an example (part 6) of an image displayed by the information processing device 50 in the processing shown in Figure 7. [Figure 14] This is a flowchart showing an example of processing of the information processing device 50 in the projector priority first mode. [Figure 15] This is an example (part 1) of an image displayed by the information processing device 50 in the processing shown in Figure 14. [Figure 16] This is an example (part 2) of an image displayed by the information processing device 50 in the processing shown in Figure 14. [Figure 17] This is an example (part 3) of an image displayed by the information processing device 50 in the processing shown in Figure 14. [Figure 18] This is an example (part 4) of an image displayed by the information processing device 50 in the processing shown in Figure 14. [Figure 19] This is an example (part 5) of an image displayed by the information processing device 50 in the processing shown in Figure 14. [Figure 20] This is a flowchart (part 1) showing an example of processing of the information processing device 50 in the projector priority second mode. [Figure 21] This is a flowchart (part 2) showing an example of processing of the information processing device 50 in the projector priority second mode. [Figure 22] This is an example (part 1) of an image displayed by the information processing device 50 in the processing shown in Figure 20 and Figure 21. [Figure 23]This is an example (part 2) of an image displayed by the information processing device 50 in the processing shown in Figure 20 and Figure 21. [Figure 24] This is an example (part 3) of an image displayed by the information processing device 50 in the processing shown in Figure 20 and Figure 21. [Figure 25] This is an example (part 4) of an image displayed by the information processing device 50 in the processing shown in Figure 20 and Figure 21. [Figure 26] This is an example (part 5) of an image displayed by the information processing device 50 in the processing shown in Figure 20 and Figure 21. [Figure 27] This is a flowchart showing an example of processing of the information processing device 50 in the third projector priority mode. [Figure 28] This is a flowchart showing an example of calculation in step S277 of Figure 27. [Figure 29] This is an example of an image displayed by the information processing device 50 in the processing of Figure 27 in the example of Figure 28. [Figure 30] This is a flowchart showing another example of calculation in step S277 of Figure 27. [Figure 31] This is an example (part 1) of an image displayed by the information processing device 50 in the processing of Figure 27 in the example of Figure 30. [Figure 32] This is an example (part 2) of an image displayed by the information processing device 50 in the processing of Figure 27 in the example of Figure 30. [Figure 33] This is a flowchart showing an example of processing of the information processing device 50 in the model search mode. [Figure 34] This is an example (part 1) of an image displayed by the information processing device 50 in the processing shown in Figure 33. [Figure 35] This is an example (part 2) of an image displayed by the information processing device 50 in the processing shown in Figure 33. [Figure 36] This is an example (part 3) of an image displayed by the information processing device 50 in the processing shown in Figure 33. [Figure 37] This is an example (part 4) of an image displayed by the information processing device 50 in the processing shown in Figure 33. [Figure 38] This is an example (part 5) of an image displayed by the information processing device 50 in the process shown in Figure 33. [Modes for carrying out the invention]

[0010] Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.

[0011] <An example of a projection device 10 whose placement can be changed by the information processing device of the embodiment> Figure 1 is a schematic diagram showing an example of a projection device 10 that is to be supported for installation by the information processing device of the embodiment.

[0012] The information processing device of this embodiment can be used, for example, to assist in the placement of a projection device 10. The projection device 10 comprises a projection unit 1, a control device 4, and an operation reception unit 2. The projection unit 1 is composed of, for example, a liquid crystal projector or a projector using LCOS (Liquid Crystal On Silicon). In the following description, the projection unit 1 will be described as a liquid crystal projector.

[0013] The control device 4 is a control device that controls the projection by the projection device 10. The control device 4 is a device that includes a control unit composed of various processors, a communication interface (not shown) for communicating with each unit, and a memory 4a such as a hard disk, SSD (Solid State Drive), or ROM (Read Only Memory), and it comprehensively controls the projection unit 1.

[0014] The various processors in the control unit of the control device 4 include a CPU (Central Processing Unit), which is a general-purpose processor that executes programs and performs various processes; a Programmable Logic Device (PLD), such as an FPGA (Field Programmable Gate Array), which is a processor whose circuit configuration can be changed after manufacturing; or a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which has a circuit configuration specifically designed to perform a particular process.

[0015] The structure of these various processors is, more specifically, an electrical circuit formed by combining circuit elements such as semiconductor elements. The control unit of the control device 4 may be composed of one of these various processors, or it may be composed of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA).

[0016] The operation reception unit 2 detects user instructions by receiving various operations from the user. The operation reception unit 2 may be a button, key, joystick, etc., provided on the control device 4, or it may be a receiving unit, etc., that receives signals from a remote controller that remotely operates the control device 4.

[0017] The object to be projected onto 6 is an object such as a screen or wall that has a projection surface on which the projected image is displayed by the projection unit 1. In the example shown in Figure 1, the projection surface of the object to be projected onto 6 is a rectangular plane. The top, bottom, left, and right of the object to be projected onto 6 in Figure 1 are assumed to be the top, bottom, left, and right of the actual object to be projected onto 6.

[0018] The projection range 11, shown by the dashed line, is the area of ​​the object to be projected onto 6 that is illuminated by the projection unit 1. In the example shown in Figure 1, the projection range 11 is rectangular. The projection range 11 is part or all of the projectionable range that can be projected by the projection unit 1.

[0019] The projection unit 1, control device 4, and operation reception unit 2 may be implemented by a single device (see, for example, Figures 3 and 4). Alternatively, the projection unit 1, control device 4, and operation reception unit 2 may be separate devices that communicate with each other to cooperate.

[0020] <Internal structure of projection unit 1 shown in Figure 1> Figure 2 is a schematic diagram showing an example of the internal configuration of the projection unit 1 shown in Figure 1.

[0021] As shown in Figure 2, the projection unit 1 comprises a light source 21, a light modulation unit 22, a projection optical system 23, and a control circuit 24.

[0022] The light source 21 includes a light-emitting element such as a laser or an LED (Light Emitting Diode), and emits, for example, white light.

[0023] The light modulation unit 22 is composed of three liquid crystal panels that modulate the red, blue, and green light emitted from the light source 21 and separated into three colors (red, blue, and green) by a color separation mechanism (not shown) based on image information to emit each color image. Alternatively, each of these three liquid crystal panels may be equipped with red, blue, and green filters, and the white light emitted from the light source 21 may be modulated by each liquid crystal panel to emit each color image.

[0024] The projection optical system 23 receives light from the light source 21 and the light modulation unit 22, and is composed of, for example, a relay optical system that includes at least one lens. The light that passes through the projection optical system 23 is projected onto the object to be projected 6.

[0025] Of the object to be projected 6, the area illuminated by light that passes through the entire range of the light modulation unit 22 becomes the projectable range that can be projected by the projection unit 1. Within this projectable range, the area illuminated by light that actually passes through the light modulation unit 22 becomes the projection range 11. For example, by controlling the size, position, and shape of the area through which light passes within the light modulation unit 22, the size, position, and shape of the projection range 11 within the projectable range can be changed.

[0026] The control circuit 24 controls the light source 21, the light modulation unit 22, and the projection optical system 23 based on the display data input from the control device 4, thereby projecting an image based on this display data onto the object to be projected onto 6. The display data input to the control circuit 24 consists of three parts: red display data, blue display data, and green display data.

[0027] Furthermore, the control circuit 24 expands or shrinks the projection range 11 (see Figure 1) of the projection unit 1 by changing the projection optical system 23 based on commands input from the control device 4. Alternatively, the control device 4 may move the projection range 11 of the projection unit 1 by changing the projection optical system 23 based on user operations received by the operation reception unit 2.

[0028] Furthermore, the projection device 10 includes a shift mechanism that mechanically or optically moves the projection range 11 while maintaining the image circle of the projection optical system 23. The image circle of the projection optical system 23 is the region in which the projected light incident on the projection optical system 23 passes through the projection optical system 23 appropriately in terms of light intensity reduction, color separation, and peripheral curvature.

[0029] The shift mechanism is implemented by at least one of the following: an optical shift mechanism that performs optical system shifting, and an electronic shift mechanism that performs electronic shifting.

[0030] The optical system shift mechanism is, for example, a mechanism that moves the projection optical system 23 in a direction perpendicular to the optical axis (see, for example, Figures 3 and 4), or a mechanism that moves the light modulation unit 22 in a direction perpendicular to the optical axis instead of moving the projection optical system 23. Alternatively, the optical system shift mechanism may be a combination of moving the projection optical system 23 and moving the light modulation unit 22.

[0031] The electronic shift mechanism is a mechanism that performs a pseudo-shift of the projection range 11 by changing the range through which light is transmitted in the optical modulation section 22.

[0032] Furthermore, the projection device 10 may include a projection direction changing mechanism that moves the projection range 11 together with the image circle of the projection optical system 23. The projection direction changing mechanism is a mechanism that changes the projection direction of the projection unit 1 by changing the orientation of the projection unit 1 through mechanical rotation (see, for example, Figures 3 and 4).

[0033] <Mechanical configuration of projection device 10> Figure 3 is a schematic diagram showing the external configuration of the projection device 10. Figure 4 is a schematic cross-sectional view of the optical unit 106 of the projection device 10 shown in Figure 3. Figure 4 shows a cross-section along the optical path of the light emitted from the main body 101 shown in Figure 3.

[0034] As shown in Figure 3, the projection device 10 comprises a main body 101 and an optical unit 106 that protrudes from the main body 101. In the configuration shown in Figure 3, the operation reception unit 2, the control device 4, the light source 21, the light modulation unit 22, and the control circuit 24 in the projection unit 1 are provided in the main body 101. The projection optical system 23 in the projection unit 1 is provided in the optical unit 106.

[0035] The optical unit 106 comprises a first member 102 supported by the main body 101 and a second member 103 supported by the first member 102.

[0036] The first member 102 and the second member 103 may be an integrated member. The optical unit 106 may be configured to be detachably attached to the main body 101 (in other words, to be replaceable).

[0037] The main body 101 has a housing 15 (see Figure 4) in which an opening 15a (see Figure 4) for allowing light to pass through is formed in the portion connected to the optical unit 106.

[0038] Inside the housing 15 of the main unit 101, as shown in Figure 3, there is a light source 21 and an optical modulation unit 12 which includes an optical modulation unit 22 (see Figure 2) that spatially modulates the light emitted from the light source 21 based on input image data to generate an image.

[0039] Light emitted from the light source 21 is incident on the light modulation section 22 of the light modulation unit 12, spatially modulated by the light modulation section 22, and then emitted.

[0040] As shown in Figure 4, the image formed by the light spatially modulated by the optical modulation unit 12 passes through the aperture 15a of the housing 15 and enters the optical unit 106, where it is projected onto the object to be projected, 6, making the image G1 visible to the observer.

[0041] As shown in Figure 4, the optical unit 106 comprises a first member 102 having a hollow section 2A connected to the inside of the main body 101, a second member 103 having a hollow section 3A connected to the hollow section 2A, a first optical system 121 and a reflective member 122 arranged in the hollow section 2A, a second optical system 31, a reflective member 32, a third optical system 33, and a lens 34 arranged in the hollow section 3A, a shift mechanism 105, and a projection direction changing mechanism 104.

[0042] The first member 102 is a member whose cross-sectional shape is, for example, rectangular, and has openings 2a and 2b formed on mutually perpendicular surfaces. The first member 102 is supported by the main body 101 with opening 2a positioned opposite opening 15a of the main body 101. Light emitted from the light modulation section 22 of the light modulation unit 12 of the main body 101 is incident on the hollow section 2A of the first member 102 through openings 15a and 2a.

[0043] The direction of incidence of light entering the hollow section 2A from the main body 101 is described as direction X1, the opposite direction of direction X1 is described as direction X2, and directions X1 and X2 are collectively referred to as direction X. In Figure 4, the direction from the front to the back of the paper and the opposite direction are described as direction Z. Of the directions Z, the direction from the front to the back of the paper is described as direction Z1, and the direction from the back to the front of the paper is described as direction Z2.

[0044] Furthermore, the direction perpendicular to directions X and Z is denoted as direction Y, and within direction Y, the upward direction in Figure 4 is denoted as direction Y1, and the downward direction in Figure 4 is denoted as direction Y2. In the example in Figure 4, the projection device 10 is positioned such that direction Y2 is the vertical direction.

[0045] The projection optical system 23 shown in Figure 2 consists of a first optical system 121, a reflector 122, a second optical system 31, a reflector 32, a third optical system 33, and a lens 34. Figure 4 shows the optical axis K of this projection optical system 23. The first optical system 121, the reflector 122, the second optical system 31, the reflector 32, the third optical system 33, and the lens 34 are arranged along the optical axis K in this order from the light modulation unit 22 side.

[0046] The first optical system 121 includes at least one lens and guides light traveling in direction X1 incident from the main body 101 to the first member 102 to the reflecting member 122.

[0047] The reflective member 122 reflects light incident from the first optical system 121 in direction Y1. The reflective member 122 is made of, for example, a mirror. The first member 102 has an aperture 2b formed in the optical path of the light reflected by the reflective member 122, and this reflected light passes through the aperture 2b and proceeds to the hollow portion 3A of the second member 103.

[0048] The second member 103 has a roughly T-shaped cross-section, with an opening 3a formed at a position facing the opening 2b of the first member 102. Light from the main body 101 that has passed through the opening 2b of the first member 102 enters the hollow portion 3A of the second member 103 through this opening 3a. Note that the cross-sectional shapes of the first member 102 and the second member 103 are arbitrary and not limited to those described above.

[0049] The second optical system 31 includes at least one lens and guides the light incident from the first member 102 to the reflecting member 32.

[0050] The reflective member 32 reflects the light incident from the second optical system 31 in direction X2 and guides it to the third optical system 33. The reflective member 32 is made up of, for example, a mirror.

[0051] The third optical system 33 includes at least one lens and guides the light reflected by the reflective member 32 to the lens 34.

[0052] The lens 34 is positioned at the end of the second member 103 so as to block the opening 3c formed at the end on the direction X2 side. The lens 34 projects light incident from the third optical system 33 onto the object to be projected 6.

[0053] The projection direction changing mechanism 104 is a rotation mechanism that rotatably connects the second member 103 to the first member 102. This projection direction changing mechanism 104 allows the second member 103 to rotate freely around a rotation axis (specifically, the optical axis K) extending in direction Y. The projection direction changing mechanism 104 only needs to be able to rotate the optical system and is not limited to the arrangement shown in Figure 4. Furthermore, the number of rotation mechanisms is not limited to one; multiple mechanisms may be provided.

[0054] The shift mechanism 105 is a mechanism for moving the optical axis K of the projection optical system (in other words, the optical unit 106) in a direction perpendicular to the optical axis K (direction Y in Figure 4). Specifically, the shift mechanism 105 is configured to change the position of the first member 102 in direction Y relative to the main body 101. The shift mechanism 105 may be used to move the first member 102 manually or electrically.

[0055] Figure 4 shows the state in which the first member 102 has been moved to its maximum extent toward direction Y1 by the shift mechanism 105. From the state shown in Figure 4, the first member 102 moves toward direction Y2 by the shift mechanism 105, which changes the relative position between the center of the image formed by the light modulation unit 22 (in other words, the center of the display surface) and the optical axis K, thereby shifting (translating) the image G1 projected onto the object to be projected 6 toward direction Y2.

[0056] Alternatively, the shift mechanism 105 may move the optical modulation unit 22 in the direction Y instead of moving the optical unit 106 in the direction Y. In this case as well, the image G1 projected onto the object 6 can be moved in the direction Y2.

[0057] <Information processing device 50 of the embodiment> Figure 5 shows an example of an information processing device 50 of the embodiment. The information processing device 50 of the embodiment is a tablet terminal having a touch panel 51. The touch panel 51 is a touch-operable display. For example, a user of the information processing device 50 brings the information processing device 50 into a space (e.g., a room) where a projection device 10 will be installed and projection will be performed. The information processing device 50 displays installation support images on the touch panel 51 to assist in the installation of the projection device 10 in that space.

[0058] Specifically, the information processing device 50 displays a second image by superimposing a first virtual projection surface, which is a virtual projection surface, the first virtual projection device or the first possible installation range of the first virtual projection device, and the second virtual projection device or the second possible installation range of the second virtual projection device, which is a second virtual projection device, onto a first image obtained by imaging the space in which the projection device 10 is installed and projected. This allows the user to consider the installation of the projection device 10 while comparing the first virtual projection device and the second virtual projection device. The first virtual projection device and the second virtual projection device virtually represent candidate projection devices for the projection device 10. The user can specify projection devices of different models for the first virtual projection device and the second virtual projection device. Alternatively, the user may specify projection devices from different manufacturers for the first virtual projection device and the second virtual projection device.

[0059] <Hardware configuration of the information processing device 50> Figure 6 shows an example of the hardware configuration of the information processing device 50 in the embodiment. The information processing device 50 shown in Figure 5 includes, for example, a processor 61, a memory 62, a communication interface 63, a user interface 64, and a sensor 65, as shown in Figure 6. The processor 61, memory 62, communication interface 63, user interface 64, and sensor 65 are connected, for example, by a bus 69.

[0060] The processor 61 is a circuit that performs signal processing, and is, for example, a CPU that controls the entire information processing device 50. The processor 61 may also be implemented using other digital circuits such as an FPGA or a DSP (Digital Signal Processor). Furthermore, the processor 61 may be implemented using a combination of multiple digital circuits.

[0061] Memory 62 includes, for example, main memory and auxiliary memory. Main memory is, for example, RAM (Random Access Memory). Main memory is used as the work area of ​​processor 61.

[0062] Auxiliary memory is a non-volatile memory such as a magnetic disk or flash memory. Various programs for operating the information processing device 50 are stored in the auxiliary memory. The programs stored in the auxiliary memory are loaded into main memory and executed by the processor 61.

[0063] Furthermore, the auxiliary memory may include portable memory that can be removed from the information processing device 50. Portable memory may include memory cards such as USB (Universal Serial Bus) flash drives and SD (Secure Digital) memory cards, or external hard disk drives.

[0064] The communication interface 63 is a communication interface that communicates with devices outside the information processing device 50. The communication interface 63 includes at least one of a wired communication interface that communicates via wires and a wireless communication interface that communicates via wireless. The communication interface 63 is controlled by the processor 61.

[0065] The user interface 64 includes, for example, an input device that accepts user input and an output device that outputs information to the user. The input device can be implemented by, for example, a key (e.g., a keyboard) or a remote control. The output device can be implemented by, for example, a display or a speaker. In the information processing device 50 shown in Figure 5, the input device and output device are implemented by a touch panel 51. The user interface 64 is controlled by the processor 61.

[0066] The sensor 65 includes imaging devices that have an imaging optical system and an image sensor and are capable of taking images, and spatial recognition sensors that can recognize the space around the information processing device 50 in three dimensions. The imaging device includes, for example, the imaging device provided on the back surface of the information processing device 50 shown in Figure 5.

[0067] One example of a spatial recognition sensor is LIDAR (Light Detection and Ranging), which emits laser light, measures the time it takes for the laser light to hit an object and reflect back, and then determines the distance and direction to the object. However, spatial recognition sensors are not limited to this; they can also include various other sensors such as radar that emits radio waves or ultrasonic sensors that emit ultrasonic waves.

[0068] The information processing device 50 performs at least one of the following processes: screen priority mode, projector priority first mode, projector priority second mode, projector priority third mode, and model search mode.

[0069] (Screen priority mode) Figure 7 is a flowchart illustrating an example of processing by the information processing device 50 in screen priority mode. Figures 8 to 13 are examples of images displayed by the information processing device 50 in the processing shown in Figure 7. For example, at the start of the processing shown in Figure 7, the information processing device 50 displays the physical space image 80 shown in Figure 8. The physical space image 80 is a through image (live view image) that shows in real time an image obtained by imaging by an imaging device provided as a sensor 65 in the information processing device 50. The physical space image 80 is an example of the first image of the present invention.

[0070] Furthermore, the information processing device 50 recognizes the physical space shown by the physical space image 80 in three dimensions. For this spatial recognition, for example, a spatial recognition sensor included in the sensor 65 of the information processing device 50 is used. The physical space shown by the physical space image 80 includes a floor 81 and walls 82. Here, it is assumed that the projection device 10 is installed on the floor 81 and projected onto the wall 82.

[0071] First, the information processing device 50 receives the user's specification of the position and size of the first virtual projection plane (step S71). The first virtual projection plane virtually represents, for example, the projection range 11 described above. The position of the first virtual projection plane is, for example, the center position of the first virtual projection plane (the position of the projection center). For example, the position of the first virtual projection plane is specified as a two-dimensional coordinate of the physical space image 80 by the user indicating a point in the physical space image 80 as the center position of the first virtual projection plane. The information processing device 50 accepts the specified two-dimensional coordinate as a three-dimensional position of the first virtual projection plane by converting it into a three-dimensional coordinate based on the recognition result of the physical space shown by the physical space image 80. The size of the first virtual projection plane is specified using an actual distance in physical space, for example, the length of the diagonal = x [inch].

[0072] Next, the information processing device 50 displays the first virtual projection plane superimposed on the physical space image 80 based on the specification received in step S71 (step S72). Specifically, the information processing device 50 calculates the three-dimensional region of the first virtual projection plane in the physical space shown by the physical space image 80, converts that three-dimensional region into a two-dimensional region corresponding to the physical space image 80 based on the recognition result of the physical space shown by the physical space image 80, and superimposes the image showing the converted region onto the physical space image 80 as the first virtual projection plane. For example, as shown in Figure 9, the information processing device 50 displays an image in which the first virtual projection plane 91 is superimposed on the physical space image 80.

[0073] Next, the information processing device 50 receives a designation from the user for the model of the first virtual projection device from among several options for the model of the projection device 10 (step S73). Next, the information processing device 50 calculates the position and size of the first installable range, which is represented by a polyhedron, for example, based on the current position and size of the first virtual projection surface 91 and the projection ratio and lens shift amount that can be set for the model of the first virtual projection device that was designated in step S73, and displays the first installable range of the calculated position and size superimposed on the physical space image 80 (step S74).

[0074] Specifically, the information processing device 50 calculates the three-dimensional region of the first installable range in the physical space shown by the physical space image 80, converts that three-dimensional region into a two-dimensional region corresponding to the physical space image 80 based on the recognition result of the physical space shown by the physical space image 80, and superimposes the image showing the converted region onto the physical space image 80 as the first installable range. For example, as shown in Figure 10, the information processing device 50 displays an image in which the first virtual projection plane 91 and the first installable range 92 are superimposed on the physical space image 80.

[0075] Next, the information processing device 50 receives a designation from the user for the model of the second virtual projection device from among several options for the model of the projection device 10 (step S75). Then, based on the current position and size of the first virtual projection surface 91 and the projection ratio and lens shift amount that can be set for the model of the second virtual projection device designated in step S75, the information processing device 50 calculates the position and size of the second installable range, which is represented, for example, as a polyhedron, and displays the second installable range of the calculated position and size superimposed on the physical space image 80 (step S76).

[0076] Specifically, the information processing device 50 calculates the three-dimensional region of the second installable range in the physical space shown by the physical space image 80, converts that three-dimensional region into a two-dimensional region corresponding to the physical space image 80 based on the recognition result of the physical space shown by the physical space image 80, and superimposes the image showing the converted region onto the physical space image 80 as the second installable range. For example, as shown in Figure 11, the information processing device 50 displays an image in which the first virtual projection plane 91, the first installable range 92, and the second installable range 93 are superimposed on the physical space image 80.

[0077] This allows users to visually compare the installable ranges (e.g., first installable range 92 and second installable range 93) of projection devices with different parameter setting ranges (e.g., first virtual projection device and second virtual projection device) relative to a desired projection surface (e.g., first virtual projection surface 91), making it easier to select the appropriate projection device model.

[0078] Next, the information processing device 50 determines whether or not it has received an instruction from the user to change the size of the first virtual projection surface 91 (step S77). If it has received an instruction to change the size of the first virtual projection surface 91 (step S77: Yes), the information processing device 50 changes the size of the first virtual projection surface 91 and displays the resized first virtual projection surface 91 superimposed on the physical space image 80 (step S78).

[0079] Furthermore, the information processing device 50 recalculates the position and size of the first installable range 92 based on the current position and size of the first virtual projection surface 91 and the projection ratio and lens shift amount that can be set for the model designated as the first virtual projection device in step S73. The information processing device 50 also recalculates the position and size of the second installable range 93 based on the current position and size of the first virtual projection surface 91 and the projection ratio and lens shift amount that can be set for the model designated as the second virtual projection device in step S75. Then, the information processing device 50 displays the recalculated positions and sizes of the first installable range 92 and the second installable range 93 superimposed on the physical space image 80 (step S79), and returns to step S77.

[0080] This allows the user to visually understand the changes in the position and size of the installable ranges of the first installable range 92 and the second installable range 93 in response to a change in the size of the first virtual projection plane 91, making it easier to select an appropriate model. For example, suppose a change instruction is given to change the size of the first virtual projection plane 91 from x [inch] to y [inch] (x > y). In this case, as shown in Figure 12, for example, the first installable range 92 and the second installable range 93 shrink in conjunction with the reduction of the first virtual projection plane 91, and the distance of the first installable range 92 and the second installable range 93 from the first virtual projection plane 91 decreases.

[0081] If, in step S77, the information processing device 50 has not received an instruction to change the size of the first virtual projection plane 91 (step S77: No), then the information processing device 50 determines whether or not it has received an instruction to change the position of the first virtual projection plane 91 (step S80). If the information processing device 50 has received an instruction to change the position of the first virtual projection plane 91 (step S80: Yes), then the information processing device 50 changes the position of the first virtual projection plane 91 and displays the repositioned first virtual projection plane 91 superimposed on the physical space image (step S81).

[0082] Furthermore, the information processing device 50, similar to step S79, recalculates the positions and sizes of the first installable range 92 and the second installable range 93, overlays the recalculated positions and sizes of the first installable range 92 and the second installable range 93 onto the physical space image 80 and displays them (step S82), and returns to step S77.

[0083] This allows the user to visually understand the changes in the position and size of the installable ranges of the first installable range 92 and the second installable range 93 in response to a change in the position of the first virtual projection plane 91, making it easier to select the appropriate model. For example, suppose a change instruction is given to move the position of the first virtual projection plane 91 to the left toward the first virtual projection plane 91. In this case, as shown in Figure 13, for example, the first installable range 92 and the second installable range 93 move to the left in conjunction with the leftward movement of the first virtual projection plane 91.

[0084] If no instruction to change the position of the first virtual projection plane 91 has been received in step S80 (step S80: No), the information processing device 50 returns to step S77.

[0085] Thus, in screen priority mode, the information processing device 50 displays a second image that includes the first virtual projection surface 91, the first installable range 92 of the first virtual projection device, and the second installable range 93 of the second virtual projection device. The information processing device 50 also sets the first installable range 92 of the first virtual projection device and the second installable range 93 of the second virtual projection device based on the position and size of the first virtual projection surface 91. Alternatively, the information processing device 50 may set the first installable range 92 of the first virtual projection device and the second installable range 93 of the second virtual projection device based on either the position or size of the first virtual projection surface 91.

[0086] Furthermore, in screen priority mode, if at least one of the position and size of the first virtual projection surface 91 is changed, the information processing device 50 resets the first installable range 92 of the first virtual projection device and the second installable range 93 of the second virtual projection device.

[0087] (Projector priority mode 1) Figure 14 is a flowchart showing an example of processing by the information processing device 50 in the projector priority first mode. Figures 15 to 19 are examples of images displayed by the information processing device 50 in the processing shown in Figure 14. Similar to the screen priority mode, the information processing device 50 displays the physical space image 80 shown in Figure 8 at the start of the processing shown in Figure 14. The information processing device 50 also recognizes the physical space shown by the physical space image 80 in three dimensions.

[0088] First, the information processing device 50 receives a specification from the user for the position of the first virtual projection device and the position of the first virtual projection plane 91 (the position of the projection center) (step S141). The position of the first virtual projection device is, for example, the position of the front element of the projection lens of the first virtual projection device. For example, the information processing device 50 receives a specification for the position P1 of the first virtual projection device and the position P2 of the first virtual projection plane 91 as shown in Figure 15. Based on the specified positions P1 and P2, the information processing device 50 calculates the projection distance D1, which is the distance between positions P1 and P2.

[0089] Next, the information processing device 50 receives a specification from the user for the model of the first virtual projection device from among several options for the model of the projection device 10 (step S142). Next, the information processing device 50 calculates the size of the first virtual projection surface 91 based on the projection distance D1 and the projection ratio corresponding to the model of the first virtual projection device specified in step S142 (step S143).

[0090] Furthermore, the information processing device 50 superimposes and displays the first virtual projection device 151 and the first virtual projection plane 91, which are three-dimensional models of the first virtual projection device specified in step S142, onto the physical space image 80 (step S144). For example, as shown in Figure 16, the information processing device 50 displays an image in which the first virtual projection device 151 and the first virtual projection plane 91 are superimposed on the physical space image 80.

[0091] The first virtual projection device 151 is positioned such that the front element of the projection lens of the first virtual projection device 151 is at position P1 shown in Figure 15. The first virtual projection plane 91 is positioned such that the center of the first virtual projection plane 91 is at position P2 shown in Figure 15. The size of the first virtual projection plane 91 is the size calculated in step S143.

[0092] Next, the information processing device 50 accepts the designation of a second virtual projection device model from among several options for the projection device 10 models (step S145). Then, based on the position and size of the first virtual projection surface 91 and the projection ratio and lens shift amount that can be set for the model designated as the second virtual projection device in step S145, the information processing device 50 calculates the position and size of the second installable range 93 in which the second virtual projection device can be installed, and displays the second installable range 93 with its calculated position and size superimposed on the physical space image 80 (step S146). For example, as shown in Figure 17, the information processing device 50 displays an image in which the first virtual projection device 151, the first virtual projection surface 91, and the second installable range 93 are superimposed on the physical space image 80.

[0093] This allows the user to visually grasp the second installable range 93 of the second virtual projection device to be compared with the first virtual projection device 151 that has been temporarily virtually placed, making it easier to select an appropriate model.

[0094] Next, the information processing device 50 determines whether or not it has received an instruction from the user to change the position of the first virtual projection device 151 (step S147). If it has received an instruction to change the position of the first virtual projection device 151 (step S147: Yes), the information processing device 50 changes the position of the first virtual projection device 151 and displays the repositioned first virtual projection device 151 superimposed on the physical space image 80 (step S148).

[0095] Furthermore, the information processing device 50 recalculates the position and size of the first virtual projection surface 91 based on the current position of the first virtual projection device 151 and the projection ratio and lens shift amount that can be set for the model of the first virtual projection device specified in step S142. Then, the information processing device 50 overlays the recalculated first virtual projection surface 91 of the recalculated position and size onto the physical space image 80 and displays it (step S149).

[0096] Next, the information processing device 50 recalculates the position and size of the second installable range 93 based on the recalculated position and size of the first virtual projection plane 91 and the projection ratio and lens shift amount that can be set for the model of the second virtual projection device that was designated in step S145. Then, the information processing device 50 displays the recalculated position and size of the second installable range 93 superimposed on the physical space image 80 (step S150) and returns to step S147.

[0097] This allows the user to visually understand the changes in the position and size of the second installable range 93 of the comparison model (second virtual projection device) in response to a change in the size of the first virtual projection surface 91, making it easier to select the appropriate model. For example, suppose a change instruction is given to move the first virtual projection device 151 closer to the first virtual projection surface 91. In this case, as shown in Figure 18, for example, the first virtual projection surface 91 shrinks in conjunction with the movement of the first virtual projection device 151 toward the first virtual projection surface 91, the second installable range 93 moves toward the first virtual projection surface 91, and the second installable range 93 shrinks.

[0098] If, in step S147, the information processing device 50 has not received an instruction to change the position of the first virtual projection device 151 (step S147: No), then the information processing device 50 determines whether or not it has received an instruction from the user to change the position of the first virtual projection surface 91 (step S151). If the information processing device 50 has received an instruction to change the position of the first virtual projection surface 91 (step S151: Yes), then the information processing device 50 changes the position of the first virtual projection surface 91 and displays the repositioned first virtual projection surface 91 superimposed on the physical space image 80 (step S152).

[0099] Next, the information processing device 50 recalculates the position and size of the second installable range 93 based on the current position and size of the first virtual projection plane 91 and the projection ratio and lens shift amount that can be set for the model of the second virtual projection device that was specified in step S145. Then, the information processing device 50 displays the recalculated second installable range 93 with its position and size superimposed on the physical space image 80 (step S153), and returns to step S147.

[0100] This allows the user to visually understand the change in the position of the second installable range 93 of the comparison model (second virtual projection device) in response to a change in the position of the first virtual projection plane 91, making it easier to select the appropriate model. For example, suppose a change instruction is given to move the position of the first virtual projection plane 91 to the left toward the first virtual projection plane 91. In this case, as shown in Figure 19, for example, the second installable range 93 moves to the left in conjunction with the movement of the first virtual projection plane 91 to the left.

[0101] If no instruction to change the position of the first virtual projection plane 91 has been received in step S151 (step S151: No), the information processing device 50 returns to step S147.

[0102] Furthermore, the information processing device 50 may receive a user instruction to change the projection ratio of the first virtual projection device 151 after step S147. For example, if the information processing device 50 receives a change instruction to reduce the projection ratio of the first virtual projection device 151, it will reduce the size of the first virtual projection surface 91 while maintaining the position of the first virtual projection device 151. In this case, the information processing device 50 will move the second installable range 93 toward the first virtual projection surface 91 and reduce the size of the second installable range 93.

[0103] Thus, in the projector-priority first mode, the information processing device 50 displays a second image that includes the first virtual projection surface 91, the first virtual projection device 151, and the second installable range 93 of the second virtual projection device. The information processing device 50 also sets at least one of the position and size of the first virtual projection surface 91 based on the position of the first virtual projection device 151, and sets the second installable range 93 of the second virtual projection device based on at least one of the set position and size of the first virtual projection surface 91.

[0104] Furthermore, in the projector priority first mode, if the position or parameters of the first virtual projection device 151 are changed, the information processing device 50 resets at least one of the position and size of the first virtual projection surface 91 and the second installable range 93 of the second virtual projection device. Also, if the position of the first virtual projection surface 91 is changed, the information processing device 50 resets the second installable range 93 of the second virtual projection device.

[0105] (Projector priority mode 2) Figures 20 and 21 are flowcharts illustrating an example of processing by the information processing device 50 in the projector-priority second mode. Figures 22 to 26 are examples of images displayed by the information processing device 50 in the processing shown in Figures 20 and 21. Similar to the screen-priority mode, the information processing device 50 displays the physical space image 80 shown in Figure 8 at the start of the processing shown in Figure 20. The information processing device 50 also recognizes the physical space represented by the physical space image 80 in three dimensions.

[0106] First, the information processing device 50 receives a specification from the user for the position of the first virtual projection device 151 and the position of the first virtual projection plane 91 (the position of the projection center), similar to step S141 shown in Figure 14 (step S201). For example, the information processing device 50 receives a specification for the position P1 of the first virtual projection device and the position P2 of the first virtual projection plane 91, as shown in Figure 15.

[0107] Next, the information processing device 50 receives a specification from the user for the model of the first virtual projection device from among several options for the model of the projection device 10, similar to step S142 shown in Figure 14 (step S202).

[0108] Next, the information processing device 50 calculates the position of the projection center when no lens shift is performed in the first virtual projection device 151, based on the position of the first virtual projection device 151 specified in step S201 (step S203). For example, as shown in Figure 22, the information processing device 50 calculates the position P3 of the projection center when no lens shift is performed in the first virtual projection device 151 (when the lens shift amount = 0). The information processing device 50 also calculates the projection distance D2, which is the distance between positions P1 and P3.

[0109] Next, the information processing device 50 calculates the size of the first virtual projection surface 91 based on the projection distance D2 and the projection ratio corresponding to the model of the first virtual projection device specified in step S202 (step S204). For example, the information processing device 50 calculates the size of the projection surface when the first virtual projection device 151 projects to position P3 without performing a lens shift as the size of the first virtual projection surface 91.

[0110] Next, the information processing device 50 projects the first virtual projection device 151 and the first virtual projection surface 91 onto the physical space image 80 and displays them (step S205). For example, as shown in Figure 23, the information processing device 50 displays an image in which the first virtual projection device 151 and the first virtual projection surface 91 are superimposed on the physical space image 80. The first virtual projection device 151 is a three-dimensional model of the model that was designated as the model of the first virtual projection device in step S202.

[0111] The first virtual projection device 151 is positioned such that the front element of the projection lens of the first virtual projection device 151 is at position P1 shown in Figure 22. The first virtual projection plane 91 is positioned such that the center of the first virtual projection plane 91 is at position P2 shown in Figure 22. The size of the first virtual projection plane 91 is the size calculated in step S204.

[0112] Furthermore, as shown in Figure 24, the information processing device 50 sets up a coordinate system in which the axis perpendicular to the bottom surface of the first virtual projection device 151 is the Y-axis, the projection direction of the first virtual projection device 151 is the Z-axis, and the axis perpendicular to the Y-axis and Z-axis is the X-axis.

[0113] Next, the information processing device 50 accepts the designation of a second virtual projection device model from among several options for the projection device 10 models (step S206). Then, based on the specifications of the model designated as the second virtual projection device in step S206, the information processing device 50 sets the position of the second virtual projection device to the position where the evaluation value using the distance between the first virtual projection device 151 and the second virtual projection device is minimized (step S207).

[0114] This evaluation value decreases as the distance between the first virtual projection device 151 and the second virtual projection device decreases. The distance between the first virtual projection device 151 and the second virtual projection device may be the distance between the lens positions (e.g., the position of the front element) of the first virtual projection device 151 and the second virtual projection device, or the distance between the centers of the base surfaces of the first virtual projection device 151 and the second virtual projection device. Here, it is assumed that the above evaluation value is minimized at the position where the Y coordinates of the base surfaces of the first virtual projection device 151 and the second virtual projection device coincide, and the X and Z coordinates of the lens positions of the first virtual projection device 151 and the second virtual projection device coincide. Alternatively, it may be simply assumed that the evaluation value is minimized when the X, Y, and Z coordinates of the lenses of the first virtual projection device 151 and the second virtual projection device coincide.

[0115] Next, the information processing device 50 calculates the position of the projection center when no lens shift is performed in the second virtual projection device, similar to position P3, based on the specifications of the model of the second virtual projection device that was received as the model in step S206 and the position of the second virtual projection device set in step S207 (step S208). For example, the information processing device 50 calculates the position of the projection center P4 when no lens shift is performed in the second virtual projection device (when the lens shift amount = 0). The information processing device 50 also calculates the projection distance D3, which is the distance between the position of the second virtual projection device set in step S207 and position P4.

[0116] Next, the information processing device 50 calculates the size of the second virtual projection plane by the second virtual projection device in the same manner as the calculation of the size of the first virtual projection plane 91 in step S204 (step S209). For example, the information processing device 50 calculates the size of the projection plane when the second virtual projection device projects to position P4 without performing a lens shift as the size of the second virtual projection plane.

[0117] Furthermore, if the second virtual projection device has a zoom function, the information processing device 50 sets the projection ratio of the second virtual projection device so that the size of the second virtual projection surface is closest to the size of the first virtual projection surface 91. Also, based on the specifications of the model of the second virtual projection device that was designated in step S206, the information processing device 50 sets the lens shift amount of the second virtual projection device so that the distance between the center position of the first virtual projection surface 91 and the center position of the second virtual projection surface is minimized (step S210).

[0118] Next, the information processing device 50 displays the second virtual projection device and the second virtual projection plane superimposed on the physical space image 80 (step S211). For example, as shown in Figure 25, the information processing device 50 displays an image in which the first virtual projection device 151, the first virtual projection plane 91, the second virtual projection device 251, and the second virtual projection plane 252 are superimposed on the physical space image 80. The second virtual projection device 251 is a three-dimensional model of the model that was designated as the model of the second virtual projection device in step S206.

[0119] The second virtual projection device 251 is positioned such that the front element of the projection lens of the second virtual projection device 251 is at the position of the second virtual projection device set in step S207. The second virtual projection plane 252 is positioned such that the center position of the second virtual projection plane 252 is shifted relative to the center position of the second virtual projection plane calculated in step S208, based on the lens shift amount set in step S210. The size of the second virtual projection plane 252 is the size calculated in step S209.

[0120] This allows users to visually understand the projection status without having to readjust the position of the virtual projection device or virtual projection surface even when changing models, making it easier to select the appropriate model.

[0121] The information processing device 50 may display the range in which the position and size of the first virtual projection plane 91 and the second virtual projection plane 252 can be adjusted by changing the parameters of the first virtual projection device 151 and the second virtual projection device 251. For example, as shown in Figure 26, the information processing device 50 displays the lens shift range 261 of the second virtual projection device 251 when the size of the first virtual projection plane 91 is closest to the size of the second virtual projection plane 252. The lens shift range 261 is a range centered on the center position of the second virtual projection plane calculated in step S208.

[0122] This allows users to visually understand the configurable range of the virtual projection surface when changing models, making it easier to select the appropriate model.

[0123] Next, the information processing device 50 determines whether or not it has received an instruction from the user to change the position of the first virtual projection device 151 (step S212). If it has received an instruction to change the position of the first virtual projection device 151 (step S212: Yes), the information processing device 50 changes the position of the first virtual projection device 151 and displays the repositioned first virtual projection device 151 superimposed on the physical space image 80 (step S213).

[0124] Furthermore, the information processing device 50 recalculates the position and size of the first virtual projection surface 91 based on the current position of the first virtual projection device 151 and the projection ratio and lens shift amount that can be set for the model of the first virtual projection device specified in step S142. Then, the information processing device 50 overlays the recalculated first virtual projection surface 91 of the recalculated position and size onto the physical space image 80 and displays it (step S214).

[0125] Next, the information processing device 50 recalculates the position of the second virtual projection device 251 so that its distance from the first virtual projection device 151 is minimized. The information processing device 50 also recalculates the parameters of the second virtual projection device 251 (e.g., lens shift amount and zoom amount) so that the position and size of the second virtual projection surface 252 are minimized compared to the position and size of the first virtual projection surface 91 recalculated in step S214. Then, the information processing device 50 superimposes the second virtual projection device 251 and the second virtual projection surface 252 based on these recalculation results onto the physical space image 80 and displays them (step S215), and returns to step S212.

[0126] This allows the user to visually understand the projection status of the comparison model (second virtual projection device 251) without performing any operations other than changing the position of the first virtual projection device 151, making it easier to select the appropriate model.

[0127] If, in step S212, the information processing device 50 has not received an instruction to change the position of the first virtual projection device 151 (step S212: No), the information processing device 50 determines whether or not it has received an instruction from the user to change the zoom amount of the first virtual projection device 151 (step S216). If the information processing device 50 has received an instruction to change the zoom amount of the first virtual projection device 151 (step S212: Yes), the information processing device 50 recalculates the size of the first virtual projection surface 91 based on the changed zoom amount of the first virtual projection device 151, i.e., the projection ratio, and displays the first virtual projection surface 91 of the recalculated size superimposed on the physical space image 80 (step S217).

[0128] Next, the information processing device 50 recalculates the parameters (e.g., zoom amount) of the second virtual projection device 251 so that the difference between the size of the second virtual projection plane 252 and the size of the first virtual projection plane 91 recalculated in step S217 is minimized. Then, the information processing device 50 displays the second virtual projection device 251 and the second virtual projection plane 252 based on this recalculation result superimposed on the physical space image 80 (step S218), and returns to step S212.

[0129] This allows users to visually understand the projection state of the comparison model (second virtual projection device 251) without having to perform any operations other than changing the zoom amount (projection ratio) for the first virtual projection device 151, making it easier to select the appropriate model.

[0130] In step S216, if no instruction to change the position of the first virtual projection device 151 has been received (step S216: No), the information processing device 50 determines whether or not an instruction to change the lens shift amount of the first virtual projection device 151 (position of the first virtual projection plane 91) has been received from the user (step S219). If an instruction to change the lens shift amount of the first virtual projection device 151 has been received (step S219: Yes), the information processing device 50 recalculates the position of the first virtual projection plane 91 based on the changed lens shift amount of the first virtual projection device 151, and displays the first virtual projection plane 91 at the recalculated position superimposed on the physical space image 80 (step S220).

[0131] Next, the information processing device 50 sets the lens shift amount of the second virtual projection device 251 based on the specifications of the model designated as the second virtual projection device in step S206, so that the distance between the center position of the first virtual projection plane 91 and the center position of the second virtual projection plane 252 is minimized. The information processing device 50 also recalculates the position of the second virtual projection plane 252 based on the set lens shift amount of the second virtual projection device 251. Then, the information processing device 50 displays the second virtual projection plane 252 at the recalculated position superimposed on the physical space image 80 (step S221), and returns to step S212.

[0132] This allows the user to visually understand the projection state of the comparison model (second virtual projection device 251) without performing any operations other than changing the lens shift amount (position of the first virtual projection surface 91) of the first virtual projection device 151, making it easier to select the appropriate model.

[0133] If, in step S219, the information processing device 50 has not received an instruction to change the lens shift amount of the first virtual projection device 151 (step S219: No), the information processing device 50 returns to step S212.

[0134] Thus, in the projector-priority second mode, the information processing device 50 displays a second image that includes the first virtual projection surface 91, the first virtual projection device 151, the second virtual projection device 251, and the second virtual projection surface 252 corresponding to the second virtual projection device 251. The information processing device 50 also sets the position of the second virtual projection surface 252 and the position of the second virtual projection device 251 based on the position of the first virtual projection device 151 and the position of the first virtual projection surface 91.

[0135] Furthermore, in the projector priority second mode, the information processing device 50 sets the size of the first virtual projection surface 91 based on the position of the first virtual projection device 151 and the position of the first virtual projection surface 91, and sets the position of the second virtual projection device based on the position of the first virtual projection device 151 (for example, to set it to a position as close as possible). Then, based on the position and size of the first virtual projection surface 91, the information processing device 50 sets the parameters of the second virtual projection device 251 (such as lens shift amount and zoom amount) related to the position and size of the second virtual projection surface 252 (for example, to set the position and size of the first virtual projection surface 91 and the second virtual projection surface 252 to be as close as possible).

[0136] Furthermore, the information processing device 50 may include in the second image an image (e.g., lens shift range 261) that indicates the range in which at least one of the position and size of the first virtual projection surface 91 or the second virtual projection surface 252 can be adjusted by changing the parameters of the first virtual projection device 151 or the second virtual projection device 251.

[0137] (Projector priority mode 3) Figure 27 is a flowchart showing an example of the processing performed by the information processing device 50 in the projector-priority third mode. Similar to the screen-priority mode, the information processing device 50 displays the physical space image 80 shown in Figure 8 at the start of the processing shown in Figure 27. The information processing device 50 also recognizes the physical space shown by the physical space image 80 in three dimensions.

[0138] Steps S271 to S276 shown in Figure 27 are the same as steps S201 to S206 shown in Figure 20. Following step S276, the information processing device 50 calculates the position of the second virtual projection device 251 and the position and size of the second virtual projection surface 252 based on the specifications of the model of the second virtual projection device that was designated in step S276 (step S277). The calculations in step S277 will be described later in Figures 28 and 30.

[0139] Next, the information processing device 50 displays the second virtual projection device 251 and the second virtual projection surface 252 superimposed on the physical space image 80 (step S278), and ends the series of processes. After step S278, the information processing device 50 may perform processing based on various change instructions, similar to steps S212 to S221 shown in Figure 21.

[0140] Figure 28 is a flowchart showing an example of the calculation in step S277 of Figure 27. Figure 29 is an example of an image displayed by the information processing device 50 in the process of Figure 27 in the example of Figure 28. In step S277 of Figure 27, the information processing device 50 executes the process shown in Figure 28, for example.

[0141] First, the information processing device 50 determines whether the first virtual projection device 151 and the second virtual projection device 251 can set the same aspect ratio to each other (step S281). The determination in step S281 is made based on the models of the first virtual projection device 151 and the second virtual projection device 251, as specified by steps S272 and S276 in Figure 27.

[0142] In step S281, if the first virtual projection device 151 and the second virtual projection device 251 can set the same aspect ratio to each other (step S281: Yes), the information processing device 50 sets the size of the second virtual projection plane 252 to the same size as the size of the first virtual projection plane 91 calculated in step S274 of Figure 27 (step S282).

[0143] In step S281, if the first virtual projection device 151 and the second virtual projection device 251 cannot be set to the same aspect ratio (step S281: No), the information processing device 50 sets the size of the second virtual projection surface 252 such that the vertical width of the second virtual projection surface 252 is greater than or equal to the vertical width of the first virtual projection device 151, and the horizontal width of the second virtual projection surface 252 is greater than or equal to the vertical width of the first virtual projection device 151 (step S283).

[0144] Next, the information processing device 50 calculates the range of possible projection distances for the second virtual projection device 251 based on the projection ratio of the model specified as the second virtual projection device in step S206 (step S284). The information processing device 50 also sets the position of the second virtual projection plane 252 to the same position as the position of the first virtual projection plane 91 (step S285).

[0145] Next, the information processing device 50 sets the position of the second virtual projection device 251 (step S286), and completes step S277 in Figure 27. Specifically, in step S286, the information processing device 50 sets the position of the second virtual projection device 251 from among the candidate installation ranges for the second virtual projection device 251, which are determined from the position of the second virtual projection plane 252 set in step S285 and the range of possible projection distances for the second virtual projection device 251 calculated in step S284, so that the distance between the first virtual projection device 151 and the second virtual projection device 251 is minimized.

[0146] In the example shown in Figure 28, in step S278 of Figure 27, for example, the information processing device 50 displays the first virtual projection plane 91 and the second virtual projection plane 252, whose center positions coincide, and the corresponding first virtual projection device 151 and the second virtual projection plane 252, as shown in Figure 29. In the example shown in Figure 29, the bottom surface of the first virtual projection device 151 and the bottom surface of the second virtual projection device 251 are not on the same plane.

[0147] Figure 30 is a flowchart illustrating another example of the calculation in step S277 of Figure 27. Figures 31 and 32 are examples of images displayed by the information processing device 50 in the process of Figure 27 in the example of Figure 30. When setting constraints on the position of the second virtual projection device 251, in step S277 of Figure 27, the information processing device 50 performs the process shown in Figure 30, for example.

[0148] Steps S301 to S305 shown in Figure 30 are the same as steps S281 to S285 shown in Figure 28. After step S305, the information processing device 50 determines whether there is a position within the installation range of the second virtual projection device 251 that satisfies the constraint conditions (step S306). If there is a position within the installation range of the second virtual projection device 251 that satisfies the constraint conditions (step S306: Yes), the information processing device 50 sets the position of the second virtual projection device 251 that is within the installation range of the second virtual projection device 251 and satisfies the constraint conditions (step S307), and terminates step S277 in Figure 27.

[0149] In step S306, if there is no position within the installation range of the second virtual projection device 251 that satisfies the constraint conditions (step S306: No), the information processing device 50 changes the position of the second virtual projection device 251 so that it satisfies the constraint conditions and is within the range of the projection distance calculated in step S304 (step S308).

[0150] Next, the information processing device 50 sets the position (center position) of the second virtual projection plane 252 to the position (center position) closest to the position of the first virtual projection plane 91 (step S309), and ends step S277 in Figure 27.

[0151] For example, in the example shown in Figure 30, suppose we set a constraint that the bottom surface of the first virtual projection device 151 and the bottom surface of the second virtual projection device 251 are on the same plane. Also, if we set the position (center position) of the second virtual projection surface 252 and the position (center position) of the first virtual projection surface 91 to the same position in the second virtual projection device 251, then we assume that there is no position within the installation range of the second virtual projection device 251 that satisfies this constraint. In this case, in step S278 of Figure 27, the information processing device 50 displays, for example, the first virtual projection device 151, the second virtual projection device 251 having a bottom surface that is on the same plane as the bottom surface of the first virtual projection device 151, the first virtual projection surface 91, and the second virtual projection surface 252 that can be projected from the second virtual projection device 251 and is close to the first virtual projection surface 91, as shown in Figure 31.

[0152] Furthermore, in the example shown in Figure 30, it is assumed that the constraint is set that the bottom surface of the second virtual projection device 251 lies on the physical plane. Also, it is assumed that the position of the first virtual projection device 151 is set so that the bottom surface of the first virtual projection device 151 lies on a plane different from the floor 81 (physical plane). Furthermore, it is assumed that the lens shift range of the second virtual projection device 251 is sufficiently large, and that if the position (center position) of the second virtual projection surface 252 and the position (center position) of the first virtual projection surface 91 are set to the same position, the installable range of the second virtual projection device 251 will overlap with the floor 81. In this case, in step S278 of Figure 27, the information processing device 50 displays, for example, the first virtual projection device 151, the second virtual projection device 251 whose bottom surface is on the floor 81, and the first virtual projection surface 91 and the second virtual projection surface 252 whose positions coincide, as shown in Figure 32.

[0153] This allows users to visually understand the differences in the position of different virtual projection devices when placing virtual projection surfaces of the closest possible size, making it easier to select the appropriate device.

[0154] Thus, in the projector-priority third mode, the information processing device 50 displays a second image including the first virtual projection surface 91, the first virtual projection device 151, the second virtual projection device 251, and the second virtual projection surface 252 corresponding to the second virtual projection device 251, similar to the projector-priority second mode. The information processing device 50 also sets the positions of the second virtual projection surface 252 and the second virtual projection device 251 based on the positions of the first virtual projection device 151 and the first virtual projection surface 91.

[0155] Furthermore, in the projector-priority third mode, the information processing device 50 sets the size of the first virtual projection surface 91 based on the position of the first virtual projection device 151 and the position of the first virtual projection surface 91, and sets the position and size of the second virtual projection surface 252 based on the position and size of the first virtual projection surface 91 (for example, setting them to be as close as possible in position and size). Then, the information processing device 50 sets the position of the second virtual projection surface 252 based on the position of the first virtual projection device 151 (for example, setting them so that the positions of the first virtual projection surface and the second virtual projection surface are as close as possible).

[0156] Furthermore, in the projector-priority third mode, after the positions and sizes of the first virtual projection device 151 and the first virtual projection surface 91 are determined, the second virtual projection device 251 and the second virtual projection surface 252 are set using these positions as constraints. However, the information processing device 50 may release these constraints in accordance with predetermined conditions such as instructions from the user. If these constraints are released, the information processing device 50 sets the positions and sizes of the second virtual projection device 251 and the second virtual projection surface 252 in accordance with instructions from the user, independently of the positions and sizes of the first virtual projection device 151 and the first virtual projection surface 91. In other words, the information processing device 50 may be able to switch between a state in which the second virtual projection device 251 and the second virtual projection surface 252 are set based on constraints derived from the settings of the first virtual projection device 151 and the first virtual projection surface 91, and a state in which the second virtual projection device 251 and the second virtual projection surface 252 are set without such constraints.

[0157] As described above, the information processing device 50 can improve user convenience regarding the selection of the projection device to be installed by displaying a second image in which the first virtual projection surface 91, the first virtual projection device 151 or the first installable range 92, and the second virtual projection device 251 or the second installable range 93 are displayed on the physical space image 80.

[0158] (Model search mode) Figure 33 is a flowchart illustrating an example of the processing performed by the information processing device 50 in model search mode. Figures 34 to 38 are examples of images displayed by the information processing device 50 during the processing shown in Figure 33. Similar to the processing shown in Figure 7, the information processing device 50 displays the physical space image 80 shown in Figure 8 at the start of the processing shown in Figure 33. The information processing device 50 also recognizes the physical space represented by the physical space image 80 in three dimensions.

[0159] First, the information processing device 50 receives a specification from the user for the position and size of the virtual projection plane (step S331). Next, based on the specification received in step S331, the information processing device 50 overlays the virtual projection plane onto the physical space image 80 and displays it (step S332). The process of displaying an image with the virtual projection plane overlaid on the physical space image 80 is similar to the process of displaying an image with the virtual projection plane overlaid on the physical space image 80 shown in Figure 9, for example. For example, as shown in Figure 34, the information processing device 50 displays an image with the virtual projection plane 341 overlaid on the physical space image 80.

[0160] Next, the information processing device 50 receives a specification from the user for the location and size of the installation candidate range, which is the area where the virtual projection device is to be installed (step S333). Next, the information processing device 50 displays the installation candidate range specified in step S333 superimposed on the physical space image 80 (step S334). For example, as shown in Figure 35, the information processing device 50 displays an image in which the virtual projection surface 341 and the installation candidate range 351 are superimposed on the physical space image 80. In this example, the installation candidate range 351 is a rectangular parallelepiped.

[0161] Next, the information processing device 50 extracts models from the registered projection device models that are capable of realizing the virtual projection surface 341 at the position and size specified in step S331 (step S335). For example, for each of the registered projection device models, the information processing device 50 performs the extraction in step S335 by placing that model within the candidate installation range 351 specified in step S333 and determining whether or not the virtual projection surface 341 can be realized by setting the parameters of that model (e.g., range shift amount and zoom amount).

[0162] Next, the information processing device 50 displays a list of models extracted in step S335 (step S336). For example, the information processing device 50 displays the list of compatible models 361 shown in Figure 36. In this example, the list of compatible models 361 shows "Projector 1", "Projector 2", and "Projector 5" as models capable of realizing the virtual projection surface 341 (compatible models). The information processing device 50 may display the list of compatible models 361 instead of the image shown in Figure 34, or it may display the list of compatible models 361 simultaneously with the image shown in Figure 34.

[0163] This allows users to narrow down the suitable models for the desired installation area 351 where they want to install the projection device, making it easier to select the appropriate model.

[0164] Next, the information processing device 50 accepts a selection from the user for the model of the virtual projection device from the list displayed in step S336 (step S337).

[0165] Next, the information processing device 50 calculates the position and size of the installable range based on the position and size of the virtual projection surface 341 specified in step S331, and the projection ratio and lens shift amount that can be set for the model of the virtual projection device specified in step S337. Then, the information processing device 50 displays the installable range, whose position and size have been calculated, superimposed on the physical space image 80 (step S338). For example, as shown in Figure 37, the information processing device 50 displays an image in which the virtual projection surface 341, the candidate installation range 351, and the installable range 371 are superimposed on the physical space image 80. In this example, the candidate installation range 351 is a rectangular parallelepiped. Note that in this case, the information processing device 50 may limit the display of the installable range 371 to only the overlapping area with the candidate installation range 351 specified by the user.

[0166] This allows users to visually understand the limited installation range depending on the model they are considering, making it easier to select the appropriate model.

[0167] Furthermore, in step S335 of Figure 33, the information processing device 50 may determine, for each of the registered projection device models, whether or not it is possible to realize a virtual projection surface 341 by setting the parameters of the model after placing that model within the installation candidate range 351 specified in step S333, for each case with or without cropping. Then, in step S336, the information processing device 50 may display the candidate models capable of realizing the virtual projection surface 341 by distinguishing between candidate models capable of realizing the virtual projection surface 341 without cropping and candidate models capable of realizing the virtual projection surface 341 with cropping. For example, the information processing device 50 displays the list of compatible models 361 shown in Figure 38. In this example, the list of compatible models 361 shows "Projector 1", "Projector 2", "Projector 5", and "Projector 6" as models capable of realizing the virtual projection surface 341 (compatible models). Of these, "Projector 1," "Projector 2," and "Projector 5" are models capable of realizing the virtual projection surface 341 without cropping, while "Projector 6" is a model capable of realizing the virtual projection surface 341 with cropping. In this case, in the list of compatible models 361, "Projector 6" will be marked with a crop mark 381 indicating that it is a model capable of realizing the virtual projection surface 341 with cropping.

[0168] This allows users to identify which models require adjustment and which do not among the models suitable for the area where they want to install the projection device, making it easier to select the appropriate model.

[0169] Thus, in model search mode, the information processing device 50 outputs a list of compatible models 361, which are candidate models of projection devices that can project onto the virtual projection surface 341 when installed in the candidate installation range 351, based on the position and size (range) of the virtual projection surface 341 (projection surface) and the specified range 351 of the projection device, which are received from the user based on the physical space image 80 (third image) captured by the imaging device of the sensor 65.

[0170] Furthermore, in the model search mode, the information processing device 50 may display the installable range 371 of the projection device in the physical space image 80, limited to the overlapping area with the candidate installation range 351 of the projection device. Also, in the model search mode, the information processing device 50 may output a list of compatible models 361 along with the usage conditions of the candidate projection device (for example, whether cropping is required).

[0171] Furthermore, the information processing device 50 may perform the processing in the model search mode in combination with the processing in the screen priority mode, projector priority first mode, projector priority second mode, or projector priority third mode described above. For example, in the screen priority mode, projector priority first mode, projector priority second mode, or projector priority third mode described above, the information processing device 50 may perform the processing of accepting the specification of the model of a projection device from the candidates output in the model search mode, instead of the processing of accepting the specification of the models of the first virtual projection device and the second virtual projection device.

[0172] As explained above, the information processing device 50 improves user convenience regarding the selection of a projection device to be installed by outputting a list of compatible models 361, which are candidate models of projection devices that can project onto the virtual projection surface 341 when installed in the candidate installation range 351, based on the position and size (range) of the virtual projection surface 341 (projection surface) and the candidate installation range 351 of the projection device, which are received from the user based on the physical space image 80 (third image).

[0173] (modified version) Modifications relating to each of the above embodiments will be described below.

[0174] <Example 1> The case described above is an example where the physical space image 80, which represents the physical space, is a through image that displays in real time an image obtained by imaging by the imaging device provided in the information processing device 50. However, the physical space image 80 is not limited to this. For example, the physical space image 80 may be a still image obtained by imaging the physical space in the past. Also, the physical space image 80 does not have to be an image obtained by imaging; for example, it may be an image generated based on 3D data that shows the structure of the physical space.

[0175] <Modification 2> Although the example described above has been of a tablet terminal having a touch panel 51, the information processing device 50 is not limited to this configuration. For example, the information processing device 50 may be an information terminal such as a smartphone or a personal computer.

[0176] <Variation 3> Although a configuration in which the information processing device 50 displays the second image has been described, the information processing device 50 may also control other devices to display the second image by transmitting the generated second image to them. In this case, the information processing device 50 may be a server device or the like that does not have a display device.

[0177] <Modification 4> The case where the physical space image 80 is an image obtained by imaging by the imaging device of the information processing device 50 has been described, but the physical space image 80 may also be an image obtained by imaging by a device other than the information processing device 50 and received by the information processing device 50 from that device. In this case, the information processing device 50 may be a device that does not have an imaging device.

[0178] <Modification 5> In the projector-priority second mode and the projector-priority third mode, if the installation state of the first virtual projection device 151 (e.g., floor-standing or ceiling-mounted) or the position of the second virtual projection device 251 is changed by user specification after displaying the second image, the information processing device 50 may reset at least one of the installation state of the second virtual projection device 251, the parameters of the second virtual projection device 251 (e.g., lens shift amount or zoom amount), and the position of the second virtual projection surface 252.

[0179] <Variation 6> When displaying the second image, the information processing device 50 may control the visibility of some of the objects superimposed on the second image (the first virtual projection plane 91, the first virtual projection device 151 or the first installable range 92, and the second virtual projection device 251 or the second installable range 93) to be lower than that of other objects. For example, the information processing device 50 may reduce the visibility of the first virtual projection device 151 by setting the opacity of the first virtual projection device 151 higher than that of the first virtual projection plane 91 and the second installable range 93, which are included in the second image shown in Figure 17. The objects whose visibility is reduced may be selected, for example, by user operation, or they may be automatically selected based on various information that the information processing device 50 can acquire (for example, projection device parameters, the position of the imaging device of the information processing device 50, the order of operations, etc.).

[0180] <Example 7> The case in which the second image is a still image including the first virtual projection surface 91, the first virtual projection device 151 or the first installable range 92, and the second virtual projection device 251 or the second installable range 93 has been described, but the second image may also be a moving image in which the first virtual projection surface 91, the first virtual projection device 151 or the first installable range 92, and the second virtual projection device 251 or the second installable range 93 are displayed alternately.

[0181] The information processing method described in the above-mentioned embodiment can be realized by executing a pre-prepared information processing program on a computer. This information processing program is recorded on a computer-readable storage medium and executed by reading it from the storage medium. This information processing program may also be provided in the form of a non-transient storage medium such as flash memory, or it may be provided via a network such as the Internet. The computer that executes this information processing program may be included in an information processing device, or it may be included in an electronic device such as a smartphone, tablet terminal, or personal computer that can communicate with the information processing device, or it may be included in a server device that can communicate with these information processing devices and electronic devices.

[0182] Each of the above embodiments and modifications can be implemented in combination.

[0183] This specification contains at least the following:

[0184] (1) An information processing device equipped with a processor, The above processor is First image data representing the first image captured by the imaging device, The following are obtained: first virtual projection plane data relating to the first virtual projection plane, first virtual projection device data relating to the first virtual projection device corresponding to the first virtual projection plane, and second virtual projection device data relating to the second virtual projection device. Based on the above first virtual projection plane data, the above first virtual projection device data, and the above second virtual projection device data, a second image data is generated that represents a second image in which the first virtual projection plane, the above first virtual projection device or the possible installation range of the above first virtual projection device, and the above second virtual projection device or the possible installation range of the above second virtual projection device are displayed in the first image. Output the above second image data to the output destination. Information processing device.

[0185] (2) (1) The information processing device described above, The second image above includes the first virtual projection surface, the installation range of the first virtual projection device, and the installation range of the second virtual projection device. The above processor is Based on at least one of the position and size of the first virtual projection surface, the installable range of the first virtual projection device and the installable range of the second virtual projection device are set. Information processing device.

[0186] (3) (2) The information processing device described above, The above processor is If at least one of the position and size of the first virtual projection plane is changed, the installation range of the first virtual projection device and the installation range of the second virtual projection device are reset. Information processing device.

[0187] (4) (1) The information processing device described above, The second image above includes the first virtual projection plane, the first virtual projection device, and the installation range of the second virtual projection device. The above processor is Based on the position of the first virtual projection device described above, set at least one of the position and size of the first virtual projection surface described above. Based on at least one of the position and size of the first virtual projection surface, the possible installation range of the second virtual projection device is set. Information processing device.

[0188] (5) (4) The information processing device described above, The above processor is If the position or parameters of the first virtual projection device are changed, the position and size of the first virtual projection surface and the installation range of the second virtual projection device are reset. Information processing device.

[0189] (6) (4) or (5) the information processing device described above, The above processor is If the position of the first virtual projection plane is changed, the installation range of the second virtual projection device is reset. Information processing device.

[0190] (7) (1) The information processing device described above, The second image above includes the first virtual projection plane, the first virtual projection device, the second virtual projection device, and the second virtual projection plane corresponding to the second virtual projection device. The above processor is Based on the position of the first virtual projection device and the position of the first virtual projection plane, the position of the second virtual projection plane and the position of the second virtual projection device are set. Information processing device.

[0191] (8) (7) An information processing device as described above, The above processor is Based on the position of the first virtual projection device and the position of the first virtual projection surface, the size of the first virtual projection surface is set. Based on the position of the first virtual projection device described above, the position of the second virtual projection device is set. Based on the position and size of the first virtual projection plane, the parameters of the second virtual projection plane relating to the position and size of the second virtual projection plane are set. Information processing device.

[0192] (9) (7) An information processing device as described above, The above processor is Based on the position of the first virtual projection device and the position of the first virtual projection surface, the size of the first virtual projection surface is set. Based on the position and size of the first virtual projection plane described above, the position and size of the second virtual projection plane are set. Based on the position of the first virtual projection device, the parameters of the second virtual projection plane are set regarding the position of the second virtual projection plane. Information processing device.

[0193] (10) An information processing device described in any one of paragraphs (7) to (9), The above processor is The process involves including an image in the second image that shows the range in which at least one of the position and size of the first virtual projection surface or the second virtual projection surface can be adjusted by changing the parameters of the first virtual projection device or the second virtual projection device. Information processing device.

[0194] (11) An information processing device described in any one of items (7) to (10), The above processor is If the installation status of the first virtual projection device or the position of the second virtual projection device is changed, at least one of the installation status of the second virtual projection device, the parameters of the second virtual projection device, and the position of the second virtual projection plane is reset. Information processing device.

[0195] (12) An information processing device described in any one of items (7) to (11), The above processor is It is possible to switch between a state in which the second virtual projection device and the second virtual projection surface are set up based on constraints derived from the settings of the first virtual projection device and the first virtual projection surface, and a state in which the second virtual projection device and the second virtual projection surface are set up without the above constraints. Information processing device.

[0196] (13) An information processing device described in any one of items (1) to (12), The above processor is Based on the third image captured by the above imaging device, and based on the user's specifications of the projection surface range and the installation range of the projection device, the system outputs candidate projection devices that can project onto the projection surface range when installed within the installation range of the projection device. The first virtual projection device and the second virtual projection device described above are virtual projection devices that represent the projection devices selected by the user from among the candidate projection devices. Information processing device.

[0197] (14) (13) An information processing device as described above, The above processor is The installation range of at least one of the first virtual projection device and the second virtual projection device is limited to the overlapping area with the installation range of the projection device and displayed in the first image. Information processing device.

[0198] (15) An information processing device as described in (13) or (14), The above processor is The above projection device candidates are output along with their operating conditions. Information processing device.

[0199] (16) An information processing device described in any one of items (1) to (15), The above processor is In the second image above, control is performed to make the visibility of some objects lower than the visibility of other objects, among the first virtual projection plane, the first virtual projection device or the installation range of the first virtual projection device, and the second virtual projection device or the installation range of the second virtual projection device. Information processing device.

[0200] (17) The processor of the information processing device, First image data representing the first image captured by the imaging device, The following are obtained: first virtual projection plane data relating to the first virtual projection plane, first virtual projection device data relating to the first virtual projection device corresponding to the first virtual projection plane, and second virtual projection device data relating to the second virtual projection device. Based on the above first virtual projection plane data, the above first virtual projection device data, and the above second virtual projection device data, a second image data is generated that represents a second image in which the first virtual projection plane, the above first virtual projection device or the possible installation range of the above first virtual projection device, and the above second virtual projection device or the possible installation range of the above second virtual projection device are displayed in the first image. Output the above second image data to the output destination. Information processing methods.

[0201] (18) In the processor of the information processing device, First image data representing the first image captured by the imaging device, The following are obtained: first virtual projection plane data relating to the first virtual projection plane, first virtual projection device data relating to the first virtual projection device corresponding to the first virtual projection plane, and second virtual projection device data relating to the second virtual projection device. Based on the above first virtual projection plane data, the above first virtual projection device data, and the above second virtual projection device data, a second image data is generated that represents a second image in which the first virtual projection plane, the above first virtual projection device or the possible installation range of the above first virtual projection device, and the above second virtual projection device or the possible installation range of the above second virtual projection device are displayed in the first image. Output the above second image data to the output destination. An information processing program used to execute a process.

[0202] (19) An information processing device equipped with a processor, The above processor is Based on the image captured by the imaging device, and the user's specified projection area and installation area of ​​the projection device, the system outputs candidate projection devices that can project onto the projection area when installed within the specified installation area. Information processing device.

[0203] (20) The processor of the information processing device, Based on the image captured by the imaging device, and the user's specified projection area and installation area of ​​the projection device, the system outputs candidate projection devices that can project onto the projection area when installed within the specified installation area. Information processing methods.

[0204] (twenty one) In the processor of the information processing device, Based on the image captured by the imaging device, and the user's specified projection area and installation area of ​​the projection device, the system outputs candidate projection devices that can project onto the projection area when installed within the specified installation area. An information processing program used to execute a process.

[0205] Although various embodiments have been described above, it goes without saying that the present invention is not limited to these examples. It is clear to those skilled in the art that various modifications or alterations can be conceived within the scope of the claims, and these will naturally also fall within the technical scope of the present invention. Furthermore, the components in the above embodiments may be combined in any way without departing from the spirit of the invention.

[0206] This application is based on the Japanese Patent Application No. 2022-049515 filed on March 25, 2022, the contents of which are incorporated by reference within this application. [Explanation of Symbols]

[0207] 1 Projection section 2. Operation reception unit 2A,3A Hollow part 2a,2b,3a,3c,15a opening 4. Control device 4a,62 memory 6 Projected object 10 Projection device 11 Projection Range 12 Optical Modulation Unit 15 cabinets 21 Light source 22 Optical Modulation Section 23 Projection optical system 24 Control circuits 31 Second optical system 32,122 Reflective material 33 Third optical system 34 lenses 50 Information Processing Devices 51 Touch panel 61 processors 63 Communication Interfaces 64 User Interfaces 65 Sensors 69 bus 80 Physical Space Images 81 beds 82 Wall 91 First virtual projection plane 92. First possible installation range 93 Second possible installation range 101 Main body 102 First Member 103 Second Member 104 Projection direction changing mechanism 105 Shift mechanism 106 Optical Units 121 1st optical system 151 First Virtual Projection Device 251 Second Virtual Projection Device 252 Second virtual projection plane 261 Lens shift range 341 Virtual projection plane 351 Potential installation area 361 Compatible Models List 371 Installation range 381 Crop Mark G1 Image P1~P4 position D1~D3 Projection distance

Claims

1. An information processing device equipped with a processor, The aforementioned processor, First image data representing the first image captured by the imaging device, The following are obtained: first virtual projection plane data relating to the first virtual projection plane, first virtual projection device data relating to the first virtual projection device corresponding to the first virtual projection plane, and second virtual projection device data relating to the second virtual projection device. Based on the first virtual projection plane data, the first virtual projection device data, and the second virtual projection device data, a second image data is generated that represents a second image in which the first virtual projection plane, the first virtual projection device or the possible installation range of the first virtual projection device, and the second virtual projection device or the possible installation range of the second virtual projection device are displayed in the first image. Output the second image data to the output destination. Information processing device.

2. An information processing apparatus according to claim 1, The second image includes the first virtual projection plane, the installation range of the first virtual projection device, and the installation range of the second virtual projection device. The aforementioned processor, Based on at least one of the position and size of the first virtual projection surface, the installable range of the first virtual projection device and the installable range of the second virtual projection device are set. Information processing device.

3. An information processing apparatus according to claim 2, The aforementioned processor, If at least one of the position and size of the first virtual projection plane is changed, the installation range of the first virtual projection device and the installation range of the second virtual projection device are reset. Information processing device.

4. An information processing apparatus according to claim 1, The second image includes the first virtual projection plane, the first virtual projection device, and the installation range of the second virtual projection device. The aforementioned processor, Based on the position of the first virtual projection device, set at least one of the position and size of the first virtual projection surface. Based on at least one of the position and size of the first virtual projection plane, the installable range of the second virtual projection device is set. Information processing device.

5. An information processing apparatus according to claim 4, The aforementioned processor, If the parameters of the first virtual projection device relating to the position of the first virtual projection device, or at least one of the position, size, and shape of the first virtual projection surface, are changed, the position and size of the first virtual projection surface and the installable range of the second virtual projection device are reset. Information processing device.

6. An information processing apparatus according to claim 4, The aforementioned processor, If the position of the first virtual projection plane is changed, the installation range of the second virtual projection device is reset. Information processing device.

7. An information processing apparatus according to claim 1, The second image includes the first virtual projection plane, the first virtual projection device, the second virtual projection device, and the second virtual projection plane corresponding to the second virtual projection device. The aforementioned processor, Based on the position of the first virtual projection device and the position of the first virtual projection plane, the position of the second virtual projection plane and the position of the second virtual projection device are set. Information processing device.

8. An information processing apparatus according to claim 7, The aforementioned processor, The size of the first virtual projection surface is set based on the position of the first virtual projection device and the position of the first virtual projection surface. Based on the position of the first virtual projection device, the position of the second virtual projection device is set. Based on the position and size of the first virtual projection plane, parameters for the second virtual projection plane relating to the position and size of the second virtual projection plane are set. Information processing device.

9. An information processing apparatus according to claim 7, The aforementioned processor, The size of the first virtual projection surface is set based on the position of the first virtual projection device and the position of the first virtual projection surface. Based on the position and size of the first virtual projection plane, the position and size of the second virtual projection plane are set. Based on the position of the first virtual projection device, parameters of the second virtual projection plane relating to the position of the second virtual projection plane are set. Information processing device.

10. An information processing apparatus according to claim 7, The aforementioned processor, The process involves including an image in the second image that shows the range in which at least one of the position, size, and shape of the first virtual projection plane or the second virtual projection plane can be adjusted by changing the parameters of the first virtual projection device or the second virtual projection device, relating to at least one of the position, size, and shape of the first virtual projection plane or the second virtual projection plane. Information processing device.

11. An information processing apparatus according to claim 7, The aforementioned processor, If the installation state of the first virtual projection device or the position of the second virtual projection device is changed, The installation status of the second virtual projection device, Parameters of the second virtual projection device relating to at least one of the position, size, and shape of the second virtual projection plane, The position of the second virtual projection plane and Reset at least one of the following: Information processing device.

12. An information processing apparatus according to claim 7, The aforementioned processor, The system can switch between a state in which the settings for the position and installation of the second virtual projection device, as well as the position and size of the second virtual projection surface, are made based on constraints arising from the settings for at least one of the planes that serve as the reference for the position and installation of the first virtual projection device, and the settings for the position and size of the first virtual projection surface, and a state in which the settings for the position and installation of the second virtual projection device, as well as the settings for the position and size of the second virtual projection surface, are made without the aforementioned constraints. Information processing device.

13. An information processing apparatus according to claim 1, The aforementioned processor, Based on the third image captured by the aforementioned imaging device, and based on the user's specifications of the projection surface range and the installation range of the projection device, the system outputs candidate projection devices that can project onto the projection surface range when installed within the installation range of the projection device. The first virtual projection device and the second virtual projection device are virtual projection devices that represent each projection device selected by the user from among the candidate projection devices. Information processing device.

14. An information processing apparatus according to claim 13, The aforementioned processor, The installation range of at least one of the first virtual projection device and the second virtual projection device is limited to the overlapping area with the installation range of the projection device and displayed in the first image. Information processing device.

15. An information processing apparatus according to claim 13, The aforementioned processor, The candidate projection device is output along with the operating conditions for the candidate projection device. Information processing device.

16. An information processing device according to any one of claims 1 to 15, The aforementioned processor, In the second image, control is performed to make the visibility of some objects lower than the visibility of other objects, among the first virtual projection plane, the first virtual projection device or the installation range of the first virtual projection device, and the second virtual projection device or the installation range of the second virtual projection device. Information processing device.

17. The processor of the information processing device, First image data representing the first image captured by the imaging device, The following are obtained: first virtual projection plane data relating to the first virtual projection plane, first virtual projection device data relating to the first virtual projection device corresponding to the first virtual projection plane, and second virtual projection device data relating to the second virtual projection device. Based on the first virtual projection plane data, the first virtual projection device data, and the second virtual projection device data, a second image data is generated that represents a second image in which the first virtual projection plane, the first virtual projection device or the possible installation range of the first virtual projection device, and the second virtual projection device or the possible installation range of the second virtual projection device are displayed in the first image. Output the second image data to the output destination. Information processing methods.

18. In the processor of the information processing device, First image data representing the first image captured by the imaging device, The following are obtained: first virtual projection plane data relating to the first virtual projection plane, first virtual projection device data relating to the first virtual projection device corresponding to the first virtual projection plane, and second virtual projection device data relating to the second virtual projection device. Based on the first virtual projection plane data, the first virtual projection device data, and the second virtual projection device data, a second image data is generated that represents a second image in which the first virtual projection plane, the first virtual projection device or the possible installation range of the first virtual projection device, and the second virtual projection device or the possible installation range of the second virtual projection device are displayed in the first image. Output the second image data to the output destination. An information processing program used to execute a process.

Citation Information

Patent Citations

  • Image projection system, operation method of image projection system, image projection device, and remote control device of image projection system

    JP2014056044A

  • Image display system, control method for image display system; image display device, and control method for image display device

    JP2015108796A

  • Projection image adjustment system and projection image adjustment method

    JP2018005115A

  • Information processing apparatus, control method thereof, projector, and program

    JP2019071567A

  • Projection device

    JP2020036120A