Display control method, display control program, and display control device

Abstract

To easily determine a mask region.SOLUTION: A display control method includes: displaying, on a display unit 3 different from a projector 1, an operation image PR obtained by superimposing a first image including lattice images GA representing lattices on a picked-up image PT obtained by picking up an image of a range including a projection object QB on which the projector 1 projects a projection image PM; receiving, through the operation image PR, a first operation QP1 to select a first region R1 that is at least one region of a plurality of regions partitioned from each other by the lattices; and determining a region of the projection image PM corresponding to a region of the picked-up image PT indicated by the first region R1 or a region of the plurality of regions other than the first region, as a mask region MR where mask processing is performed to hide the projection image PM.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a display control method, a display control program, and a display control device. [Background technology]

[0002] Patent Document 1 discloses a projection system that includes a projection device that projects invisible measurement light onto an object, an imaging device that receives the measurement light reflected from the object, and a calculation device that calculates position information of the object based on the reflected measurement light, and that performs a masking process to limit part of the projection range of the measurement light. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-219289 Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 states that a user may manually set a mask area. However, Patent Document 1 does not disclose a specific user interface for a user to set a mask area, and there has been a demand for a technology that allows a user to easily set a mask area. [Means for solving the problem]

[0005] One aspect of the present disclosure is a display control method that includes displaying, on a display device other than the projector, an operation image in which a first image including a grid image representing a grid is superimposed on an captured image of an area including a projection object onto which a projected image is to be projected by a projector; accepting, via the operation image, a first operation to select a first area, which is at least one area among a plurality of areas partitioned by the grid; and determining, by mask processing, an area of ​​the projected image that corresponds to an area of ​​the captured image indicated by the first area or an area among the plurality of areas other than the first area, as a mask area in which the projected image is not displayed.

[0006] Another aspect of the present disclosure is a display control program that causes a computer to execute processing including: displaying, on a display device other than the projector, an operation image in which a first image including a grid image representing a grid is superimposed on an captured image of an area including a projection object onto which a projected image is to be projected by a projector; accepting, via the operation image, a first operation to select a first area which is at least one area among a plurality of areas partitioned by the grid; and determining, by mask processing, an area of ​​the projected image corresponding to an area of ​​the captured image indicated by the first area or an area among the plurality of areas other than the first area, as a mask area in which the projected image is not displayed.

[0007] Yet another aspect of the present disclosure is a display control device that includes at least one processor that performs the following operations: displaying, on a display device other than the projector, an operation image in which a first image including a grid image representing a grid is superimposed on an captured image of an area including a projection object onto which a projected image is to be projected by a projector; accepting, via the operation image, a first operation to select a first area that is at least one area among a plurality of areas partitioned by the grid; and determining, by mask processing, an area of ​​the projected image that corresponds to an area of ​​the captured image indicated by the first area or an area among the plurality of areas other than the first area, as a mask area in which the projected image is not displayed. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a display control system according to an embodiment. [Figure 2] FIG. 1 is a diagram showing an example of the configuration of a projector. [Figure 3] FIG. 10 is a screen diagram showing an example of an operation screen including an operation image. [Figure 4] FIG. 10 is a screen diagram showing an example of a first operation on the operation screen. [Figure 5] FIG. 10 is a screen diagram showing another example of the first operation on the operation screen. [Figure 6] FIG. 10 is a screen diagram showing an example of a second operation on the operation screen. [Figure 7] FIG. 10 is a screen diagram of an operation screen showing an example of a determined mask area. [Figure 8] FIG. 10 is an image diagram showing an example of a mask image that hides the projected image in the mask area. [Figure 9] 10 is a flowchart showing an example of processing by a second control unit. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, this embodiment will be described with reference to the drawings. FIG. 1 is a diagram showing an example of the configuration of a display control system 100 according to this embodiment. As shown in FIG. 1, the display control system 100 includes a projector 1, a personal computer 2, and a display device 3.

[0010] The projector 1 includes a first control unit 150, a projection unit 110, and a camera 160. The first control unit 150 controls each unit of the projector 1. The first control unit 150 includes a first processor 150A and a first memory 150B. The first memory 150B stores a first control program PG1. The first processor 150A controls each unit of the projector 1 by reading and executing the first control program PG1 from the first memory 150B. The projector 1 will be further described with reference to FIG.

[0011] The projection unit 110 projects image light PL corresponding to the projection image PM onto the projection target QB. The projection unit 110 also projects the image light PL so as to form the projection image PM on the projection target QB. The projection unit 110 will be further described with reference to FIG. The projection object QB is an object onto which the projector projects the projection image PM. The projection object QB includes, for example, a building, a castle, or other structure. The projection object QB may be a three-dimensional object other than a structure, or a flat object, and may be a part or the entire object. The projection object QB will be further described with reference to FIG. 3.

[0012] The camera 160 generates a captured image PT by capturing an image of an area including the projection target QB. The camera 160 also transmits the generated captured image PT to the first control unit 150. In this embodiment, a case will be described in which the projector 1 includes the camera 160, but the embodiment is not limited to this. For example, the camera may be configured separately from the projector 1.

[0013] The captured image PT is a first captured image PT1 or a second captured image PT2. The first captured image PT1 is a captured image PT including a projection object QB onto which a projection image PM is not projected. The second captured image PT2 is a captured image PT including a projection object QB onto which a projection image PM is projected. In this embodiment, a case will be described in which the captured image PT is the first captured image PT1.

[0014] The personal computer 2 includes a second control unit 21. The second control unit 21 controls each unit of the personal computer 2. The second control unit 21 includes a second processor 21A and a second memory 21B. The second memory 21B is a storage device that non-volatilely stores programs and data executed by the second processor 21A. The second memory 21B is configured with a magnetic storage device, a semiconductor storage element such as a flash ROM (Read Only Memory), or other types of non-volatile storage device. The second memory 21B may also include a RAM (Random Access Memory) that configures the work area of ​​the second processor 21A. The second memory 21B stores data processed by the second control unit 21, the second control program PG2 executed by the second processor 21A, and the like.

[0015] The second processor 21A may be configured as a single processor, or may be configured as multiple processors functioning as the second processor 21A. The second processor 21A executes the second control program PG2 to control each part of the personal computer 2. The personal computer 2 corresponds to an example of a "display control device." The second processor 21A corresponds to an example of a "computer." The second control program PG2 corresponds to an example of a "display control program."

[0016] The second processor 21A may be configured as a SoC (System on Chip) integrated with part or all of the second memory 21B and other circuits. The second processor 21A may also be configured as a combination of a CPU (Central Processing Unit) that executes programs and a DSP (Digital Signal Processor) that executes predetermined arithmetic processing. All of the functions of the second processor 21A may be implemented in hardware, or may be configured using a programmable device.

[0017] The personal computer 2 acquires the captured image PT from the projector 1. The personal computer 2 acquires the captured image PT from the projector 1 via, for example, a USB (registered trademark) (Universal Serial Bus) cable.

[0018] In this embodiment, the personal computer 2 is connected to the projector 1 via a USB (registered trademark) cable or the like to enable wired communication, but may also be connected via Wi-Fi (registered trademark) or the like to enable wireless communication.

[0019] The display device 3 is communicably connected to the personal computer 2. The display device 3 includes a display 31 such as an LCD (Liquid Crystal Display). The display device 3 displays various images such as captured images PT on the display 31 in accordance with instructions from the personal computer 2.

[0020] Next, the configuration of the projector 1 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the projector 1. 2, the projector 1 includes a projection unit 110 and a drive unit 120 that drives the projection unit 110. The projection unit 110 forms an optical image and projects image light PL onto a projection object QB. Note that in this embodiment, the projection unit 110 projects, for example, image light PL corresponding to a projection image PM onto the projection object QB. The projection unit 110 includes a light source unit 111, a light modulation device 112, and a projection optical system 113. The drive unit 120 includes a light source drive unit 121 and a light modulation device drive unit 122.

[0021] The light source unit 111 includes a solid-state light source 111A such as an LED (Light Emitting Diode) or a laser light source. The solid-state light source 111A corresponds to an example of a "light source." In this embodiment, the light source unit 111 is described as including the solid-state light source 111A, but is not limited to this. The light source unit 111 may include a lamp light source such as a halogen lamp, a xenon lamp, or an ultra-high pressure mercury lamp instead of the solid-state light source 111A. In the following description, the solid-state light source 111A may be referred to as the light source 111A.

[0022] The light source unit 111 may also include a reflector and an auxiliary reflector that guide the light emitted by the light source 111A to the light modulation device 112. Furthermore, the light source unit 111 may also include a lens group for improving the optical characteristics of the projected light, a polarizing plate, or a dimming element that reduces the amount of light emitted by the light source 111A on the path leading to the light modulation device 112. The light source driving unit 121 is connected to the internal bus 107, and turns on and off the light source 111A of the light source unit 111 in accordance with instructions from the first control unit 150 also connected to the internal bus 107, thereby controlling the output of the light source 111A.

[0023] The light modulation device 112 includes, for example, three liquid crystal panels 115 corresponding to the three primary colors of R, G, and B. R indicates red, G indicates green, and B indicates blue. That is, the light modulation device 112 includes a liquid crystal panel 115 corresponding to R light, a liquid crystal panel 115 corresponding to G light, and a liquid crystal panel 115 corresponding to B light. The light emitted by the light source unit 111 is separated into three color lights of RGB, and each color light is incident on a corresponding liquid crystal panel 115. Each of the three liquid crystal panels 115 is a transmissive liquid crystal panel, and modulates the light that passes through it to generate image light PL. The image light PL that has passed through each liquid crystal panel 115 and been modulated is combined by a combining optical system such as a cross dichroic prism, and is emitted to the projection optical system 113. In this embodiment, the light modulation device 112 is described as having a transmissive liquid crystal panel 115 as a light modulation element, but is not limited to this. The light modulation element may be a reflective liquid crystal panel or a digital micromirror device.

[0024] The light modulation device 112 is driven by a light modulation device driving unit 122. The light modulation device driving unit 122 is connected to the image processing unit 145. Image data corresponding to each of the primary colors R, G, and B is input to the light modulation device driving unit 122 from the image processing unit 145. The light modulation device driving unit 122 converts the input image data into a data signal suitable for the operation of the liquid crystal panel 115. Based on the converted data signal, the light modulation device driving unit 122 applies a voltage to each pixel of each liquid crystal panel 115, and draws an image on each liquid crystal panel 115.

[0025] The projection optical system 113 includes a first projection lens 113A that forms an image of the incident image light PL on the projection object QB, a mirror, etc. The projection optical system 113 also includes a zoom mechanism that enlarges or reduces the image projected onto the projection object QB, a focus adjustment mechanism that adjusts the focus, and a lens shift mechanism that adjusts the projection direction of the image light PL.

[0026] The projector 1 also includes a camera 160. The camera 160 includes an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The camera 160 captures an image of the projection target QB and generates a captured image PT in accordance with instructions from the first control unit 150. The first control unit 150 causes the camera 160 to generate the captured image PT in accordance with instructions from the second control unit 21 of the personal computer 2. The camera 160 also transmits the generated captured image PT to the first control unit 150. The first control unit 150 transmits the captured image PT to the second control unit 21 of the personal computer 2. The camera 160 is disposed near the first projection lens 113A of the projector 1, for example.

[0027] The projector 1 further includes an operation unit 131, a remote control receiver 133, an input interface 135, a storage unit 137, a first communication interface 141, a frame memory 143, an image processing unit 145, and a first control unit 150. The input interface 135, the storage unit 137, the first communication interface 141, the image processing unit 145, the first control unit 150, and the camera 160 are connected to each other via an internal bus 107 so as to be able to communicate data with each other.

[0028] The operation unit 131 includes various buttons and switches provided on the surface of the housing of the projector 1, generates operation signals corresponding to the operation of these buttons and switches, and outputs the operation signals to the input interface 135. The input interface 135 includes a circuit that outputs the operation signals input from the operation unit 131 to the first control unit 150.

[0029] The remote control light receiving unit 133 receives an infrared signal transmitted from the remote control 5, decodes the received infrared signal, and generates an operation signal. The remote control light receiving unit 133 outputs the generated operation signal to the input interface 135. The input interface 135 includes a circuit that outputs the operation signal input from the remote control light receiving unit 133 to the first control unit 150.

[0030] The storage unit 137 is, for example, a magnetic recording device such as an HDD (Hard Disk Drive), or a storage device using a semiconductor storage element such as a flash memory or an SSD (Solid State Drive). The storage unit 137 stores programs executed by the first control unit 150, data processed by the first control unit 150, image data, etc.

[0031] The first communication interface 141 is a communication interface that communicates with the personal computer 2 in accordance with the USB (registered trademark) standard. The first communication interface 141 includes a connector for connecting a USB (registered trademark) cable and an interface circuit for processing signals transmitted through the connector. The first communication interface 141 is an interface board having a connector and an interface circuit, and is connected to a main board on which the first processor 150A of the first control unit 150 and the like are mounted. Alternatively, the connector and interface circuit that constitute the first communication interface 141 are mounted on the main board of the first control unit 150. The first communication interface 141 transmits, for example, a captured image PT to the personal computer 2.

[0032] The first control unit 150 includes a first memory 150B and a first processor 150A. The first memory 150B is a storage device that nonvolatilely stores programs and data executed by the first processor 150A. The first memory 150B is configured with a magnetic storage device, a semiconductor storage element such as a flash ROM, or other types of nonvolatile storage devices. The first memory 150B may also include RAM that configures the work area of ​​the first processor 150A. The first memory 150B stores data processed by the first control unit 150, the first control program PG1 executed by the first processor 150A, and the like.

[0033] The first processor 150A may be configured as a single processor, or multiple processors may function as the first processor 150A. The first processor 150A executes a first control program PG1 to control each unit of the projector 1. For example, the first processor 150A outputs to the image processing unit 145 an instruction to execute image processing corresponding to operations received via the operation unit 131 and the remote control 5, and parameters used for this image processing. The parameters include, for example, geometric correction parameters for correcting geometric distortion of the image projected onto the projection target QB. The first processor 150A also controls the light source driving unit 121 to turn on and off the light source unit 111, and adjusts the output of the light source unit 111, i.e., the light intensity.

[0034] The first processor 150A may be configured as an SoC integrated with part or all of the first memory 150B and other circuits. The first processor 150A may also be configured as a combination of a CPU that executes programs and a DSP that executes predetermined arithmetic processing. All of the functions of the first processor 150A may be implemented in hardware, or may be configured using a programmable device.

[0035] The image processing unit 145 and the frame memory 143 can be configured, for example, by an integrated circuit. Integrated circuits include large-scale integration (LSI), application-specific integrated circuits (ASIC), and programmable logic devices (PLD). PLDs include, for example, field-programmable gate arrays (FPGA). An integrated circuit may also include an analog circuit as part of its configuration, or may be a combination of a processor and an integrated circuit. A combination of a processor and an integrated circuit is called a microcontroller (MCU), SoC, system LSI, chipset, etc.

[0036] The image processing unit 145 loads image data corresponding to the projection image PM into the frame memory 143. The frame memory 143 includes a plurality of banks. Each bank has a storage capacity sufficient to store one frame's worth of image data. The frame memory 143 is configured, for example, by an SDRAM (Synchronous Dynamic Random Access Memory).

[0037] The image processing unit 145 performs image processing on the image data expanded in the frame memory 143, such as resolution conversion processing, resizing processing, distortion correction, shape correction processing, digital zoom processing, and adjustment of the color tone and brightness of the image. The image processing unit 145 also generates a vertical synchronization signal by converting the input frame frequency of the vertical synchronization signal into a drawing frequency. The generated vertical synchronization signal is called an output synchronization signal. The image processing unit 145 outputs the generated output synchronization signal to the light modulation device driving unit 122.

[0038] Next, the processing executed by the second control unit 21 of the personal computer 2 will be described with reference to FIGS. 3 is a screen diagram showing an example of an operation screen 400 including an operation image PR. The second control unit 21 displays the operation screen 400 on the display 31 of the display device 3. The operation screen 400 includes an operation image PR. The operation image PR is arranged approximately in the center of the operation screen 400. The second control unit 21 generates an operation image PR by superimposing a grid image GA representing a grid on the captured image PT. The grid image GA is generated by the second control unit 21, for example. The grid image GA corresponds to an example of a "first image."

[0039] In FIG. 3, a case will be described in which the "first image" is a grid image GA, but the embodiment is not limited to this. As will be described with reference to FIG. 6, the "first image" may be composed of a grid image GA and a grid image GB. In other words, the "first image" may be composed of multiple grid images. Furthermore, the shape of the grid is not limited to a rectangle, and may be a triangle, etc. Furthermore, the "first image" is not limited to a grid image. The "first image" may be any image that divides the captured image PT into multiple regions.

[0040] The second control unit 21 causes the camera 160 of the projector 1 to capture an image of an area including the projection object QB and generate a captured image PT. The projection object QB is an object onto which the projector 1 projects a projection image PM. The second control unit 21 acquires the captured image PT from the camera 160.

[0041] 3, the captured image PT includes images corresponding to the projection object QB, the first object QA, and the second object QC. In the following description, for convenience, the image corresponding to the projection object QB may be referred to as the projection object QB, the image corresponding to the first object QA may be referred to as the first object QA, and the image corresponding to the second object QC may be referred to as the second object QC. 3, the projection object QB is a building located approximately in the center of the captured image PT. The first object QA is a building located on the left side of the projection object QB. The second object QC is a tree located in the foreground on the right side of the projection object QB.

[0042] The grid image GA is an image of grid lines that define a grid. In the grid image GA, four line segments extending in the left-right direction are arranged at equal intervals in the up-down direction, and five line segments extending in the up-down direction are arranged at equal intervals in the left-right direction. The four line segments extending in the left-right direction and the five line segments extending in the up-down direction represent grid lines. Furthermore, of the four line segments extending in the left-right direction, the line segment located at the top is located at the top end of the captured image PT. Of the four line segments extending in the left-right direction, the line segment located at the bottom is located at the bottom end of the captured image PT. Of the five line segments extending in the up-down direction, the line segment located at the rightmost is located at the right end of the captured image PT. Of the five line segments extending in the up-down direction, the line segment located at the leftmost is located at the left end of the captured image PT. In other words, the lattice image GA divides the captured image PT into three regions in the vertical direction and four regions in the horizontal direction, i.e., the lattice image GA divides the captured image PT into 12 regions. The 12 regions correspond to an example of "multiple regions." 3, each of the 12 regions is formed in a rectangular shape. Each of the 12 regions is approximately square.

[0043] Next, the first operation QP1 will be described with reference to Fig. 4. The left diagram of Fig. 4 is a screen diagram showing an example of the first operation QP1 on the operation screen 410. The first operation QP1 is an operation for selecting a first region R1, which is at least one region out of 12 regions partitioned by the grid shown in the grid image GA. Operation screen 410 differs from operation screen 400 shown in Fig. 3 in that it includes a cursor CR. The position of cursor CR displayed on operation screen 410 moves when the user operates a mouse (not shown). The mouse (not shown) is connected to the personal computer 2. An operation signal indicating operation of the mouse (not shown) is output to the second control unit 21 of the personal computer 2. The second control unit 21 controls the position at which cursor CR is displayed based on the operation signal indicating operation of the mouse (not shown).

[0044] On the operation screen 410, the cursor CR points to the first region R1 in the upper left of the 12 regions defined by the grid image GA. In this state, the user can select the first region R1 by, for example, left-clicking the mouse (not shown). The above operation corresponds to an example of the first operation QP1.

[0045] The right diagram in FIG. 4 shows an example of an operation screen 420 that is displayed on the display 31 of the display device 3 when the first operation QP1 is accepted. The operation screen 420 is displayed on the display 31 of the display device 3 by the second control unit 21. The operation screen 420 differs from the operation screen 400 shown in Fig. 3 in that a selection mark M1 is displayed in the first region R1. The selection mark M1 is, for example, a thick solid line surrounding the first region R1. The selection mark M1 indicates that the first region R1 has been selected as the mask region MR. The mask region MR is a region in which the second control unit 21 hides the projection image PM.

[0046] Next, the first operation QP1 will be further described with reference to Fig. 5. The left diagram of Fig. 5 is a screen diagram showing another example of the first operation QP1 on the operation screen 430. The first operation QP1 is, for example, an operation to select the second region R2, which is at least one region out of the 12 regions partitioned by the grid shown in the grid image GA. The second region R2 includes at least one region different from the first region R1. Furthermore, between the first region R1 and the second region R2, the regions other than the first region R1 and the second region R2 are arranged among the 12 regions. In FIG. 5, two regions are arranged in the left-right direction between the first region R1 and the second region R2. In other words, the first region R1 and the second region R2 are regions spaced apart from each other. 4 in that it includes a cursor CR. The second control unit 21 controls the position at which the cursor CR is displayed based on an operation signal indicating the operation of a mouse (not shown).

[0047] On the operation screen 430, the cursor CR points to the second region R2 in the middle right of the 12 regions partitioned by the grid image GA. In this state, the user can select the second region R2 by left-clicking the mouse (not shown). The above operation corresponds to an example of the first operation QP1. When such a first operation QP1 is received, the second control unit 21 displays a selection mark M2 in the second region R2 on the operation screen 430. The selection mark M2 is, for example, a thick solid line surrounding the second region R2. For convenience, the selection mark M2 is depicted in a thick dashed line on the operation screen 430. The selection mark M2 indicates that the second region R2 has been selected as the mask region MR.

[0048] Furthermore, when the cursor CR is pointing at the second region R2, the user presses the shift key on the keyboard (not shown) and also left-clicks the mouse (not shown), causing the second control unit 21 to select the third region R3. The keyboard (not shown) is connected to the personal computer 2. An operation signal indicating an operation of the keyboard (not shown) is output to the second control unit 21 of the personal computer 2. The second control unit 21 determines an area to be selected based on the operation signal indicating an operation of the keyboard (not shown). 5, when the user left-clicks the not-illustrated mouse while the not-illustrated shift key on the keyboard is not pressed, the second control unit 21 selects the second region R2. When the user left-clicks the not-illustrated mouse while the not-illustrated shift key on the keyboard is pressed, the second control unit 21 selects the third region R3. This operation corresponds to an example of the first operation QP1.

[0049] The right diagram of FIG. 5 shows an example of an operation screen 440 displayed on the display 31 of the display device 3. The operation screen 440 is displayed on the display 31 of the display device 3 by the second control unit 21. The operation screen 440 differs from the operation screen 400 shown in Fig. 3 in that a selection mark M3 is displayed in the third region R3. The selection mark M3 is, for example, a thick solid line surrounding the third region R3. The selection mark M3 indicates that the third region R3 has been selected as the mask region MR. The third region R3 includes the first region R1 and the second region R2 and is a region defined by the first region R1 and the second region R2. That is, the third region R3 is a rectangular region with the first region R1 located at its upper left corner and the second region R2 located at its lower right corner. In other words, the third region R3 is composed of the top four regions and the middle four regions of the 12 regions defined by the grid indicated by the grid image GA. That is, the third region R3 is composed of eight regions of the 12 regions defined by the grid indicated by the grid image GA. The third region R3 corresponds to an example of a "first region."

[0050] Also displayed on the left and right sides of FIG. 5 are interval selection marks MA1, MA2, and MA3, which are selected when changing the interval of the grid shown by the grid image GA. The interval selection mark MA1 is selected when the interval of the grid shown in the grid image GA is set to the first interval D1, which is the default interval. The interval selection mark MA2 is selected when changing the interval of the grid shown in the grid image GA from the first interval D1 to the second interval D2. The second interval D2 is, for example, 1 / 2 of the first interval D1. When the second operation QP2 is performed with the interval selection mark MA2 selected, one area partitioned by the grid shown in the grid image GA is divided into four areas. The interval selection mark MA3 is selected when changing the interval of the grid shown in the grid image GA from the first interval D1 to a third interval D3. The third interval D3 is, for example, 1 / 3 of the first interval D1. If the second operation QP2 is performed while the interval selection mark MA3 is selected, one area partitioned by the grid shown in the grid image GA is divided into nine areas.

[0051] Next, the second operation QP2 will be described with reference to Fig. 6. Fig. 6 is a screen diagram showing an example of the second operation QP2 on the operation screen 500. The second operation QP2 is an operation for changing the spacing of the grid shown in the grid image GA from the first spacing D1 to a second spacing D2 different from the first spacing D1. 6 shows an example of an operation screen 500 displayed on the display 31 of the display device 3. The operation screen 500 includes an operation image PR. The operation image PR is arranged substantially in the center of the operation screen 500. The second control unit 21 generates an operation image PR by superimposing a grid image GA representing a grid on the captured image PT. The grid image GA is generated by the second control unit 21, for example.

[0052] The grid image GA constituting the operation image PR shown in the left diagram of FIG. 6 has a narrower grid interval than the grid images GA constituting the operation image PR shown in FIGS. When the interval selection mark MA2 is selected on the grid image GA shown in FIGS. 3 to 5 and the second operation QP2 is performed, the grid image GA constituting the operation image PR shown in the left diagram of FIG. 6 is displayed. The grid image GA constituting the operation image PR shown in the left diagram of Fig. 6 is an image of grid lines that define a grid. In the grid image GA constituting the operation image PR shown in the left diagram of Fig. 6, seven line segments extending in the left-right direction are arranged at equal intervals in the up-down direction, and nine line segments extending in the up-down direction are arranged at equal intervals in the left-right direction. The seven line segments extending in the left-right direction and the nine line segments extending in the up-down direction represent grid lines. Furthermore, of the seven line segments extending in the left-right direction, the line segment located at the top is located at the top end of the captured image PT. Of the seven line segments extending in the left-right direction, the line segment located at the bottom is located at the bottom end of the captured image PT. Of the nine line segments extending in the up-down direction, the line segment located at the rightmost is located at the right end of the captured image PT. Of the nine line segments extending in the up-down direction, the line segment located at the leftmost is located at the left end of the captured image PT. In other words, the lattice image GA shown in the left diagram of Fig. 6 divides the captured image PT into six regions in the vertical direction and eight regions in the horizontal direction, i.e., the lattice image GA divides the captured image PT into 48 regions. The 48 regions correspond to an example of a "plurality of regions."

[0053] The operation screen 500 shown in the left diagram of FIG. 6 includes a fourth region R4 and a selection mark M4, and a fifth region R5 and a selection mark M5. The fourth region R4 includes the first object QA described with reference to Figure 3. The fourth region R4 is composed of six of the 48 regions partitioned by the grid shown in the grid image GA. The selection mark M4 is, for example, a thick solid line surrounding the fourth region R4. The selection mark M4 indicates that the fourth region R4 has been selected as the mask region MR. The fifth region R5 includes the second object QC described with reference to Figure 3. The fifth region R5 is composed of 12 of the 48 regions partitioned by the grid shown in the grid image GA. The selection mark M5 is, for example, a thick solid line surrounding the fifth region R5. The selection mark M5 indicates that the fifth region R5 has been selected as the mask region MR. The fourth region R4 and the fifth region R5 are each selected in the same manner as the third region R3 described with reference to FIG.

[0054] The operation screen 500 shown in the left diagram of FIG. 6 further includes a reference target mark MS. The reference target mark MS indicates that the fifth area R5 is the target area for instructions corresponding to an operation from the user. A fourth area R4 and a fifth area R5 are displayed on the operation screen 500 shown in the left diagram of FIG. 6. The reference target mark MS is added to the fifth area R5 on the operation screen 500. That is, the reference target mark MS indicates that the fifth area R5 is the target area for instructions corresponding to an operation from the user. The target mark MS is displayed on the display 31 of the display device 3 by the second control unit 21 in response to a user's operation of at least one of the mouse (not shown) and the keyboard (not shown). The target mark MS may be displayed on the display 31 of the display device 3 by the second control unit 21 when an area is selected by the first operation QP1.

[0055] Furthermore, with the interval selection mark MA2 selected, clicking on the referent mark MS with, for example, the mouse (not shown) changes the grid spacing of the fifth region R5 in which the referent mark MS is displayed from the first spacing D1 to the second spacing D2. That is, the second operation QP2 is, for example, an operation of selecting the interval selection mark MA2 and an operation of clicking on the referent mark MS with the mouse (not shown).

[0056] The pointing target mark MS may also be displayed in all of the regions selected by the first operation QP1. For example, the pointing target mark MS may be displayed in each of the fourth region and the fifth region R5.

[0057] When the second control unit 21 receives a second operation QP2 from the user while the operation screen 500 is displayed on the display 31 of the display device 3, the second control unit 21 displays an operation screen 510 shown in the right diagram of Fig. 6 on the display 31 of the display device 3. The second operation QP2 is an operation for changing the spacing of the grid shown in the grid image GA from the first spacing D1 to a second spacing D2 different from the first spacing D1.

[0058] 6, the fifth region R5 is the region targeted by instructions corresponding to user operations, and therefore, when the second operation QP2 is received, the second control unit 21 executes the following process: That is, the second control unit 21 changes the grid spacing indicated by the grid image GA included in the fifth region R5 from the first spacing D1 to the second spacing D2. The fifth region R5 corresponds to an example of a “first region.” The fourth region R4 corresponds to an example of a “second region.” An operation screen 510 shown in the right diagram of FIG. 6 shows the result of changing the grid spacing shown in the grid image GA included in the fifth region R5 from the first spacing D1 to the second spacing D2.

[0059] In the present embodiment, the case where second control unit 21 changes the spacing of the grid indicated by grid image GA included in fifth region R5 from first spacing D1 to second spacing D2 has been described, but the embodiment is not limited to this. For example, second control unit 21 may change the spacing of the grid that defines regions other than fifth region R5, among the 48 regions defined by the grid indicated by grid image GA, from first spacing D1 to second spacing D2.

[0060] As shown on the operation screen 510, the second control unit 21 displays a grid image GB in the fifth region R5. In this embodiment, the second interval D2 is 1 / 2 of the first interval D1. The grid image GB is an image of grid lines that define a grid. In the grid image GB, nine line segments extending in the left-right direction are arranged at equal intervals in the up-down direction, and seven line segments extending in the up-down direction are arranged at equal intervals in the left-right direction. The nine line segments extending in the left-right direction and the seven line segments extending in the up-down direction represent grid lines. Furthermore, of the nine line segments extending in the left-right direction, the topmost line segment is located at the top end of the fifth region R5. Of the nine line segments extending in the left-right direction, the bottommost line segment is located at the bottom end of the fifth region R5. Of the seven line segments extending in the up-down direction, the rightmost line segment is located at the right end of the fifth region R5. Of the seven line segments extending in the up-down direction, the leftmost line segment is located at the left end of the fifth region R5. In other words, the lattice image GB divides the fifth region R5 into eight regions in the vertical direction and six regions in the horizontal direction, i.e., the lattice image GB divides the fifth region R5 into 48 regions.

[0061] The second control unit 21 superimposes the grid image GA and the grid image GB on the captured image PT to generate an operation image PR, and displays the operation image PR on the display 31 of the display device 3. Specifically, the second control unit 21 superimposes the grid image GA on an area other than the fifth area R5 of the captured image PT, and superimposes the grid image GB on the fifth area R5 of the captured image PT to generate an operation image PR, and displays the operation image PR on the display 31 of the display device 3. The grid image GB forms part of the "first image." The grid images GA and GB correspond to examples of the "first image."

[0062] The user determines the mask region MR by repeatedly performing the first operation QP1 on at least one region out of the 48 regions included in the fifth region R5. As a result of receiving the first operation QP1, the second control unit 21 displays the operation screen 520 shown in FIG.

[0063] 7 is a screen diagram of the operation screen 520 showing an example of the determined mask region MR. The mask region MR is composed of a first mask region MR1 and a second mask region MR2. The first mask region MR1 corresponds to the first object QA. The second mask region MR2 corresponds to the second object QC.

[0064] 8 is an image diagram showing an example of a mask image MC that hides the mask region MR of the projection image PM. The mask image MC is composed of a first mask image MC1 and a second mask image MC2. The first mask image MC1 is a mask image MC that hides the first mask region MR1 of the projection image PM. The second mask image MC2 is a mask image MC that hides the second mask region MR2 of the projection image PM.

[0065] The mask image MC is an image in which the mask area MR of the projection image PM is hidden by, for example, making the mask area MR a black monochrome image. The process of placing the mask image MC in the mask area MR is referred to as "mask processing."

[0066] In FIG. 8, the first mask image MC1 hides the first mask region MR1 of the projection image PM. Therefore, the first mask image MC1 prevents, for example, the image light PL from being projected onto the first object QA included in the first mask region MR1. As a result, the image light PL projected onto the first object QA and the reflected light reflected by the first object QA can be prevented from reducing the visibility of the projection image PM. Furthermore, for example, when a captured image PT capturing a projection object QB onto which various known pattern images are projected is used to correct the position of the projection image QB relative to the projection object QB, the correction accuracy can be improved. For example, this is because it can reduce the phenomenon in which the exposure of the camera 160 becomes difficult to adjust to the projection object QB due to the captured reflected light. 8, the second mask image MC2 hides the second mask region MR2 of the projection image PM. Therefore, the second mask image MC2 prevents, for example, the image light PL from being projected onto the second object QC included in the second mask region MR2. As a result, it is possible to prevent the image light PL from being projected onto the second object QC and the light reflected by the second object QC from reducing the visibility of the projection image PM. Furthermore, for example, when a captured image PT captured of a projection object QB onto which various known pattern images are projected is used to correct the position of the projection image QB relative to the projection object QB, the correction accuracy can be improved.

[0067] In Fig. 8, the outline of the projection area of ​​the projection image PM is indicated by a thick solid line. In Fig. 8, the projection image PM includes a first mask image MC1 and a second mask image MC2. The second control unit 21 generates the projection image PM in which a portion of the pattern image or the content image is hidden by superimposing the first mask image MC1 and the second mask image MC2 on the pattern image or the content image. In other words, the first mask image MC1 and the second mask image MC2 hide a portion of the projection image PM.

[0068] The second control unit 21 determines the area of ​​the projection image PM that corresponds to the first mask area MR1. Specifically, the second control unit 21 determines the area of ​​the projection image PM that corresponds to the first mask area MR1 based on the correspondence between the coordinate system of the projector 1 and the coordinate system of the camera 160. Here, the first mask area MR1 is illustrated as an example, but for any coordinate in the coordinate system of the camera 160, it is possible to similarly determine the corresponding coordinate in the coordinate system of the projector 1.

[0069] The second control unit 21 determines the correspondence between the coordinate system of the projector 1 and the coordinate system of the camera 160 based on the captured image PT of the projection target QB onto which the pattern image is projected as the projection image PM and the pattern image. Determining the correspondence between the coordinate system of the projector 1 and the coordinate system of the camera 160 means generating a transformation matrix for projective transformation from one coordinate system to the other. The coordinate system of the projector 1 is a coordinate system that indicates a position on the projected image PM by the projector 1. An example of the coordinate system of the projector 1 is a two-dimensional coordinate system with the upper left corner of the projected image as the origin. The coordinate system of the camera 160 is a coordinate system that indicates a position on the captured image PT. A specific example of the coordinate system of the camera 160 is a two-dimensional coordinate system with the upper left corner of the captured image as the origin.

[0070] In this embodiment, the above correspondence is specified in advance. The above correspondence can be specified even in the presence of reflected light. Since the coordinate system of the captured image PT displayed on the display 31 can be expressed in the coordinate system of the camera 160, the coordinate system of the camera 160 is projected into the coordinate system of the projector 1 based on the above correspondence, thereby specifying the position of the projected image PM corresponding to the position in the operation image PR and determining the area of ​​the projected image PM corresponding to the first mask area MR1.

[0071] Next, the processing of the second control unit 21 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the processing of the second control unit 21. 9 illustrates a case where the captured image PT is the first captured image PT1, but the embodiment is not limited to this. The second control unit 21 may, for example, receive a third operation QP3 for selecting the first captured image PT1 or the second captured image PT2. When the second captured image PT2 is selected by the third operation QP3, the second control unit 21 displays the second captured image PT2 as the captured image PT. In other words, the second control unit 21 generates an operation image PR by superimposing a grid image GA representing a grid on the second captured image PT2. The first captured image PT1 is a captured image PT including the projection object QB onto which the projection image PM is not projected, and the second captured image PT2 is a captured image PT including the projection object QB onto which the projection image PM is projected.

[0072] As shown in FIG. 9, first, in step S101, the second control unit 21 generates an operation image PR by superimposing a grid image GA on a captured image PT, and displays the operation image PR on the display 31 of the display device 3. Next, in step S103, the second control unit 21 accepts a first operation QP1 for selecting a first region R1. The first operation QP1 is, for example, an operation for selecting the first region R1, which is at least one region among 48 regions partitioned by the grid shown in the grid image GA. Next, in step S105, the second control unit 21 selects the first region R1.

[0073] Next, in step S107, the second control unit 21 determines whether or not a first operation QP1 for selecting the second region R2 has been accepted. If the second control unit 21 determines that the first operation QP1 for selecting the second region R2 has not been accepted (step S107; NO), the process proceeds to step S111. If the second control unit 21 determines that the first operation QP1 for selecting the second region R2 has been accepted (step S107; YES), the process proceeds to step S109. Then, in step S109, the second control unit 21 selects a second region R2. The second region R2 includes at least one region different from the first region R1. Furthermore, among the 48 regions, regions other than the first region R1 and the second region R2 are arranged between the first region R1 and the second region R2.

[0074] Next, in step S111, the second control unit 21 determines whether or not a second operation QP2 has been received. The second operation QP2 is an operation for changing the grid spacing shown in the grid image GA in a selected region, for example, the fifth region R5 shown in Fig. 6, from the first spacing D1 to a second spacing D2 different from the first spacing D1. If the second control unit 21 determines that the second operation QP2 has not been accepted (step S111; NO), the process proceeds to step S115. If the second control unit 21 determines that the second operation QP2 has been accepted (step S111; YES), the process proceeds to step S113. Then, in step S113, the second control unit 21 changes the grid spacing shown by the grid image GA in the selected region from the first spacing D1 to a second spacing D2 different from the first spacing D1.

[0075] Next, in step S115, the second control unit 21 accepts a selection operation of the mask region MR. The selection operation of the mask region MR is performed by, for example, moving the mouse (not shown) and left-clicking the mouse (not shown) to select at least one region from among the multiple regions partitioned by the grid shown in the grid image GA. Next, in step S117, the second control unit 21 determines whether or not the selection of the mask region MR is complete. The second control unit 21 determines whether or not the selection of the mask region MR is complete, for example, via an image for accepting a completion operation (not shown) and based on operations from the user using the mouse (not shown) and the keyboard (not shown).

[0076] If the second control unit 21 determines that the selection of the mask region MR is not complete (step S117; NO), the process returns to step S107. If the second control unit 21 determines that the selection of the mask region MR is complete (step S117; YES), the process proceeds to step S119. Then, in step S119, the second control unit 21 determines the area of ​​the projection image PM corresponding to the area selected as the mask area MR as the mask area MR. Then, the second control unit 21 forms a mask image MC that hides the projection image PM in the mask area MR, and then the process ends.

[0077] Step S101 corresponds to an example of "displaying an operation image." Steps S103 and S107 correspond to an example of "accepting a first operation." Step S111 corresponds to an example of "accepting a second operation." Step S119 corresponds to an example of "determining a mask area."

[0078] [Embodiment and Effects] As described above with reference to Figures 1 to 9, the display control method of this embodiment includes displaying, on a display device 3 different from the projector 1, an operation image PR in which a first image including a grid image GA representing a grid is superimposed on a captured image PT captured by the projector 1 of an area including the projection object QB onto which the projector 1 projects the projection image PM; receiving, via the operation image PR, a first operation QP1 that selects a first area R1, which is at least one area among a plurality of areas partitioned by the grid; and determining, by mask processing, an area of ​​the projection image PM corresponding to the area of ​​the captured image PT indicated by the first area R1 or an area among the plurality of areas other than the first area R1, as a mask area MR in which the projection image PM is not displayed.

[0079] That is, a first operation QP1 that selects the first region R1 is received via the operation image PR, and the region of the projection image PM that corresponds to the region of the captured image PT indicated by the first region R1 or a region other than the first region R1 among the multiple regions is determined to be the mask region MR. Therefore, the mask region MR can be easily determined, which improves user convenience.

[0080] Further, in the display control method, receiving a first operation QP1 includes receiving an operation to select a first region R1, which is at least one region among the plurality of regions, and receiving an operation to select a second region R2, which includes at least one region different from the first region R1, among the plurality of regions, and between the first region R1 and the second region R2, regions among the plurality of regions other than the first region R1 and the second region R2 are arranged. Therefore, among the multiple regions, regions other than the first region R1 and the second region R2 are disposed between the first region R1 and the second region R2, and the first region R1 and the second region R2 can be easily selected. In other words, the first region R1 and the second region R2, which are regions spaced apart from each other, can be easily selected. Therefore, the mask region MR can be easily determined. For example, the region of the projected image PM corresponding to the region of the captured image PT indicated by the first region R1 and the second region R2 can be determined as the mask region MR. Furthermore, for example, the region of the projected image PM corresponding to the region of the captured image PT indicated by the region of the multiple regions other than the first region R1 and the second region R2 can be determined as the mask region MR.

[0081] In addition, the display control method further includes accepting a second operation QP2 that changes the grid spacing shown by the grid image GA from a first spacing D1 to a second spacing D2 different from the first spacing D1, and displaying the operation image PR includes displaying an operation image PR in which the grid image GA showing the grid with the second spacing D2 is superimposed on the captured image PT. Therefore, it is possible to easily display an operation image PR in which a grid image GA showing a grid of the second interval D2 is superimposed on the captured image PT. Therefore, for example, by setting the second interval D2 narrower than the first interval D1, it is possible to make each area partitioned by the grid shown by the grid image GA smaller. As a result, it is possible to determine the mask area MR in detail.

[0082] Furthermore, in the display control method, when a second operation QP2 is received after the first operation QP1 is received and before the area of ​​the projection image PM is determined to be the mask area MR, displaying the operation image PR includes changing the spacing of the grid that divides the first area R1 to the second spacing D2. Therefore, the spacing of the grid that divides the first region R1 can be easily changed to the second spacing D2. Therefore, for example, by setting the second spacing D2 to a spacing narrower than the first spacing D1, it is possible to reduce the size of each region divided by the grid indicated by the grid image GA included in the first region R1. As a result, the mask region MR can be determined more precisely.

[0083] Further, in the display control method, when a second operation QP2 is received after the first operation QP1 is received and before the area of ​​the projection image PM is determined to be the mask area MR, displaying the operation image PR includes changing the spacing of the grid that divides the areas other than the first area R1 among the multiple areas to a second spacing D2. Therefore, the spacing of the grid that divides the regions other than the first region R1 among the plurality of regions can be easily changed to the second spacing D2. Therefore, for example, by setting the second spacing D2 to a spacing narrower than the first spacing D1, it is possible to reduce the size of each region divided by the grid indicated by the grid image GA included in the regions other than the first region R1 among the plurality of regions. As a result, the mask region MR can be determined more precisely.

[0084] In addition, in the display control method, the captured image PT is a first captured image PT1 including a projection object QB onto which the projection image PM is not projected, or a second captured image PT2 including a projection object QB onto which the projection image PM is projected, and the method further includes accepting a third operation QP3 to select the first captured image PT1 or the second captured image PT2, and displaying the operation image PR includes displaying the first captured image PT1 or the second captured image PT2 selected by the third operation QP3 as the captured image PT. Therefore, the first captured image PT1 or the second captured image PT2 selected by the third operation QP3 can be displayed as the captured image PT. The first captured image PT1 is a captured image PT including the projection object QB onto which the projection image PM is not projected. The second captured image PT2 is a captured image PT including the projection object QB onto which the projection image PM is projected. This improves user convenience.

[0085] The second control program PG2 in this embodiment causes the second processor 21A to execute processing including: displaying, on a display device 3 different from the projector 1, an operation image PR in which a first image including a grid image GA representing a grid is superimposed on an captured image PT captured of an area including the projection object QB onto which the projector 1 projects the projection image PM; accepting, via the operation image PR, a first operation QP1 to select a first area R1, which is at least one area among the multiple areas partitioned by the grid; and determining, as a mask area MR in which the projection image PM is not displayed by mask processing, an area in the captured image PT indicated by the first area R1 or an area among the multiple areas other than the first area R1. Therefore, the second control program PG2 according to this embodiment can achieve the same effects as the display control method according to this embodiment.

[0086] The personal computer 2 of this embodiment includes a second processor 21A that performs the following operations: displaying, on a display device 3 different from the projector 1, an operation image PR in which a first image including a grid image GA representing a grid is superimposed on an captured image PT captured of an area including a projection object QB onto which the projector 1 projects the projection image PM; accepting, via the operation image PR, a first operation QP1 that selects a first area R1, which is at least one area among multiple areas partitioned by the grid; and determining, by mask processing, an area of ​​the projection image PM corresponding to the area of ​​the captured image PT indicated by the first area R1 or an area among the multiple areas other than the first area R1, as a mask area MR in which the projection image PM is not displayed. Therefore, the personal computer 2 according to this embodiment can achieve the same effects as the display control method according to this embodiment.

[0087] [Other embodiments] The above-described embodiment is a preferred embodiment, but is not limited to the above-described embodiment, and various modifications are possible within the scope of the gist of the present invention.

[0088] In this embodiment, the "display control device" is configured as a personal computer 2, but the embodiment is not limited to this. For example, the "display control device" may be configured as a tablet terminal, a smartphone, or the like. Also, for example, the "display control device" may be configured as a server device, or the like.

[0089] In this embodiment, a case will be described in which the personal computer 2 controls the display device 3, but the embodiment is not limited to this. For example, the projector 1 may control the display device 3. Also, for example, a server device or the like may control the display device 3.

[0090] In this embodiment, a case where the projection image PM includes a first mask region MR1 and a second mask region MR2 will be described, but the embodiment is not limited to this. It is sufficient that the projection image PM includes at least a portion of the first mask region MR1 and at least a portion of the second mask region MR2. Also, for example, the projected image PM may include only one mask region MR, or for example, the projected image PM may include three or more mask regions MR.

[0091] In this embodiment, the projection subject QB, the first subject QA, and the second subject QC are included in the range in which the projector 1 projects the projection image PM and the range in which the camera 160 captures the image, but the embodiment is not limited to this. There may be multiple projection subjects QB, or there may be one or three or more subjects. Also, a portion of an object may be the projection subject QB, and another portion may be the first subject QA. For example, the wall of a building may be the projection subject QB, and a window of the same building may be the first subject QA.

[0092] In this embodiment, a case will be described in which the first region R1 and the second region R2 selected by the first operation QP1 are determined as the mask region MR, but the embodiment is not limited to this. A region other than the region selected by the first operation QP1 may be determined as the mask region MR. In this case, for example, the second control unit 21 receives, via the operation image PR, a first operation QP1 that selects, as the first region R1, a portion of the projection object QB that does not overlap with the first object QA and the second object QC. The second control unit 21 determines, as the mask region MR, the region other than the first region R1, i.e., the region of the projection image PM corresponding to the unselected region, to be hidden.

[0093] 1 and 2 show functional configurations, and the specific implementation form is not particularly limited. In other words, it is not necessary to implement hardware corresponding to each functional unit individually, and it is also possible to configure a system in which a single processor executes a program to realize the functions of multiple functional units. Furthermore, some of the functions realized by software in the above embodiments may be realized by hardware, or some of the functions realized by hardware may be realized by software. In addition, the specific detailed configurations of each unit of the projector 1, personal computer 2, and display device 3 may be changed as desired without departing from the spirit of the present invention.

[0094] 9 are divided according to the main processing content to facilitate understanding of the processing of the second control unit 21 of the personal computer 2. The division method and names of the processing units shown in the flowchart of FIG. 9 are not limiting, and the processing can be divided into more processing units according to the processing content, or one processing unit can be divided to include more processes. The processing order of the above flowchart is also not limited to the example shown in the figure.

[0095] The display control method of the display device 3 can be realized by causing the second processor 21A included in the personal computer 2 to execute a second control program PG2 corresponding to the display control method of the display device 3. The second control program PG2 can also be recorded on a computer-readable recording medium. The recording medium may be a magnetic or optical recording medium or a semiconductor memory device. Specific examples include portable or fixed recording media such as a flexible disk, HDD, CD-ROM (Compact Disk Read Only Memory), DVD, Blu-ray (registered trademark) Disc, magneto-optical disk, flash memory, and card-type recording medium. The recording medium may also be a non-volatile storage device such as RAM, ROM, or HDD, which is an internal storage device of the personal computer 2. The second control program PG2 may be stored in a server device or the like, and the second control program PG2 may be downloaded from the server device to the personal computer 2, thereby implementing the display control method for the display device 3.

[0096] [Note] A summary of this disclosure is provided below. (Appendix 1) A display control method including: displaying, on a display device other than the projector, an operation image in which a first image including a grid image representing a grid is superimposed on an image captured of an area including a projection object onto which a projected image is to be projected by a projector; accepting, via the operation image, a first operation to select a first area which is at least one area among a plurality of areas partitioned by the grid; and determining, by mask processing, an area of ​​the projected image corresponding to an area of ​​the captured image indicated by the first area or an area other than the first area among the plurality of areas, as a mask area in which the projected image is not displayed.

[0097] This allows the mask area to be easily determined, improving user convenience.

[0098] (Appendix 2) The display control method described in Appendix 1, wherein accepting the first operation includes accepting an operation to select the first area, which is at least one area among the plurality of areas, and accepting an operation to select a second area, which includes at least one area different from the first area, among the plurality of areas, and between the first area and the second area, areas among the plurality of areas other than the first area and the second area are arranged.

[0099] This allows the spacing of the grid that divides the first region to be easily changed to the second spacing. Therefore, for example, by setting the second spacing to a spacing narrower than the first spacing, it is possible to reduce the size of each region divided by the grid indicated by the grid image included in the first region. As a result, it is possible to determine the mask region more precisely.

[0100] (Appendix 3) The display control method described in Appendix 1 or Appendix 2, further including accepting a second operation to change the grid spacing from a first spacing to a second spacing different from the first spacing, and displaying the operation image includes displaying an operation image in which the first image including a grid at the second spacing is superimposed on the captured image.

[0101] This makes it easy to display an operation image in which a grid image showing a grid of the second interval is superimposed on the captured image. Therefore, for example, by setting the second interval to an interval narrower than the first interval, each area partitioned by the grid shown in the grid image can be made smaller. As a result, the mask area can be determined more precisely.

[0102] (Appendix 4) A display control method as described in Appendix 3, wherein when the second operation is received after the first operation is received and before the area of ​​the projection image is determined to be the mask area, displaying the operation image includes changing the spacing of the grid that divides the first area to the second spacing.

[0103] This allows the spacing of the grid that divides the first region to be easily changed to the second spacing. Therefore, for example, by setting the second spacing to a spacing narrower than the first spacing, it is possible to reduce the size of each region divided by the grid indicated by the grid image included in the first region. As a result, it is possible to determine the mask region more precisely.

[0104] (Appendix 5) The display control method described in Appendix 3, wherein when the second operation is received after the first operation is received and before the area of ​​the projection image is determined to be the mask area, displaying the operation image includes changing the spacing of a grid that divides areas other than the first area among the multiple areas to the second spacing.

[0105] This allows the spacing of the grid that divides the regions other than the first region among the plurality of regions to be easily changed to the second spacing. Therefore, for example, by setting the second spacing to a spacing narrower than the first spacing, it is possible to reduce the size of each region divided by the grid indicated by the grid image included in the regions other than the first region among the plurality of regions. As a result, it is possible to determine the mask region more precisely.

[0106] (Appendix 6) The display control method according to any one of Supplementary Note 1 to Supplementary Note 5, wherein the captured image is a first captured image including the projection object onto which the projection image is not projected, or a second captured image including the projection object onto which the projection image is projected, and further includes accepting a third operation to select the first captured image or the second captured image, and displaying the operation image includes displaying the first captured image or the second captured image selected by the third operation as the captured image.

[0107] This allows the first captured image or the second captured image selected by the third operation to be displayed as the captured image. The first captured image is a captured image including a projection object onto which no projection image is projected. The second captured image is a captured image including a projection object onto which a projection image is projected. This improves user convenience.

[0108] (Appendix 7) A display control program that causes a computer to execute processing including: displaying, on a display device different from the projector, an operation image in which a first image including a grid image representing a grid is superimposed on an image captured of an area including a projection object onto which a projected image is to be projected by a projector; accepting, via the operation image, a first operation to select a first area which is at least one area among a plurality of areas partitioned by the grid; and determining, by mask processing, an area of ​​the projected image corresponding to an area of ​​the captured image indicated by the first area or an area among the plurality of areas other than the first area, as a mask area in which the projected image is not displayed.

[0109] As a result, the display control program described in Supplementary Note 7 has the same effects as the display control method described in Supplementary Note 1.

[0110] (Appendix 8) A display control device including at least one processor that executes the following operations: displaying, on a display device other than the projector, an operation image in which a first image including a grid image representing a grid is superimposed on an captured image of an area including a projection object onto which a projector projects a projection image; accepting, via the operation image, a first operation to select a first area which is at least one area among a plurality of areas partitioned by the grid; and determining, by mask processing, an area of ​​the projection image corresponding to an area of ​​the captured image indicated by the first area or an area other than the first area among the plurality of areas, as a mask area in which the projection image is not displayed.

[0111] As a result, the display control device described in Supplementary Note 8 achieves the same effects as the display control method described in Supplementary Note 1. [Explanation of symbols]

[0112] 100...display control system, 1...projector, 113...projection optical system, 113A...first projection lens, 115...liquid crystal panel, 150...first control unit, 150A...first processor, 150B...first memory, 160...camera, 2...personal computer (display control device), 21...second control unit, 21A...second processor (computer), 21B...second memory, 3...display device, 31...display, 400, 410, 420, 430, 440, 500, 510, 520...operation screen, CR...cursor, D1...first interval, D2...second interval, GA...grid image (first image), GB...grid image (first image), MC...mask image, MC1...first mask image, MC2...second mask image, MR...mask area, MR1...first mask area, MR2...second mask area, PG1...first control program, PG2...second control program (display control program), PL...image light, PM...projected image, PR...operation image, PT...captured image, PT1...first captured image, PT2...second captured image, QA...first object, QB...projection object, QC...second object, QP1...first operation, QP2...second operation, QP3...third operation, R1...first area, R2...second area, R3...third area, R4...fourth area, R5...fifth area (first area).

Claims

1. displaying, on a display device different from the projector, an operation image in which a first image including a lattice image representing a lattice is superimposed on a captured image obtained by capturing an image of an area including a projection object onto which a projection image is to be projected by the projector; receiving, via the operation image, a first operation to select a first area that is at least one area among the plurality of areas partitioned by the grid; determining an area of ​​the projection image corresponding to an area of ​​the captured image indicated by the first area or an area other than the first area among the plurality of areas as a mask area in which the projection image is not displayed by mask processing; A display control method comprising:

2. Receiving the first operation includes: receiving an operation to select the first area, which is at least one area among the plurality of areas; receiving an operation to select a second area from the plurality of areas, the second area including at least one area different from the first area; Including, Between the first region and the second region, among the plurality of regions, regions other than the first region and the second region are arranged. The display control method according to claim 1 .

3. receiving a second operation to change the spacing of the grid from a first spacing to a second spacing different from the first spacing; Further comprising: Displaying the operation image includes: displaying an operation image in which the first image including the grid at the second intervals is superimposed on the captured image; The display control method according to claim 1 .

4. When the second operation is received after the first operation is received and before the area of ​​the projection image is determined as the mask area, Displaying the operation image includes: changing the spacing of the lattice that defines the first region to the second spacing; The display control method according to claim 3 .

5. When the second operation is received after the first operation is received and before the area of ​​the projection image is determined as the mask area, Displaying the operation image includes: changing the spacing of a lattice that partitions regions other than the first region among the plurality of regions to the second spacing; The display control method according to claim 3 .

6. the captured image is a first captured image including the projection object onto which the projection image is not projected, or a second captured image including the projection object onto which the projection image is projected, receiving a third operation to select the first captured image or the second captured image; Further comprising: Displaying the operation image includes: displaying the first captured image or the second captured image selected by the third operation as the captured image. The display control method according to any one of claims 1 to 5.

7. On the computer, displaying, on a display device different from the projector, an operation image in which a first image including a lattice image representing a lattice is superimposed on a captured image obtained by capturing an image of an area including a projection object onto which a projection image is to be projected by the projector; receiving, via the operation image, a first operation to select a first area that is at least one area among the plurality of areas partitioned by the grid; determining an area of ​​the projection image corresponding to an area of ​​the captured image indicated by the first area or an area other than the first area among the plurality of areas as a mask area in which the projection image is not displayed by mask processing; A display control program that causes a process including the above to be executed.

8. displaying, on a display device different from the projector, an operation image in which a first image including a lattice image representing a lattice is superimposed on a captured image obtained by capturing an image of an area including a projection object onto which a projection image is to be projected by the projector; receiving, via the operation image, a first operation to select a first area that is at least one area among the plurality of areas partitioned by the grid; determining an area of ​​the projection image corresponding to an area of ​​the captured image indicated by the first area or an area other than the first area among the plurality of areas as a mask area in which the projection image is not displayed by mask processing; A display control device comprising at least one processor that executes

Images