Display control method, display control program, and projector

Abstract

To display a first area that is an area the image of which is not normally picked up by a camera.SOLUTION: A display control method includes: causing a projector 100 to project and display a first pattern image PT1 on a screen SC; causing a camera 160 to pick up the first pattern image PT1 to acquire a picked-up image PD; specifying, on the basis of the picked-up image PD, a first area AR1 that is an area not normally detected in the first pattern image PT1; and causing the projector 100 to display a second pattern image PT2 obtained through processing on the first pattern image PT1 so that the first area AR1 can be distinguished from a second area AR2 other than the first area AR1 in the first pattern image PT1.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a display control method, a display control program, and a projector. [Background technology]

[0002] Patent Document 1 discloses a pattern image detection method including: displaying a first pattern image on a projector; acquiring a first captured image of the first pattern image captured by a camera; executing a process of detecting a second pattern image corresponding to the first pattern image from the first captured image; and, if detection of the second pattern image from the first captured image fails, performing a first correction process on the first captured image to increase the possibility of successful detection of the second pattern image, thereby acquiring a second captured image; and executing a process of detecting the second pattern image from the second captured image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2024-018681 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, for example, when the first pattern image is a dot pattern image including a plurality of dot images, if detection of the second pattern image from the first captured image fails, it is not possible to know which dot image has failed to be detected. Therefore, it is not possible to meet the need of a user who wants to know which dot image has failed to be detected. [Means for solving the problem]

[0005] One aspect of the present disclosure is a display control method including: causing a projector to project and display a first pattern image onto a projection surface; acquiring a captured image by causing a camera to capture the first pattern image; identifying a first region in the first pattern image that is not normally detected based on the captured image; and displaying a second pattern image, in which the first pattern image has been processed, on a display device including the projector so that the first region can be distinguished from a second region in the first pattern image other than the first region.

[0006] Another aspect of the present disclosure is a display control program that causes a computer to execute a process including: causing a projector to project and display a first pattern image onto a projection surface; acquiring a captured image by causing a camera to capture the first pattern image; identifying a first region that is an area of ​​the first pattern image that is not normally detected based on the captured image; and displaying a second pattern image, which is obtained by processing the first pattern image, on a display device including the projector so that the first region can be distinguished from a second region that is an area of ​​the first pattern image other than the first region.

[0007] Yet another aspect of the present disclosure is a projector that performs the following operations: projecting and displaying a first pattern image on a projection surface; acquiring a captured image by having a camera capture the first pattern image; identifying a first region in the first pattern image that is not normally detected based on the captured image; and projecting and displaying a second pattern image, which is obtained by processing the first pattern image, on the projection surface so that the first region can be distinguished from a second region in the first pattern image other than the first region. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a projector according to the present embodiment. [Figure 2]FIG. 2 is a diagram showing an example of the configuration of a control unit of the projector. [Figure 3] FIG. 4 is a diagram showing an example of a first pattern image. [Figure 4] FIG. 10 is a diagram showing an example of a second pattern image. [Figure 5] 10 is a flowchart showing an example of processing by a control unit. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, this embodiment will be described with reference to the drawings.

[0010] First, the configuration of the projector 100 will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of the projector 100. As shown in FIG. 1, the projector 100 includes a projection unit 110 and a drive unit 120 that drives the projection unit 110. The projection unit 110 forms an optical image and projects image light PL corresponding to the projection image PM onto the screen SC. The projection unit 110 also projects the image light PL so as to display the projection image PM on the screen SC. The screen SC corresponds to an example of a "projection surface."

[0011] In this embodiment, the "projection surface" is a screen SC, but the embodiment is not limited to this. The "projection surface" may be, for example, a building, a castle, or other structure. The "projection surface" may also be a three-dimensional object or a flat surface other than a structure, and may be a part of an object or the entire object.

[0012] The projection unit 110 includes a light source unit 111, a light modulation device 112, and a projection optical system 113. The drive unit 120 includes a light source drive unit 121 and a light modulation device drive unit 122. The light source unit 111 includes a solid-state light source 111A such as an LED (Light Emitting Diode) or a laser light source. In this embodiment, the light source unit 111 is described as including the solid-state light source 111A, but is not limited to this. The light source unit 111 may include a lamp light source such as a halogen lamp, a xenon lamp, or an ultra-high pressure mercury lamp instead of the solid-state light source 111A. In the following description, the solid-state light source 111A may be referred to as the light source 111A.

[0013] The light source unit 111 may also include a reflector and an auxiliary reflector that guide the light emitted by the light source 111A to the light modulation device 112. Furthermore, the light source unit 111 may also include a lens group for improving the optical characteristics of the projected light, a polarizing plate, or a dimming element that reduces the amount of light emitted by the light source 111A on the path leading to the light modulation device 112. The light source driving unit 121 is connected to the internal bus 107, and turns on and off the light source 111A of the light source unit 111 in accordance with instructions from the control unit 150 also connected to the internal bus 107, thereby controlling the output of the light source 111A.

[0014] The light modulation device 112 includes, for example, three liquid crystal panels 115 corresponding to the three primary colors of R, G, and B. R indicates red, G indicates green, and B indicates blue. That is, the light modulation device 112 includes a liquid crystal panel 115 corresponding to R light, a liquid crystal panel 115 corresponding to G light, and a liquid crystal panel 115 corresponding to B light. The light emitted by the light source unit 111 is separated into three color lights of RGB, and each color light is incident on a corresponding liquid crystal panel 115. Each of the three liquid crystal panels 115 is a transmissive liquid crystal panel, and modulates the light that passes through it to generate image light PL. The image light PL that has passed through each liquid crystal panel 115 and been modulated is combined by a combining optical system such as a cross dichroic prism, and is emitted to the projection optical system 113. In this embodiment, the light modulation device 112 is described as having a transmissive liquid crystal panel 115 as a light modulation element, but is not limited to this. The light modulation element may be a reflective liquid crystal panel or a digital micromirror device.

[0015] The light modulation device 112 is driven by a light modulation device driving unit 122. The light modulation device driving unit 122 is connected to the image processing unit 145. Image data corresponding to each of the primary colors R, G, and B is input to the light modulation device driving unit 122 from the image processing unit 145. The light modulation device driving unit 122 converts the input image data into a data signal suitable for the operation of the liquid crystal panel 115. Based on the converted data signal, the light modulation device driving unit 122 applies a voltage to each pixel of each liquid crystal panel 115, and draws an image on each liquid crystal panel 115.

[0016] The projection optical system 113 includes a projection lens 113A that forms an image of the incident image light PL on the screen SC, a mirror, etc. The projection optical system 113 also includes a zoom mechanism that enlarges or reduces the image projected onto the screen SC, a focus adjustment mechanism that adjusts the focus, and a lens shift mechanism that adjusts the projection direction of the image light PL.

[0017] The projector 100 also includes a camera 160 . The camera 160 includes an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The camera 160 captures an image of the screen SC and generates a captured image PD in accordance with instructions from the control unit 150. The control unit 150 causes the camera 160 to generate the captured image PD. The camera 160 also transmits the generated captured image PD to the control unit 150. The control unit 150 transmits the captured image PD to the personal computer 200. The captured image PD includes a projected image PM. The camera 160 is disposed near the projection lens 113A of the projector 100, for example. Camera 160 is further described with reference to FIG.

[0018] The projector 100 further includes an operation unit 131, a remote control receiver 133, an input interface 135, a storage unit 137, a communication interface 141, a frame memory 143, an image processing unit 145, and a control unit 150. The input interface 135, the storage unit 137, the communication interface 141, the image processing unit 145, the control unit 150, and the camera 160 are connected to each other via an internal bus 107 so as to be able to communicate data with each other.

[0019] The operation unit 131 includes various buttons and switches provided on the surface of the housing of the projector 100, generates operation signals corresponding to the operation of these buttons and switches, and outputs the operation signals to the input interface 135. The input interface 135 includes a circuit that outputs the operation signals input from the operation unit 131 by the user to the control unit 150.

[0020] The remote control light receiving unit 133 receives an infrared signal transmitted from the remote control 5, decodes the received infrared signal, and generates an operation signal. The remote control light receiving unit 133 outputs the generated operation signal to the input interface 135. The input interface 135 includes a circuit that outputs the operation signal input from the remote control light receiving unit 133 to the control unit 150.

[0021] The storage unit 137 is, for example, a magnetic recording device such as an HDD (Hard Disk Drive), or a storage device using a semiconductor storage element such as a flash memory or an SSD (Solid State Drive). The storage unit 137 stores programs executed by the control unit 150, data processed by the control unit 150, image data, etc.

[0022] The communication interface 141 is a communication interface that executes communication with the personal computer 200 in accordance with, for example, the USB (Universal Serial Bus) (registered trademark) standard. The communication interface 141 includes a connector for connecting a USB (registered trademark) cable and an interface circuit for processing signals transmitted through the connector. The communication interface 141 is an interface board having a connector and an interface circuit, and is connected to a main board on which the processor 150A of the control unit 150 and the like are mounted. Alternatively, the connector and interface circuit that constitute the communication interface 141 are mounted on the main board of the control unit 150. The communication interface 141 transmits, for example, a captured image PD to the personal computer 200. The communication interface 141 may be a communication interface that communicates with the personal computer 200 via a LAN (Local Area Network) or a WLAN (Wireless Local Area Network).

[0023] The control unit 150 includes a memory 150B and a processor 150A. The memory 150B is a storage device that non-volatilely stores programs and data executed by the processor 150A. The memory 150B is configured by a semiconductor storage element such as a magnetic storage device or a flash ROM (Read Only Memory), or other types of non-volatile storage device. The memory 150B may also include a RAM (Random Access Memory) that configures the work area of ​​the processor 150A. The memory 150B stores data processed by the control unit 150, the control program PG executed by the processor 150A, and the like.

[0024] The processor 150A may be configured as a single processor, or multiple processors may function as the processor 150A. The processor 150A executes a control program PG to control each unit of the projector 100. For example, the processor 150A outputs to the image processing unit 145 an instruction to execute image processing corresponding to operations received by the operation unit 131 and the remote control 5, and parameters used for this image processing. The parameters include, for example, geometric correction parameters for correcting geometric distortion of the image projected onto the screen SC. The processor 150A also controls the light source driving unit 121 to turn on and off the light source unit 111, and adjusts the output of the light source unit 111, i.e., the light intensity.

[0025] The processor 150A may be configured as a system on chip (SoC) integrated with part or all of the memory 150B and other circuits. The processor 150A may also be configured as a combination of a central processing unit (CPU) that executes programs and a digital signal processor (DSP) that executes predetermined arithmetic processing. All of the functions of the processor 150A may be implemented in hardware, or may be configured using a programmable device.

[0026] The image processing unit 145 and the frame memory 143 can be configured, for example, by an integrated circuit. Integrated circuits include large-scale integration (LSI), application-specific integrated circuits (ASIC), and programmable logic devices (PLD). PLDs include, for example, field-programmable gate arrays (FPGA). An integrated circuit may also include an analog circuit as part of its configuration, or may be a combination of a processor and an integrated circuit. A combination of a processor and an integrated circuit is called a microcontroller (MCU), SoC, system LSI, chipset, etc.

[0027] The image processing unit 145 loads image data corresponding to the projection image PM into the frame memory 143. The frame memory 143 includes a plurality of banks. Each bank has a storage capacity sufficient to store one frame's worth of image data. The frame memory 143 is configured, for example, by an SDRAM (Synchronous Dynamic Random Access Memory).

[0028] The image processing unit 145 performs image processing on the image data expanded in the frame memory 143, such as resolution conversion processing, resizing processing, distortion correction, shape correction processing, digital zoom processing, and adjustment of the color tone and brightness of the image. The image processing unit 145 also generates a vertical synchronization signal by converting the input frame frequency of the vertical synchronization signal into a drawing frequency. The generated vertical synchronization signal is called an output synchronization signal. The image processing unit 145 outputs the generated output synchronization signal to the light modulation device driving unit 122.

[0029] Next, the configuration of the control unit 150 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the control unit 150 of the projector 100. In this embodiment, a case will be described in which the processor 150A of the control unit 150 controls each unit of the projector 100 by reading and executing the control program PG from the memory 150B. The control unit 150 corresponds to an example of a "computer." The control program PG corresponds to an example of a "display control program."

[0030] The control unit 150 includes a first projection instruction unit 151 , an acquisition unit 152 , a specification unit 153 , a second projection instruction unit 154 , a position adjustment unit 155 , and an image storage unit 156 . Specifically, the processor 150A executes the control program PG to function as a first projection instruction unit 151, an acquisition unit 152, a specification unit 153, a second projection instruction unit 154, and a position adjustment unit 155. The processor 150A also executes the control program PG to cause the memory 150B to function as an image storage unit 156.

[0031] The image storage unit 156 stores the first pattern image PT1, the captured image PD, the second pattern image PT2, the query image PQ, and the third pattern image PT3. The first pattern image PT1 is a pattern image stored in advance in the image storage unit 156. The first pattern image PT1 is, for example, a pattern image including a plurality of dot images DT arranged at equal intervals. In the first pattern image PT1, the color of the background image of the plurality of dot images DT is, for example, white. The first pattern image PT1 corresponds to an example of a projection image PM. The first pattern image PT1 is an image for adjusting the projection image PM displayed on the screen SC by the projector 100. The dot images DT are part of the first pattern image PT1. The dot image DT is a circular image of a specific color CP. The specific color CP is, for example, blue. The specific color CP may also be green or red. The first pattern image PT1 will be further described with reference to FIG. At least one of the first pattern image PT1, the second pattern image PT2, and the third pattern image PT3 may be generated by the control program PG each time.

[0032] The captured image PD is acquired by the acquisition unit 152 from the camera 160 and stored in the image storage unit 156 by the acquisition unit 152. The second pattern image PT2 is generated by the specification unit 153 based on the first pattern image PT1 and the captured image PD. The second pattern image PT2 is also stored in the image storage unit 156 by the specification unit 153. The second pattern image PT2 corresponds to an example of the projection image PM.

[0033] The query image PQ is an image that queries the user as to whether or not to adjust the position of the first pattern image PT1 included in the first region AR1 based on an operation from the user. The query image PQ is stored in advance in the image storage unit 156. The query image PQ is included in the second pattern image PT2. The first region AR1 is identified by the identification unit 153. The third pattern image PT3 is generated by the position adjustment unit 155 and stored in the image storage unit 156 by the position adjustment unit 155. The third pattern image PT3 corresponds to an example of the projection image PM.

[0034] The first projection instruction unit 151 projects and displays the first pattern image PT1 on the screen SC. For example, first, the first projection instruction unit 151 reads out the first pattern image PT1 from the image storage unit 156. Then, the first projection instruction unit 151 causes the projection unit 110 to project and display the first pattern image PT1 on the screen SC.

[0035] The acquiring unit 152 acquires a captured image PD by having the camera 160 capture the first pattern image PT1 projected on the screen SC. For example, first, the acquiring unit 152 causes the camera 160 to capture the first pattern image PT1 projected on the screen SC. The camera 160 captures the first pattern image PT1 to generate a captured image PD. Then, the acquiring unit 152 acquires the captured image PD from the camera 160.

[0036] The specifying unit 153 specifies a first area AR1 that is an area that is not normally detected in the first pattern image PT1 based on the captured image PD. The "area that is not normally detected" is an area that includes a dot image DT that is not normally detected in the captured image PD, among the multiple dot images DT that make up the first pattern image PT1. The first area AR1 will be further described with reference to FIG.

[0037] The identification unit 153 detects dot images DT that are not detected normally. The dot images DT that are not detected normally are detected based on, for example, at least one of the contrast, position, size, shape, brightness, and color of each of the multiple dot images DT included in the captured image PD. For example, the identification unit 153 identifies, among the multiple dot images DT included in the captured image PD, a dot image DT whose brightness is lower than a threshold value as a dot image DT that is not detected normally.

[0038] Next, the reason why dot images DT that are not normally detected appear in the captured image PD will be described below. First, dot images DT that are not detected correctly occur due to the influence of external light. For example, dot images DT that are not detected correctly occur when the ambient light at the location where the screen SC is placed is bright. Also, for example, when spot light is incident on the screen SC, dot images DT that are not detected correctly occur. Furthermore, when external light such as fluorescent light is reflected on the screen SC, dot images DT that are not detected correctly occur.

[0039] Next, due to the influence of the screen SC, dot images DT that are not detected correctly occur. For example, if the curvature of the screen SC is larger than a predetermined value, in other words, if the radius of curvature of the screen SC is smaller than a predetermined value, dot images DT that are not detected correctly occur. Also, if the unevenness of the screen SC is larger than a predetermined value, dot images DT that are not detected correctly occur.

[0040] The identification unit 153 generates a second pattern image PT2 by processing the first pattern image so that the first region AR1 of the first pattern image PT1 can be distinguished from the second region AR2, which is the region of the first pattern image PT1 other than the first region AR1. For example, the specifying unit 153 generates the dot images DT included in the first area AR1 of the second pattern image PT2 in a first color C1. Also, the specifying unit 153 generates the dot images DT included in the second area AR2 of the second pattern image PT2 in a second color C2 different from the first color C1. The second color C2 is, for example, the same color as the specific color CP, which is the color of the dot image DT that constitutes the first pattern image PT1. In this embodiment, the second color C2 is, for example, blue. The first color C1 is, for example, red.

[0041] In this embodiment, the identification unit 153 generates the dot image DT included in the first area AR1 in the first color C1, and generates the dot image DT included in the second area AR2 in the first color C1, but the embodiment is not limited to this. The specification unit 153 may generate the dot image DT included in the first region AR1 in a display mode according to the cause of the improper detection. For example, if the cause of the improper detection of the dot image DT in the captured image PD is the influence of external light, the specification unit 153 generates the dot image DT included in the first region AR1 in a first color C1. For example, if the cause of the improper detection of the dot image DT in the captured image PD is the influence of the screen SC, the specification unit 153 generates the dot image DT included in the first region AR1 in a third color C3 different from the first color C1 and the second color C2. The third color C3 is, for example, green. In this case, the user can understand the cause of the dot image DT that is not normally detected in the captured image PD.

[0042] The specification unit 153 includes the query image PQ in the second pattern image PT2. For example, the specification unit 153 combines the second pattern image PT2 and the query image PQ and defines the combined image as the second pattern image PT2. Then, the specification unit 153 stores the second pattern image PT2 in the image storage unit 156. It is preferable that the query image PQ be placed in the second area AR2 of the second pattern image PT2. In this case, when the second pattern image PT2 is displayed on the screen SC by the second projection instruction unit 154, the user can visually recognize the dot images DT included in the first area AR1.

[0043] The second projection instruction unit 154 projects and displays the second pattern image PT2 generated by the specification unit 153 onto the screen SC. For example, first, the second projection instruction unit 154 reads out the second pattern image PT2 from the image storage unit 156. Then, the second projection instruction unit 154 causes the projection unit 110 to project and display the second pattern image PT2 onto the screen SC. The projector 100 corresponds to an example of a "display device."

[0044] The position adjustment unit 155 calculates geometric correction parameters based on the captured image PD obtained by capturing the first pattern image PT1 or the second pattern image PT2, and outputs the geometric correction parameters to the image processing unit 145. The image processing unit 145 generates corrected image data (not shown) representing a corrected image in which the first pattern image PT1 or the second pattern image PT2 has been subjected to geometric correction based on the geometric correction parameters. The corrected image includes, for example, the same number of cross marks as the number of dot images DT included in the first pattern image PT1 or the second pattern image PT2. Specifically, the corrected image includes at least one cross mark corresponding to at least one dot image DT included in the first region AR1 and at least one cross mark corresponding to at least one dot image DT included in the second region AR2. The center position of each cross mark is the same as the center position of the dot image DT. The cross marks are an example of adjustment points.

[0045] Here, when the position adjustment unit 155 receives an answer AU from the operation unit 131, it adjusts the position of the cross mark corresponding to the first pattern image PT1 included in the first region AR1 in accordance with the user's operation. The answer AU indicates that the position of the first pattern image PT1 included in the first region AR1 is adjusted based on the user's operation. The first pattern image PT1 included in the first region AR1 is the dot image DT included in the first region AR1 in the first pattern image PT1. This allows the user to manually perform geometric correction on the first region AR1 that was not subjected to geometric correction due to an improperly detected dot image DT. The cross marks correspond to examples of "adjustment points."

[0046] When the first region AR1 in the first pattern image PT1 includes a plurality of dot images DT, the position adjustment unit 155 accepts an operation to select a cross mark corresponding to one of the plurality of dot images DT included in the first region AR1. Then, the position adjustment unit 155 accepts an operation to adjust the position of the selected cross mark. In other words, the position adjustment unit 155 moves the position of the selected cross mark in a direction corresponding to the user's operation by a distance corresponding to the user's operation.

[0047] By repeatedly receiving the above-described operation of selecting a dot image DT and the operation of adjusting the position, it is possible to adjust the positions of all of the multiple dot images DT included in the first area AR1. When the positions of all of the cross marks corresponding to the dot images DT included in the first area AR1 have been adjusted, the pattern image generated after adjustment is stored as a third pattern image PT3 in the image storage unit 156. Note that it is preferable that the display mode of the cross marks corresponding to the dot images DT included in the first area AR1 differs from the display mode of the cross marks corresponding to the dot images DT included in the second area AR2. This allows the user to easily determine which cross mark positions should be manually adjusted.

[0048] In this embodiment, a case will be described in which the position adjustment unit 155 adjusts the positions of the cross marks corresponding to the dot images DT included in the first area AR1 in accordance with a user operation when an answer AU is received from the operation unit 131, but the embodiment is not limited to this. Note that the answer AU indicates that the positions of the cross marks corresponding to the dot images DT included in the first area AR1 are adjusted based on a user operation. For example, if the dot images DT included in the first region AR1 satisfy a predetermined condition, the position adjustment unit 155 may adjust the positions of the cross marks corresponding to the dot images DT included in the first region AR1 in response to a user operation. The predetermined condition may be, for example, that the ratio of the dot images DT included in the first region AR1 to the dot images DT included in the first pattern image PT1 is equal to or less than a predetermined threshold value TH. The threshold value TH may be, for example, 10%. The ratio of the dot images DT included in the first region AR1 to the dot images DT included in the first pattern image PT1 may be, for example, the ratio of the number of dot images DT included in the first region AR1 to the number of dot images DT included in the first pattern image PT1. If the number of cross marks that the user must manually adjust is large, the effort required for manual adjustment may outweigh the benefits of geometric correction through manual adjustment. Therefore, by setting the above-mentioned predetermined condition as an example, the increase in the effort required for manual adjustment is reduced, improving usability.

[0049] Next, the first pattern image PT1 will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the first pattern image PT1. As shown in Fig. 3, the first pattern image PT1 is a pattern image including a plurality of dot images DT arranged at equal intervals. Each of the plurality of dot images DT is a blue circular image. For convenience, Fig. 3 will explain the case where the first pattern image PT1 is properly displayed on the screen SC. In Fig. 3, the blue dot images DT are indicated by light hatching.

[0050] 3 shows an X-axis and a Y-axis that are orthogonal to each other. The X-axis is a direction parallel to the long side direction of the screen SC, for example, the left-right direction of the screen SC. The Y-axis is a direction parallel to the short side direction of the screen SC, for example, the up-down direction of the screen SC. The positive direction of the X-axis indicates the rightward direction. The positive direction of the Y-axis indicates the downward direction.

[0051] 3, in the first pattern image PT1, the dot images DT are arranged at equal intervals in the X-axis direction and the Y-axis direction. In the first pattern image PT1, for example, 16 dot images DT are arranged in the X-axis direction. Also, in the first pattern image PT1, for example, 14 dot images DT are arranged in the Y-axis direction.

[0052] Next, the second pattern image PT2 will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the second pattern image PT2. As shown in Fig. 4, the second pattern image PT2 includes a first region AR1 and a second region AR2. The first region AR1 is a region in the first pattern image PT1 where the dot image DT is not detected normally. The second region AR2 is a region in the first pattern image PT1 where the dot image DT is detected normally. In other words, the second region AR2 is a region in the first pattern image PT1 other than the first region AR1.

[0053] 4, three dot images DT are included in the first area AR1. The specification unit 153 generates the dot images DT included in the first area AR1 in the second pattern image PT2 in red. Red corresponds to an example of the first color C1. The specifying unit 153 generates the dot images DT included in the second area AR2 of the second pattern image PT2 in blue, which corresponds to an example of the second color C2. In FIG. 4, the blue dot image DT is depicted with light hatching, and the red dot image DT is depicted with dark hatching.

[0054] The second pattern image PT2 also includes a query image PQ in the second area AR2. The query image PQ includes a text image that asks the user, for example, "Do you want to adjust the positions of the adjustment points that were not properly geometrically corrected?" The query image PQ also includes a YES button BY and a NO button BN that accept input in response to this query.

[0055] When the user performs an operation to select the YES button BY, for example, by pressing a specific key arranged on the operation unit 131, the position adjustment unit 155 adjusts the position of the cross mark corresponding to the dot image DT included in the first area AR1 in accordance with the user's operation. The operation to select the YES button BY corresponds to an example of an answer AU. The answer AU indicates that the position of the cross mark corresponding to the dot image DT included in the first area AR1 is adjusted based on the user's operation.

[0056] If the user performs an operation to select the NO button BN, for example, if another specific key located on the operation unit 131 is pressed, the position adjustment unit 155 does not perform the process of adjusting the position of the cross mark corresponding to the dot image DT included in the first area AR1 in accordance with the user's operation.

[0057] Next, the processing of the control unit 150 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the processing of the control unit 150. As shown in FIG. 5, first, in step S101, the first projection instruction unit 151 projects and displays the first pattern image PT1 on the screen SC. Next, in step S103, the acquisition unit 152 acquires a captured image PD by causing the camera 160 to capture the first pattern image PT1 projected onto the screen SC. Next, in step S105, the specifying unit 153 specifies a first area AR1, which is an area that is not normally detected, in the first pattern image PT1 in the captured image PD.

[0058] Next, in step S107, the specifying unit 153 processes the first pattern image PT1 to generate a second pattern image PT2 so that the first region AR1 in the first pattern image PT1 can be distinguished from the second region AR2. The second region AR2 is the region of the first pattern image PT1 other than the first region AR1. The specifying unit 153 also includes a query image PQ in the second pattern image PT2. The query image PQ is an image that asks the user whether or not to adjust the position of a cross mark corresponding to a dot image DT included in the first pattern image PT1 based on a user operation. Next, in step S109, the second projection instruction unit 154 projects and displays the second pattern image PT2 generated by the specification unit 153 onto the screen SC.

[0059] Next, in step S111, the position adjustment unit 155 determines whether or not to adjust the position of the cross mark corresponding to the dot image DT included in the first area AR1 in response to a user operation. If the position adjustment unit 155 determines that the position of the cross mark corresponding to the dot image DT included in the first area AR1 will not be adjusted in response to the user's operation (step S111; NO), the process then ends. If the position adjustment unit 155 determines that the position of the cross mark corresponding to the dot image DT included in the first area AR1 will be adjusted in response to the user's operation (step S111; YES), the process proceeds to step S113.

[0060] Then, in step S113, the position adjustment unit 155 determines whether or not an operation to adjust the position of a cross mark corresponding to an improperly detected dot image DT in the corrected image generated by the image processing unit 145 has been accepted. That is, in step S113, a corrected image including a plurality of cross marks is displayed instead of the second pattern image PT2. In the corrected image, the display mode (e.g., color) of the cross mark corresponding to an improperly detected dot image DT differs from the display mode (e.g., color) of the cross mark corresponding to a properly detected dot image DT. If it is determined that an operation to adjust the position of the cross mark corresponding to the dot image DT has not been received (step S113; NO), the process goes to a standby state. If it is determined that an operation to adjust the position of the cross mark corresponding to the dot image DT has been received (step S113; YES), the process proceeds to step S115. Next, in step S115, the position adjustment unit 155 adjusts the position of the dot image DT and the corresponding cross mark to the position corresponding to the operation accepted in step S113, and displays them. In this embodiment, the cross mark in the second area AR2 of the corrected image is fixed because the geometric correction is performed correctly based on the correctly detected dot image DT. Therefore, the position of the cross mark in the second area AR2 is used as a reference position when the user manually adjusts the position of the cross mark in the first area AR1.

[0061] Next, in step S117, the position adjustment unit 155 determines whether or not adjustment of the position of the dot image DT and the corresponding cross mark has been completed based on an operation from the user. If the position adjustment unit 155 determines that the adjustment of the position of the dot image DT and the corresponding cross mark is not complete (step S117; NO), the process returns to step S113. If the position adjustment unit 155 determines that the adjustment of the position of the dot image DT and the corresponding cross mark is complete (step S117; YES), the process then ends.

[0062] Step S101 corresponds to an example of "displaying a first pattern image." Step S103 corresponds to an example of "acquiring a captured image." Step S105 corresponds to an example of "identifying a first region." Step S109 corresponds to an example of "displaying a second pattern image."

[0063] [Embodiment and Effects] As described above with reference to Figures 1 to 5, the display control method of this embodiment includes causing the projector 100 to project and display the first pattern image PT1 onto the screen SC, acquiring a captured image PD by causing the camera 160 to capture the first pattern image PT1, identifying a first area AR1 in the captured image PD, which is an area in the first pattern image PT1 that is not normally detected, and causing the projector 100 to display a second pattern image PT2 obtained by processing the first pattern image PT1 so that the first area AR1 can be distinguished from a second area AR2, which is an area in the first pattern image PT1 other than the first area AR1.

[0064] That is, the projector 100 displays the second pattern image PT2 obtained by processing the first pattern image PT1 so that the first region AR1 can be distinguished from the second region AR2, which is a region of the first pattern image PT1 other than the first region AR1. The first region AR1 is a region of the first pattern image PT1 that is not detected correctly. The second region AR2 is a region of the first pattern image PT1 other than the first region AR1. Therefore, the user can visually recognize the first area AR1, which is an area that is not normally detected, and can easily understand that the first pattern image PT1 included in the first area AR1 is not normally detected.

[0065] Furthermore, in the above-described display control method, displaying the second pattern image PT2 on the projector 100 means causing the projector 100 to display the portion of the second pattern image PT2 in the first area AR1 in a first color C1, and causing the projector 100 to display the portion of the second pattern image PT2 in the second area AR2 in a second color C2 different from the first color C1. Therefore, the portion of the second pattern image PT2 in the first region AR1 is displayed in the first color C1, and the portion of the second pattern image PT2 in the second region AR2 is displayed in the second color C2 different from the first color C1, so that the user can easily visually recognize the first region AR1. Therefore, the user can easily understand that the first pattern image PT1 included in the first region AR1 is not detected normally.

[0066] Furthermore, in the above-described display control method, displaying the second pattern image PT2 on the projector 100 causes the projector 100 to display the first pattern image PT1 included in the first area AR1 in a display mode that corresponds to the cause of the image not being detected normally. This allows the user to understand the reason why the detection is not performed correctly, thereby improving user convenience.

[0067] Furthermore, in the above display control method, the second pattern image PT2 includes a query image PQ that queries the user as to whether or not to adjust the position of the adjustment point corresponding to the portion of the first pattern image PT1 in the first area AR1 based on an operation from the user, and the method includes causing the projector 100 to display the second pattern image PT2 that includes the query image PQ, and, when a response indicating that the position of the portion of the first pattern image PT1 in the first area AR1 will be adjusted based on an operation from the user is received from the projector 100, adjusting the position of the adjustment point in accordance with the operation from the user. Therefore, the user can decide whether or not to adjust the position of the portion of the first pattern image PT1 in the first area AR1 based on the user's operation. This improves user convenience. Furthermore, the position of the adjustment point corresponding to the portion of the first pattern image PT1 in the first area AR1 can be adjusted in accordance with the user's operation. This improves user convenience.

[0068] The display control method also includes adjusting the position of the first pattern image PT1 included in the first area AR1 based on an operation from the user when the proportion of the first pattern image PT1 included in the first area AR1 to the first pattern image PT1 is equal to or less than a predetermined threshold value TH. Therefore, for example, when the position of the first pattern image PT1 included in the first area AR1 is within a range that can be adjusted by a user operation, the position of the first pattern image PT1 included in the first area AR1 is adjusted based on the user operation, thereby improving user convenience.

[0069] The control program PG of this embodiment causes the control unit 150 of the projector 100 to execute processing including causing the projector 100 to project and display the first pattern image PT1 onto the screen SC, acquiring a captured image PD by causing the camera 160 to capture the first pattern image PT1, identifying a first area AR1, which is an area in the first pattern image PT1 that is not normally detected, based on the captured image PD, and displaying a second pattern image PT2, which is obtained by processing the first pattern image PT1, on a display device including the projector 100 so that the first area AR1 can be distinguished from a second area AR2, which is an area in the first pattern image PT1 other than the first area AR1. Therefore, the control program PG according to this embodiment can achieve the same effects as the display control method according to this embodiment.

[0070] The projector 100 of this embodiment projects and displays a first pattern image PT1 on a screen SC, acquires a captured image PD by having a camera 160 capture the first pattern image PT1, identifies a first area AR1 that is an area in the first pattern image PT1 that is not normally detected based on the captured image PD, and projects and displays a second pattern image PT2 that is obtained by processing the first pattern image PT1 on the screen SC so that the first area AR1 can be distinguished from a second area AR2 that is an area in the first pattern image PT1 other than the first area AR1. Therefore, the projector 100 according to this embodiment can achieve the same effects as the display control method according to this embodiment.

[0071] [Other embodiments] The above-described embodiment is a preferred embodiment, but is not limited to the above-described embodiment, and various modifications are possible within the scope of the gist of the present invention.

[0072] In this embodiment, a case will be described in which the projector 100 includes the camera 160, but the embodiment is not limited to this. The camera 160 may be disposed separately from the projector 100. For example, the camera 160 may be supported on a tripod and disposed so as to capture an image of the screen SC. In this case, the camera 160 and the projector 100 need to be connected so as to be able to communicate with each other.

[0073] In this embodiment, the "pattern image" is a plurality of dot images DT arranged at equal intervals, but the embodiment is not limited to this. The "pattern image" may be a grid pattern image, a checkerboard pattern image, a phase shift pattern image, etc. Note that when the "pattern image" is a phase shift pattern, the portion of the pattern image is a pixel.

[0074] In this embodiment, a case will be described in which the control unit 150 of the projector 100 includes a first projection instruction unit 151, an acquisition unit 152, a specification unit 153, a second projection instruction unit 154, a position adjustment unit 155, and an image storage unit 156, but the embodiment is not limited to this. For example, an information processing device such as a personal computer 200, a tablet device, or a smartphone may include the first projection instruction unit 151, the acquisition unit 152, the specification unit 153, the second projection instruction unit 154, the position adjustment unit 155, and the image storage unit 156. In other words, some or all of the functions executed by the control unit 150 of the projector 100 may be executed by the information processing device. Furthermore, for example, in the case where the information processing device is equipped with a camera, such as a smartphone, the first pattern image PT1 may be captured by the camera of the smartphone instead of the camera 160 of the projector 100. In other words, the display control method according to this embodiment may be executed by the smartphone.

[0075] In this embodiment, the case where the "display device" is the projector 100 will be described, but the embodiment is not limited to this. The "display device" may be a display device different from the projector 100. The "display device" may be, for example, a display such as an LCD (Liquid Crystal Display) connected to the personal computer 200.

[0076] In the present embodiment, the case where the control unit 150 of the projector 100 displays the second pattern image PT2 on the projector 100 has been described, but the embodiment is not limited to this. The control unit 150 of the projector 100 may transmit the second pattern image PT2 to a server device. In this case, a plurality of second pattern images PT2 for each of the plurality of projectors 100 can be stored in the server device.

[0077] In this embodiment, a case will be described in which the second pattern image PT2 includes a query image PQ, but the embodiment is not limited to this. The second projection instruction unit 154 may, for example, display the query image PQ on the screen SC as an OSD (On-Screen Display). In this case, the YES button BY and the NO button BN can be easily operated. Also, in this case, the query image PQ can be displayed in an area of ​​the screen SC separate from the area in which the second pattern image PT2 is displayed. As a result, all dot images DT included in the second pattern image PT2 can be viewed.

[0078] 1 and 2 show functional configurations, and the specific implementation form is not particularly limited. In other words, it is not necessary to implement hardware corresponding to each functional unit individually, and it is also possible to implement a configuration in which a single processor executes a program to realize the functions of multiple functional units. Also, some of the functions realized by software in the above embodiments may be realized by hardware, or some of the functions realized by hardware may be realized by software. In addition, the specific detailed configurations of each unit of projector 100 can also be changed as desired without departing from the spirit of the invention.

[0079] 5 are divided according to the main processing content in order to make it easier to understand the processing of control unit 150 of projector 100. There is no limitation to the division method or names of the processing units shown in the flowchart of FIG. 5, and the processing can be divided into more processing units according to the processing content, or one processing unit can be divided so as to include more processes. Furthermore, the processing order of the above flowchart is not limited to the example shown in the figure.

[0080] The display control method can be realized by causing the processor 150A included in the projector 100 to execute a control program PG corresponding to the display control method. The control program PG can also be recorded on a computer-readable recording medium. The recording medium may be a magnetic or optical recording medium or a semiconductor memory device. Specific examples include portable or fixed recording media such as a flexible disk, HDD, CD-ROM (Compact Disk Read Only Memory), DVD, Blu-ray (registered trademark) Disc, magneto-optical disk, flash memory, and card-type recording medium. The recording medium may also be a non-volatile storage device such as RAM, ROM, or HDD, which is an internal storage device provided in projector 100. The display control method can also be realized by storing the control program PG in a server device or the like and downloading the control program PG from the server device to the projector 100.

[0081] [Note] A summary of this disclosure is provided below. (Supplementary Note 1) A display control method including: causing a projector to project and display a first pattern image onto a projection surface; acquiring a captured image by causing a camera to capture the first pattern image; identifying a first region in the captured image, which is a region of the first pattern image that is not normally detected; and displaying a second pattern image, which is obtained by processing the first pattern image, on a display device including the projector, so that the first region can be distinguished from a second region in the first pattern image other than the first region.

[0082] This allows the user to visually recognize the first region, which is the region that is not normally detected, and therefore makes it easy to understand that the first pattern image included in the first region is not normally detected.

[0083] (Appendix 2) The display control method described in Appendix 1, wherein displaying the second pattern image on the display device includes displaying the portion of the second pattern image in the first region on the display device in a first color, and displaying the portion of the second pattern image in the second region on the display device in a second color different from the first color.

[0084] This allows the user to easily visually recognize the first region, and therefore easily understand that the portion of the first pattern image in the first region is not being detected normally.

[0085] (Appendix 3) A display control method according to appendix 1 or appendix 2, in which the display device displays the second pattern image included in the first area in a display mode corresponding to the cause of the image not being detected normally.

[0086] This allows the user to understand the reason why detection is not performed correctly, thereby improving user convenience.

[0087] (Appendix 4) The display control method described in any one of Appendices 1 to 3, wherein the second pattern image includes a query image that queries a user as to whether or not to adjust the position of an adjustment point corresponding to a portion of the first pattern image included in the first region based on an operation from the user, and includes displaying the second pattern image including the query image on the display device; and, when a response indicating that the position of the portion of the first pattern image in the first region will be adjusted based on the operation from the user is received from the display device, adjusting the position of the adjustment point in accordance with the operation from the user.

[0088] This allows the user to decide whether or not to adjust the positions of the adjustment points corresponding to the portions of the first pattern image included in the first area based on the user's operation. This improves user convenience. Also, the positions of the adjustment points corresponding to the portions of the first pattern image included in the first area can be adjusted in accordance with the user's operation. This improves user convenience.

[0089] (Appendix 5) A display control method described in any one of Appendices 1 to 3, including adjusting the position of the first pattern image included in the first area based on an operation from a user when the proportion of the first pattern image included in the first area to the first pattern image is equal to or less than a preset threshold.

[0090] As a result, for example, when the amount of the first pattern image included in the first area AR1 is within a range that can be adjusted by user operation, the position of the first pattern image included in the first area is adjusted based on the user operation, thereby improving user convenience.

[0091] (Supplementary Note 6) A display control program that causes a computer to execute processing including: causing a projector to project and display a first pattern image on a projection surface; acquiring a captured image by causing a camera to capture the first pattern image; identifying a first region that is a region of the first pattern image that is not normally detected based on the captured image; and displaying a second pattern image, which is obtained by processing the first pattern image, on a display device including the projector so that the first region can be distinguished from a second region that is a region of the first pattern image other than the first region.

[0092] As a result, the display control program described in Supplementary Note 6 has the same effects as the display control method described in Supplementary Note 1.

[0093] (Supplementary Note 7) A projector that projects and displays a first pattern image on a projection surface; acquires a captured image by having a camera capture the first pattern image; identifies a first region that is a region of the first pattern image that is not normally detected based on the captured image; and projects and displays a second pattern image that is obtained by processing the first pattern image on the projection surface so that the first region can be distinguished from a second region that is a region of the first pattern image other than the first region.

[0094] As a result, the projector described in Supplementary Note 7 achieves the same effects as the display control method described in Supplementary Note 1. [Explanation of symbols]

[0095] 100...projector, 110...projection unit, 131...operation unit, 150...control unit, 150A...processor, 150B...memory, 151...first projection control unit, 152...acquisition unit, 153...identification unit, 154...second projection control unit, 155...position adjustment unit, 156...image memory unit, 160...camera, AR1...first area, AR2...second area, AU...answer, C1...first color, C2...second color, PD...captured image, PG...control program (display control program), PM...projected image, PQ...query image, PT1...first pattern image, PT2...second pattern image, SC...screen (projection surface), TH...threshold.

Claims

1. causing the projector to project and display the first pattern image on a projection surface; acquiring a captured image by causing a camera to capture the first pattern image; Identifying a first region in the captured image, which is a region that is not normally detected in the first pattern image; displaying a second pattern image obtained by processing the first pattern image on a display device including the projector so that the first region can be distinguished from a second region of the first pattern image other than the first region; A display control method comprising:

2. Displaying the second pattern image on the display device includes: displaying a portion of the second pattern image in the first region on the display device in a first color; displaying a portion of the second pattern image in the second region on the display device in a second color different from the first color; The display control method according to claim 1 .

3. Displaying the second pattern image on the display device includes: displaying the second pattern image included in the first region on the display device in a display mode corresponding to a cause of the abnormal detection; The display control method according to claim 1 .

4. the second pattern image includes a query image that queries a user as to whether or not to adjust positions of adjustment points corresponding to portions of the first pattern image included in the first region based on an operation from the user; displaying the second pattern image including the query image on the display device; adjusting a position of the adjustment point in response to a user operation when a response indicating that a position of a portion of the first pattern image in the first region is to be adjusted based on a user operation is received from the display device; The display control method according to claim 1 , further comprising:

5. adjusting a position of the first pattern image included in the first area based on an operation by a user when a ratio of the first pattern image included in the first area to the first pattern image is equal to or smaller than a preset threshold value; The display control method according to claim 1 , further comprising:

6. causing the projector to project and display the first pattern image on a projection surface; acquiring a captured image by causing a camera to capture the first pattern image; Identifying a first region that is an area that is not normally detected in the first pattern image based on the captured image; displaying a second pattern image obtained by processing the first pattern image on a display device including the projector so that the first region can be distinguished from a second region of the first pattern image other than the first region; A display control program that causes a computer to execute a process including the steps of:

7. projecting and displaying a first pattern image on a projection surface; acquiring a captured image by causing a camera to capture the first pattern image; Identifying a first region that is an area that is not normally detected in the first pattern image based on the captured image; projecting and displaying a second pattern image obtained by processing the first pattern image onto the projection surface so that the first region can be distinguished from a second region of the first pattern image other than the first region; Run the projector.

Images