Projection method

Abstract

To improve convenience for a user.SOLUTION: A projection method includes: projecting a first reference image on a projection target; acquiring a first picked-up image by an imaging apparatus having an image pick-up device picking up an image of the first reference image; projecting a second reference image on the projection target; acquiring a second picked-up image by the imaging apparatus picking up an image of the second reference image; determining, by a processing device, the kind of structured light to be projected from a projection device on the basis of a difference image between the first picked-up image and second picked-up image, the number of pixels of a light modulation element, and the number of pixels of the image pick-up device; and projecting the structured light from the projector according to a result of determining the kind of the structured light.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a projection method. [Background technology]

[0002] Patent Document 1 discloses a technology for identifying the correspondence between multiple pixels of an image projected from a projection device and multiple pixels of an image captured by an imaging device, using a gray code image, which is an example of structured light. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2022-174999 Summary of the Invention [Problem to be solved by the invention]

[0004] It is preferable to select the type of structured light according to the properties of the display surface or the installation environment of the projection device, etc. However, the technology described in Patent Document 1 does not allow the type of structured light to be changed. In addition, it is difficult for users to determine what type of structured light to select in what situation. [Means for solving the problem]

[0005] A projection method according to one embodiment of the present disclosure is a projection method in a projection device equipped with a light modulation element that modulates light from a light source, and includes: projecting a first reference image including a first portion having a first number of pixels onto a projection target; acquiring a first captured image by an imaging device having an imaging element capturing the first reference image; projecting a second reference image onto the projection target; acquiring a second captured image by the imaging device capturing the second reference image; determining by a processing device the type of structured light to be projected from the projection device based on a difference image between the first captured image and the second captured image, the first number of pixels, and a second number of pixels which is the number of pixels of a second portion in the first captured image corresponding to the first portion; and projecting the structured light from the projection device according to the result of determining the type of structured light.

[0006] A projection method according to another aspect of the present disclosure is a projection method in a projection device equipped with a light modulation element that modulates light from a light source, and includes displaying an interface image for accepting input of information regarding projection conditions, determining by a processing device the type of structured light to be projected from the projection device based on the projection conditions input using the interface image, and projecting the structured light from the projection device in accordance with the result of determining the type. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a diagram showing an outline of a system used in a projection method according to a first embodiment. [Figure 2] 1 is a block diagram of a projection device used in a projection method according to a first embodiment. [Figure 3] 4 is a flowchart showing the flow of a projection method according to the first embodiment. [Figure 4] 4 is a flowchart showing the flow of a projection method according to the first embodiment. [Figure 5] 10A and 10B are diagrams for explaining projection and capture of a first reference image. [Figure 6]10A and 10B are diagrams for explaining projection and capture of a second reference image. [Figure 7] 1A and 1B are diagrams for explaining structured light projection and imaging. [Figure 8] 10 is a flowchart showing the flow of a projection method according to a second embodiment. [Figure 9] 10 is a flowchart showing the flow of a projection method according to a second embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a message in step S170. [Figure 11] 10 is a flowchart showing the flow of a projection method according to a third embodiment. [Figure 12] 10 is a flowchart showing the flow of a projection method according to a third embodiment. [Figure 13] FIG. 10 is a diagram showing an example of a message in step S180. [Figure 14] 10 is a flowchart showing the flow of a projection method according to a fourth embodiment. [Figure 15] FIG. 10 is a diagram illustrating an example of an interface image. [Figure 16] 10 is a flowchart showing the flow of a projection method according to Modification 1. DETAILED DESCRIPTION OF THE INVENTION

[0008] Preferred embodiments of the present disclosure will be described below with reference to the accompanying drawings. Note that the dimensions and scale of each part in the drawings may differ from the actual dimensions and are shown schematically to facilitate understanding. Furthermore, the scope of the present disclosure is not limited to these embodiments unless otherwise specified in the following description to the effect that the present disclosure is limited.

[0009] 1. First embodiment 1-1. System Overview 1 is a diagram showing an outline of a system 100 used in a projection method according to the first embodiment. The system 100 is a projection system that projects a projection image G onto a projection target SC.

[0010] The projection target SC is formed, for example, by the surface of an object such as a screen. In the example shown in Fig. 1, the projection target SC has a rectangular outer shape. Note that the outer shape of the projection target SC is not limited to the example shown in Fig. 1 and can be any shape. Furthermore, the projection target SC is not limited to a flat surface and may be, for example, a concavely or convexly curved surface.

[0011] As shown in FIG. 1, the system 100 includes a projection device 10, an imaging device 20, and a terminal device 30.

[0012] The projection device 10 is a display device that projects a projection image G represented by video data IMG output from a terminal device 30 onto a projection target SC. In the example shown in FIG. 1, the projection image G is projected onto a rectangular area that covers substantially the entire area of ​​the projection target SC. The projection device 10 can also project the projection image G onto an area RP that encompasses the projection target SC. The area RP is, for example, an area on a wall on which the projection target SC is installed. Note that in FIG. 1, the projection image G is displayed with a cross-hatched pattern. The projection position and shape of the projection image G onto the projection target SC are not limited to the example shown in FIG. 1 and are arbitrary.

[0013] The projection device 10 of this embodiment has a function of controlling the operation of the imaging device 20 and a function of adjusting the shape of the projection image G using the imaging result of the imaging device 20.

[0014] The imaging device 20 is a digital camera having an imaging element 21, which will be described later, such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).

[0015] The imaging device 20 captures an image of an area RC. The area RC is an area that includes a projection image G that is projected onto a projection target SC. In the example shown in Fig. 1, the area RC includes the area RP. Note that the imaging device 20 may be a component of the projection device 10.

[0016] The terminal device 30 is a computer that has the function of supplying video data IMG to the projection device 10. In the example shown in Fig. 1, the terminal device 30 is a notebook computer. Note that the terminal device 30 is not limited to a notebook computer, and may be, for example, a desktop computer, a smartphone, a tablet terminal, a video playback device, a DVD (Digital Versatile Disk) player, a Blu-ray Disc player, a hard disk recorder, a television tuner device, a CATV (Cable television) set-top box, a video game console, or the like.

[0017] 1-2. Projection device Fig. 2 is a block diagram of a projection device 10 used in the projection method according to the first embodiment. In addition to the projection device 10, Fig. 2 also shows the connection state of an imaging device 20 and a terminal device 30 to the projection device 10. In the example shown in Fig. 2, the terminal device 30 includes a display device 31. The display device 31 is a display device including various display panels such as a liquid crystal display panel and an organic EL display panel.

[0018] 2, the projection device 10 has a storage device 11, a processing device 12, a communication device 13, an image processing circuit 14, an optical device 15, and an operation device 16. These are connected to each other so that they can communicate with each other.

[0019] The storage device 11 is a storage device that stores programs executed by the processing device 12 and data processed by the processing device 12. The storage device 11 includes, for example, a hard disk drive or a semiconductor memory. Note that part or all of the storage device 11 may be provided in an external storage device or server outside the projection device 10.

[0020] The memory device 11 stores a program PR1, first image information DG1, second image information DG2, first imaging data D1, second imaging data D2, differential image information D3, gray code information DGa, phase shift information DGb, panel information PPX, camera information CPX, and correspondence information DC.

[0021] The program PR1 is a program for executing a projection method, which will be described in detail later.

[0022] The first image information DG1 is information representing a first reference image G1 (described later). The first reference image G1 is an image for measuring the contrast of the projection image G, and is a uniform image of a light color such as white. The first reference image G1 is projected onto the projection target SC by the projection device 10. The first reference image G1 also includes a first portion PP1 having a first pixel number PN1. In this embodiment, the first pixel number PN1 is 1280 pixels, and the first portion PP1 is the upper edge of the first reference image G1. In this embodiment, the color of the first portion PP1 is white. The color of the first portion PP1 may be red, blue, or green, or a composite color of two of these colors. The first image information DG1 may be included in the gray code information DGa or the phase shift information DGb.

[0023] The second image information DG2 is information indicating a second reference image G2, which will be described later. The second reference image G2 is an image for measuring the contrast of the projection image G, and is a uniform image of a dark color such as black, which is projected onto the projection target SC by the projection device 10. The second image information DG2 may be included in the gray code information DGa or the phase shift information DGb.

[0024] The first imaging data D1 is information indicating a first captured image obtained by using the imaging device 20 to capture the first reference image G1 projected onto the projection target SC.

[0025] The second imaging data D2 is information indicating a second captured image obtained by using the imaging device 20 to capture the second reference image G2 projected onto the projection target SC.

[0026] The differential image information D3 is information indicating a differential image between the first captured image indicated by the first imaging data D1 and the second captured image indicated by the second imaging data D2.

[0027] The Gray code information DGa is information indicating a plurality of Gray code patterns used for measurement in the Gray code method as structured light GS, which will be described later.

[0028] The phase shift information DGb is information indicating a plurality of phase shift patterns used as structured light GS for measurement in the phase shift method.

[0029] In this embodiment, either a Gray code pattern or a phase shift pattern is selectively used as the structured light GS. Note that the types of structured light GS to be selected are not limited to Gray code patterns and phase shift patterns, and may be, for example, a binary code pattern, a dot pattern, a rectangular pattern, a polygonal pattern, a checkered pattern, or a random dot pattern. Furthermore, the number of types of structured light GS to be selected is not limited to two, and may be three or more.

[0030] The imaging data D0 is information indicating a captured image obtained by using the imaging device 20 to capture each of the multiple structured light beams GS sequentially projected onto the projection target SC.

[0031] The correspondence information DC is information indicating the correspondence relationship between the coordinates of the display coordinate system of the projection device 10 and the coordinates of the imaging coordinate system of the imaging device 20. The display coordinate system of the projection device 10 is a coordinate system in which the pixels of a light modulation element 15b (described later) are used as coordinate values. The imaging coordinate system of the imaging device 20 is a coordinate system in which the pixels of the imaging element of the imaging device 20 are used as coordinate values.

[0032] The panel information PPX is information indicating the number of pixels of the light modulation element 15b (described later). Note that the panel information PPX may include information other than the above information. The number of pixels of the light modulation element 15b may also be expressed as the resolution of the light modulation element 15b.

[0033] The camera information CPX is information that indicates the number of pixels or resolution of the image sensor 21. Note that the camera information CPX may include information other than the above information.

[0034] The processing device 12 has the function of controlling each part of the projection device 10 and the function of processing various data. The processing device 12 includes, for example, a processor such as a CPU (Central Processing Unit). The processing device 12 may be configured with a single processor or multiple processors. Some or all of the functions of the processing device 12 may be realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The processing device 12 may also be integrated with at least a part of the image processing circuit 14.

[0035] The communication device 13 is a communication device capable of communicating with various devices, and acquires video data IMG from the terminal device 30 and communicates with the imaging device 20. For example, the communication device 13 is a wired communication device such as a wired LAN (Local Area Network), USB (Universal Serial Bus), or HDMI (High Definition Multimedia Interface), or a wireless communication device such as LPWA (Low Power Wide Area), wireless LAN including Wi-Fi, or Bluetooth. "HDMI," "Wi-Fi," and "Bluetooth" are all registered trademarks.

[0036] The image processing circuit 14 is a circuit that performs necessary processing on the video data IMG from the communication device 13 and inputs the data to the optical device 15. The image processing circuit 14 has, for example, a frame memory (not shown), and loads the video data IMG into the frame memory, appropriately performs various processes such as resolution conversion, resizing, and distortion correction, and inputs the data to the optical device 15. The various processes here use the aforementioned correspondence information DC as appropriate. Note that the image processing circuit 14 may also perform processes such as OSD (On Screen Display) processing, where necessary, that generate image information for menu display or operation guides, etc., and combine it with the video data IMG.

[0037] The optical device 15 is a device that projects image light onto the projection target SC, and includes a light source 15a, a light modulation element 15b, and an optical system 15c.

[0038] The light source 15a includes a light source such as a halogen lamp, a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode), or a laser light source, and emits red, green, and blue light, respectively. The light modulation element 15b is a light modulator including three light modulation elements corresponding to red, green, and blue. Each light modulation element includes, for example, a transmissive liquid crystal panel, a reflective liquid crystal panel, or a DMD (Digital Mirror Device), and generates image light of each color by modulating light of the corresponding color. The image light of each color generated by the light modulation element 15b is combined by a color combining optical system to become full-color image light. Each light modulation element also has multiple pixels for forming the image light of each color. In this embodiment, the number of pixels PH of each light modulation element in the horizontal direction is 1280 pixels, and the number of pixels PV of each light modulation element in the vertical direction is 1080 pixels. The first pixel number PN1 described above corresponds to the pixel number PH in this embodiment. Therefore, the panel information PPX includes at least one of first panel information, which is information indicating the number of pixels PH, and second panel information, which is information indicating the number of pixels PV. In this embodiment, the panel information PPX only needs to include the first panel information. The first reference image G1 in this embodiment is image light formed using all vertical and horizontal pixels of each light modulation element. The number of pixels PH and the number of pixels PV of each light modulation element are not particularly limited. The optical system 15c is a projection optical system including a projection lens and the like that forms an image of the full-color image light from the light modulation element 15b onto the projection target SC and projects it.

[0039] The operation device 16 is a device that accepts operations from the user. For example, the operation device 16 includes an operation panel and a remote control light receiving unit, not shown. The operation panel is provided on the exterior housing of the projection device 10, and outputs a signal based on an operation from the user. The remote control light receiving unit receives an infrared signal from a remote control, not shown, decodes the infrared signal, and outputs a signal based on the operation of the remote control. The operation device 16 may be provided as needed, or may be omitted.

[0040] In the above-described projection device 10, the processing device 12 functions as a projection control unit 12a, an imaging control unit 12b, and a processing unit 12c by executing the program PR1 stored in the storage device 11. Therefore, the processing device 12 includes the projection control unit 12a, the imaging control unit 12b, and the processing unit 12c.

[0041] The projection control unit 12a controls the operation of the image processing circuit 14 and the optical device 15. More specifically, the projection control unit 12a controls the operation of the optical device 15 to project a projection image G onto the projection target SC. More specifically, the projection control unit 12a projects a first reference image G1 (described below) based on first image information DG1 onto the projection target SC, projects a second reference image G2 (described below) based on second image information DG2 onto the projection target SC, and projects structured light GS (described below) based on Gray code information DGa or phase shift information DGb onto the projection target SC.

[0042] The imaging control unit 12b controls the operation of the imaging device 20. More specifically, the imaging control unit 12b acquires first imaging data D1 by having the imaging device 20 capture a first reference image G1 projected onto the projection target SC, acquires second imaging data D2 by having the imaging device 20 capture a second reference image G2 projected onto the projection target SC, and acquires imaging data D0 by having the imaging device 20 capture structured light GS projected onto the projection target SC. The imaging control unit 12b then stores the acquired imaging data D0, first imaging data D1, and imaging data D0 in the storage device 11.

[0043] The processing unit 12c generates the correspondence information DC based on the imaging data D0, and adds processing necessary for generating the correspondence information DC based on the first imaging data D1 and the second imaging data D2 as appropriate.

[0044] More specifically, the processing unit 12c generates differential image information D3 based on the first imaging data D1 and the second imaging data D2. The processing unit 12c also selects one of the Gray code method and the phase shift method based on the differential image information D3. Furthermore, the processing unit 12c generates correspondence information DC based on the imaging data D0.

[0045] The Gray code method is characterized by being less susceptible to the effects of unevenness on the projection target SC and ambient light, but by the need to satisfy the sampling theorem. In contrast, the phase shift method requires a lower resolution image capture device 20, but is characterized by being more susceptible to the effects of unevenness on the projection target SC and ambient light.

[0046] Therefore, either the Gray code method or the phase shift method is selected depending on the projection conditions. Here, if the projection target SC has an uneven surface or is exposed to ambient light, the contrast ratio of the projected image G will be small. Furthermore, the sampling theorem is satisfied when the ratio PN2 / PN1 of the second pixel number PN2, which is the number of pixels in the second portion PC1 corresponding to the first portion PP1 in the first captured image, to the first pixel number PN1 is two or more.

[0047] 1-3. Projection method 3 and 4 are flowcharts showing the flow of the projection method according to the first embodiment. The projection method is performed by the processing device 12, which is an example of a "computer," executing the program PR1 using the system 100.

[0048] 3 and 4, the projection method in the projection device 10 includes steps S10 to S100. Here, the projection device 10 includes the light modulation element 15b that modulates the light from the light source 15a, as described above.

[0049] The processing device 12 first executes step S10. In step S10, panel information PPX and camera information CPX are acquired. This acquisition is performed, for example, by reading the panel information PPX and camera information CPX when they are stored in advance in the storage device 11. The camera information CPX may be acquired through communication with the imaging device 20. The panel information PPX and camera information CPX may also be acquired based on user input to an interface image that accepts input of this information. The acquisition of the camera information CPX may also be omitted. Step S10 may be executed at any time before step S40, which will be described later.

[0050] After step S10, the processing device 12 executes step S20. In step S20, a difference image indicated by the difference image information D3 is acquired. Specifically, step S20 includes steps S21 to S25 in this order. Note that steps S23 and S24 may be executed before steps S21 and S22.

[0051] In step S21, the processing device 12 executes control so that the projection device 10 projects a first reference image G1 including a first portion PP1 indicated by the first image information DG1 onto the projection target SC. This projection is performed by the projection control unit 12a controlling the operations of the image processing circuit 14 and the optical device 15 based on the first image information DG1. A specific example of the first reference image G1 will be described later with reference to FIG. 5.

[0052] In step S22, the processing device 12 causes the imaging device 20 to capture the first reference image G1 projected from the projection device 10 onto the projection target SC, and acquires a first captured image. This imaging is performed by the imaging control unit 12b controlling the operation of the imaging device 20. Through this imaging, first imaging data D1 indicating the first captured image, which is the imaging result, is acquired, and the first imaging data D1 is stored in the storage device 11.

[0053] In step S23, the processing device 12 causes the projection device 10 to project the second reference image G2 indicated by the second image information DG2 onto the projection target SC. This projection is performed by the projection control unit 12a controlling the operations of the image processing circuit 14 and the optical device 15 based on the second image information DG2. A specific example of the second reference image G2 will be described later with reference to FIG. 6.

[0054] In step S24, the processing device 12 executes control so that the imaging device 20 captures the second reference image G2 projected from the projection device 10 onto the projection target SC. This imaging is performed by the imaging control unit 12b controlling the operation of the imaging device 20. Through this imaging, second imaging data D2 indicating the second captured image, which is the imaging result, is acquired, and the second imaging data D2 is stored in the storage device 11.

[0055] In step S25, the processing device 12 generates differential image information D3 based on the first imaging data D1 and the second imaging data D2 by, for example, calculating the difference between each pixel constituting the first captured image represented by the first imaging data D1 and each pixel constituting the second captured image represented by the second imaging data D2.

[0056] After step S20, the processing device 12 executes step S30. In step S30, the processing device 12 detects a projection region RP1 (described later) that is a region on the projection target SC onto which structured light GS is projected, based on one or both of the first imaging data D1 and the second imaging data D2.

[0057] After step S30, the processing device 12 executes step S35. In step S35, the processing device 12 analyzes the first captured data D1 to identify the first portion PP1, i.e., the second portion PC1, which is a portion corresponding to the upper side of the first reference image G1, in the first captured image. The processing device 12 may detect the second portion PC using a known algorithm, such as edge detection, and identify the coordinates of the pixels of the second portion PC in the coordinate system of the first captured image. Note that step S35 may be executed at any time after step S22 and before step S40, which will be described later.

[0058] After step S30, the processing device 12 executes step S1. Step S1 determines the type of structured light GS to be projected from the projection device 10 based on the differential image indicated by the differential image information D3, the first number of pixels PN1 of the first portion PP1 of the first reference image G1, and the second number of pixels PN2, which is the number of pixels of the second portion PC1 corresponding to the first portion PP1 in the first captured image. Specifically, step S1 includes steps S40 to S80. In this embodiment, the first number of pixels PN1 is equal to the number of pixels PH indicated by the first panel information included in the panel information PPX.

[0059] In step S40, the processing device 12 executes a process of determining whether a ratio PN2 / PN1 of a second number of pixels PN2, which is the number of pixels in a second portion PC1 corresponding to the first portion PP1 in the first captured image, to a first number of pixels PN1 in the first portion PP1, satisfies a first condition. The first condition is, for example, a condition that satisfies the sampling theorem, and specifically, the ratio (PN2 / PN1) of the second number of pixels PN2 to the first number of pixels PN1 is 2 or greater. That is, the first condition is that, in the coordinate system of the first captured image, one pixel in the first reference image G1 is observed by two pixels (two elements) of the image sensor 21. For example, in step S40, the processing device 12 determines whether the second portion PC1 corresponding to the first portion PP1 having 1920 pixels is 3840 pixels or greater in the coordinate system of the first captured image.

[0060] If the first condition is satisfied (step S40: YES), the processing device 12 executes step S50 to determine whether the gradation values ​​of the differential image indicated by the differential image information D3 satisfy a second condition. The second condition is, for example, a condition that the contrast ratio is equal to or greater than a predetermined threshold. Specifically, the second condition is that the average value of all pixel gradation values ​​constituting the differential image indicated by the differential image information D3 is equal to or greater than a predetermined threshold. If the average value is less than the predetermined threshold, the processing device 12 can determine that the differential image has many relatively dark areas.

[0061] If the second condition is satisfied (step S50: YES), the processing device 12 executes step S60, where it determines whether to prioritize the speed of the structured light method. This determination is made, for example, based on the setting of a mode related to the generation of correspondence information DC. The setting of the mode is made by receiving a user's input to an interface image (not shown), for example. The interface image has, for example, a button for selecting whether to prioritize the speed, or a button for determining whether to prioritize the use of a specific structured light method (in this embodiment, either the Gray code method or the phase shift method). Displaying the interface image may be performed in step S60 or may be performed before step S60.

[0062] In this way, in step S60, the processing device 12 receives from the user a setting of a mode for generating a correspondence between the pixels of the light modulation element 15b and the pixels of the imaging element 21 based on structured light, and if the first condition is met (step S40: YES) and the second condition is met (step S50: YES), determines the type of structured light to be projected from the projection device 10 according to the setting.

[0063] Here, the mode is either a first mode in which the number of structured light beams is M (M is a natural number equal to or greater than 2), or a second mode in which the number of structured light beams is N (N is a natural number greater than M). When the setting is the first setting corresponding to the first mode, the processing device 12 places emphasis on speed (step S60: YES). On the other hand, when the setting is the second setting corresponding to the second mode, the processing device 12 does not place emphasis on speed (step S60: NO).

[0064] If the first condition is not met (step S40: NO) or if the speed of the structured light method is prioritized (step S60: YES), the processing device 12 selects the phase shift method as the type of structured light to be projected from the projection device 10 in step S70. Thus, in step S70, if the ratio PN2 / PN1 does not satisfy the first condition (step S40: NO) and the gradation value of the differential image indicated by the differential image information D3 satisfies the second condition (step S50: YES), the processing device 12 determines that the type of structured light to be projected from the projection device 10 is a phase shift pattern. Furthermore, if the setting is the first setting corresponding to the first mode, the processing device 12 prioritizes speed (step S60: YES), and in step S70, the processing device 12 determines that the type of structured light to be projected from the projection device 10 is a phase shift pattern.

[0065] On the other hand, if the second condition is not met (step S50: NO) or if the speed of the structured light method is not emphasized (step S60: NO), the processing device 12 selects the Gray code method as the type of structured light to be projected from the projection device 10 in step S80. Thus, in step S80, if the ratio PN2 / PN1 satisfies the first condition (step S40: YES) and the gradation value of the differential image indicated by the differential image information D3 does not satisfy the second condition (step S50: NO), it is determined that the type of structured light to be projected from the projection device 10 is a Gray code pattern. Also, if the setting is the second setting corresponding to the second mode, it is determined that the speed is not emphasized (step S60: NO), and it is determined in step S80 that the type of structured light to be projected from the projection device 10 is a Gray code pattern.

[0066] After step S70 or step S80, the processing device 12 executes step S90. Step S90 executes acquiring a plurality of pieces of imaging data D0 by capturing, with the imaging device 20, each of a plurality of beams of structured light GS projected from the projection device 10 onto the projection target SC using the structured light method selected in step S70 or step S80. Specifically, step S90 includes steps S91 and S92.

[0067] In step S91, depending on the result of determining the type of structured light, a plurality of beams of structured light GS (described later) are sequentially projected from the projection device 10 onto the projection target SC. This projection is performed by the projection control unit 12a controlling the operations of the image processing circuit 14 and the optical device 15 based on the Gray code information DGa or the phase shift information DGb.

[0068] In step S92, each of a plurality of beams of structured light GS (described later) projected from the projection device 10 onto the projection target SC is imaged by the imaging device 20. This imaging is performed by the imaging control unit 12b controlling the operation of the imaging device 20. As a result of this imaging, imaging data D0 is generated, and the generated imaging data D0 is stored in the storage device 11.

[0069] After step S90, the processing device 12 executes step S100. In step S100, the coordinates of the display coordinate system of the projection device 10 are associated with the coordinates of the imaging coordinate system of the imaging device 20 based on the imaging data D0. This association is performed by the processing unit 12c using a known structured light method based on the imaging data D0. As a result of this association, correspondence information DC is generated, and the generated correspondence information DC is stored in the storage device 11.

[0070] 5 is a diagram for explaining the projection and capture of the first reference image G1. In step S21, as shown in FIG. 5, the first reference image G1 indicated by the first image information DG1 is projected from the projection device 10 onto the projection target SC. In the example shown in FIG. 5, the first reference image G1 is a uniform white image. Note that, although the outer edge of the first reference image G1 is located inside the outer edge of the region RP in FIG. 5, the outer edge of the first reference image G1 may coincide with the outer edge of the region RP.

[0071] In step S22, the first reference image G1 is captured by the imaging device 20 in the area RC that includes the first reference image G1 projected onto the projection target SC.

[0072] FIG. 6 is a diagram for explaining the projection and capture of the second reference image G2. In step S23, as shown in FIG. 6, the second reference image G2 indicated by the second image information DG2 is projected from the projection device 10 onto the projection target SC. In the example shown in FIG. 6, the second reference image G2 is a uniform black image. Note that in FIG. 6, the outer edge of the second reference image G2 is located inside the outer edge of the region RP, but the outer edge of the second reference image G2 may coincide with the outer edge of the region RP. However, it is preferable that the outer edge of the second reference image G2 coincide with the outer edge of the first reference image G1.

[0073] In step S24, the second reference image G2 is captured by the imaging device 20 in the area RC that includes the second reference image G2 projected onto the projection target SC.

[0074] In step S25, a difference image between the first reference image G1 and the second reference image G2 is generated. In step S30, edges of the first reference image G1 or the second reference image G2 are detected based on the first imaging data D1 and the second imaging data D2, thereby detecting a projection area RP1. Therefore, the range onto which the first reference image G1 or the second reference image G2 is projected is determined as the projection area RP1.

[0075] If the ratio PN2 / PN1 is 2 or greater, that is, if the first condition is met, the sampling theorem is satisfied, and therefore the phase shift method is selected.

[0076] Furthermore, if the gradation value of the difference image between the first reference image G1 and the second reference image G2 is equal to or greater than a predetermined threshold, i.e., if the second condition is satisfied, it can be determined that the projection target SC does not have any irregularities and that ambient light is not hitting the projection target SC. Therefore, in this case (step S50: YES), the phase shift method can be selected. On the other hand, if the gradation value of the difference image is less than the predetermined threshold, it can be determined that the projection target SC has irregularities or that ambient light is hitting the projection target SC. Therefore, in this case (step S50: NO), the Gray code method is selected.

[0077] 7 is a diagram for explaining the projection and imaging of structured light GS, which shows the structured light GS when the Gray code method is selected.

[0078] In step S91, as shown in Fig. 7, the structured light GS selected in step S70 or step S80 is projected from the projection device 10 onto the projection target SC. In the example shown in Fig. 7, the structured light GS is a Gray code pattern. Note that in Fig. 7, the outer edge of the structured light GS is located inside the outer edge of the region RP, but the outer edge of the structured light GS may also coincide with the outer edge of the region RP. However, it is preferable that the outer edge of the structured light GS coincide with the outer edges of the first reference image G1 and the second reference image G2.

[0079] In step S92, the image capturing device 20 captures an image of the structured light GS in the region RC that includes the structured light GS projected onto the projection target SC.

[0080] Although not shown, the above steps S91 and S92 are repeatedly executed for the number of frames of structured light DS. As a result, imaging data D0 is obtained. Thereafter, in step S100, correspondence information DC is generated based on the imaging data D0.

[0081] As described above, the projection method of this embodiment includes steps S21, S22, S23, S24, S1, and S91. This allows the processing device 12 to automatically determine the structured light to be projected from the projection device 10, thereby improving convenience for the user.

[0082] In this embodiment, as described above, the processing device 12 executes steps S40, S50, S70, and S80. Therefore, if the ratio PN2 / PN1 satisfies the first condition (step S40: YES) and the gradation value does not satisfy the second condition (step S50: NO), the type of structured light to be projected from the projection device 10 is determined to be a Gray code pattern. Also, if the ratio PN2 / PN1 does not satisfy the first condition (step S40: NO) and the gradation value of the differential image indicated by the differential image information D3 satisfies the second condition, the type of structured light to be projected from the projection device 10 is determined to be a phase shift pattern. In this way, it is possible to determine the type of structured light that is more accurate and appropriate, between the Gray code pattern and the phase shift pattern, depending on the projection conditions.

[0083] As described above, the processing device 12 also executes step S60. In step S60, the processing device 12 receives from the user a mode setting for generating a correspondence between the pixels of the light modulation element 15b and the pixels of the image sensor 21 based on structured light, and if the ratio PN2 / PN1 satisfies the first condition (step S40: YES) and the gradation value of the differential image indicated by the differential image information D3 satisfies the second condition (step S50: YES), the processing device 12 determines the type of structured light to be projected from the projection device 10 in accordance with the setting. This allows the type of structured light to be determined in accordance with the user's preference when the required measurement accuracy can be ensured for both the Gray code pattern and the phase shift pattern.

[0084] Furthermore, as described above, the mode is either a first mode in which the number of structured light beams is M (M is a natural number equal to or greater than 2) or a second mode in which the number of structured light beams is N (N is a natural number greater than M). If the setting is the first setting corresponding to the first mode (step S60: YES), the processing device 12 determines that the type of structured light to be projected from the projection device 10 is a phase shift pattern. If the setting is the second setting corresponding to the second mode (step S60: NO), the processing device 12 determines that the type of structured light to be projected from the projection device 10 is a Gray code pattern. In the above-described embodiment, if the required measurement accuracy can be ensured for both the Gray code pattern and the phase shift pattern, the type of structured light can be determined according to the user's preference.

[0085] 2. Second embodiment A second embodiment of the present disclosure will be described below. In the following exemplary embodiment, for elements whose actions and functions are similar to those of the first embodiment, the reference numerals used in the description of the first embodiment will be used, and detailed descriptions of each element will be omitted as appropriate.

[0086] 8 and 9 are flowcharts showing the flow of a projection method according to the second embodiment. This projection method is similar to the projection method of the first embodiment, except that it includes step S1A instead of step S1. Step S1A is similar to step S1 of the first embodiment, except that steps S110 to S170 are added.

[0087] In step S1A, after step S30, the processing device 12 sets each of the Q1 flag and the Q2 flag to 0 in step S110, and then executes step S40.

[0088] If the first condition is satisfied (step S40: YES), the processor 12 sets the Q1 flag to 1 and then proceeds to step S50, whereas if the first condition is not satisfied (step S40: NO), the processor 12 proceeds directly to step S50.

[0089] If the second condition is satisfied (step S50: YES), the processor 12 sets the Q2 flag to 1 and then proceeds to step S140, whereas if the second condition is not satisfied (step S50: NO), the processor 12 proceeds directly to step S140.

[0090] In step S140, the processing device 12 determines whether or not the condition that the Q1 flag is 1 and the Q2 flag is 0 is met. The condition is that the first condition is met and the second condition is not met. Therefore, if the condition that the Q1 flag is 1 and the Q2 flag is 0 is met (step S140: YES), the processing device 12 executes step S80, similarly to the first embodiment.

[0091] If the condition that the Q1 flag is 1 and the Q2 flag is 0 is not satisfied (step S140: NO), the processing device 12 determines in step S150 whether the condition that the Q1 flag is 0 and the Q2 flag is 1 is satisfied. The condition is that the first condition is not satisfied and the second condition is satisfied. Therefore, if the condition that the Q1 flag is 0 and the Q2 flag is 1 is satisfied (step S150: YES), the processing device 12 executes step S70, similarly to the first embodiment.

[0092] If the condition that the Q1 flag is 0 and the Q2 flag is 1 is not satisfied (step S140: NO), the processing device 12 determines in step S160 whether the condition that the Q1 flag is 1 and the Q2 flag is 1 is satisfied. The condition is that the first condition is satisfied and the second condition is satisfied. Therefore, if the condition that the Q1 flag is 1 and the Q2 flag is 1 is satisfied (step S160: YES), the processing device 12 executes step S60, similarly to the first embodiment.

[0093] If the condition that the Q1 flag is 1 and the Q2 flag is 1 is not met (step S160: NO), the processing device 12 executes step S170 and then proceeds to step S60. In step S170, the projection device 10 projects a message Ra, which will be described later.

[0094] 10 is a diagram showing an example of the message Ra in step S170. In step S170, the projection device 10 projects the image G-M1 onto the projection target SC.

[0095] Image G-M1 is a projected image G that includes a message Ra. The message Ra indicates that the accuracy of generating a correspondence between the pixels of the light modulation element 15b and the pixels of the image sensor 21 based on the structured light GS will decrease. In the example shown in FIG. 10, the message Ra displays the following text: "The environment is not suitable for measurement. Sufficient measurement accuracy may not be achieved." Note that the message Ra may be any message indicating that the accuracy of generating the correspondence will decrease, and is not limited to the example shown in FIG. 10.

[0096] The second embodiment described above also improves user convenience. In this embodiment, as described above, if the ratio PN2 / PN1 does not satisfy the first condition and the gradation value of the differential image indicated by the differential image information D3 does not satisfy the second condition, the processing device 12 causes the projection device 10 to project a message Ra indicating that the accuracy of generating the correspondence between the pixels of the light modulation element 15b and the pixels of the image sensor 21 based on structured light will decrease. This makes it possible to notify the user when the required measurement accuracy cannot be ensured for both the Gray code pattern and the phase shift pattern.

[0097] 3. Third embodiment A third embodiment of the present disclosure will be described below. In the following exemplary embodiments, elements that have the same actions and functions as those in the first embodiment will be designated by the same reference numerals as those in the first embodiment, and detailed descriptions of each element will be omitted where appropriate.

[0098] 11 and 12 are flowcharts showing the flow of a projection method according to the third embodiment. This projection method is similar to the projection method according to the first embodiment, except that it includes step S1B instead of step S1. Step S1B is similar to step S1 in the first embodiment, except that steps S110 to S160 and steps S180 and S190 are added. That is, step S1B is similar to step S1A in the second embodiment, except that it includes step S180 instead of step S170 and adds step S190.

[0099] In step S1B, if the condition that the Q1 flag is 1 and the Q2 flag is 1 is not met (step S160: NO), the processing device 12 in step S180 causes the projection device 10 to project an image G-M2, which will be described later. This image G-M2 includes a message Rb, which will be described later, in addition to the message Ra described above, and is capable of accepting a selection of whether or not to continue processing.

[0100] After step S180, the processor 12 determines in step S190 whether to continue or not, based on the selection result in step S180.

[0101] If the measurement is to be continued (step S190: YES), the processing device 12 proceeds to step S80. As a result, similar to the second embodiment, the user is notified that the required measurement accuracy cannot be guaranteed for both the Gray code pattern and the phase shift pattern, and then the Gray code method is executed. On the other hand, if the measurement is not to be continued (step S190: NO), the processing device 12 returns to step S10. This allows the measurement to be performed again after improving the projection conditions.

[0102] 13 is a diagram showing an example of the message in step S 180. In step S180, the projection device 10 projects the image G-M2 onto the projection target SC.

[0103] The image G-M1 is a projected image G that includes the message Rb and buttons B1 and B2 in addition to the message Ra of the second embodiment.

[0104] The message Rb includes potential solutions for improving accuracy. In the example shown in Fig. 13, the message Rb displays the text "If possible, please improve the following and re-evaluate," along with the text "Increase the camera resolution," "Move the camera closer to the projection surface," "Darken the room," and "Turn off the surrounding lights," which indicate potential solutions. Note that the message Rb is not limited to the example shown in Fig. 13 and may be any display that includes potential solutions for improving accuracy.

[0105] Button B1 is a button for continuing the process as is. When button B1 is operated, it is determined in step S190 that the process should continue (step S190: YES). Button B2 is a button for restarting the process. When button B2 is operated, it is determined in step S190 that the process should not continue (step S190: NO).

[0106] The third embodiment described above also improves user convenience. In this embodiment, as described above, the message Rb includes potential solutions for improving accuracy. This allows the user to be notified of solutions when the required measurement accuracy cannot be ensured for both the Gray code pattern and the phase shift pattern.

[0107] 4. Fourth embodiment A fourth embodiment of the present disclosure will be described below. In the following exemplary embodiments, elements whose actions and functions are similar to those of the first embodiment will be designated by the same reference numerals as those used in the description of the first embodiment, and detailed descriptions of each element will be omitted where appropriate.

[0108] 14 is a flowchart showing the flow of a projection method according to the fourth embodiment. This projection method is similar to the projection method of the first embodiment, except that it includes step S200 instead of steps S10 to S30 and step S1C instead of step S1. Step S1C is similar to step S1 of the first embodiment, except that it includes steps S210 and S220 instead of steps S40 to S60.

[0109] In the projection method of this embodiment, first, in step S200, the processing device 12 causes the projection device 10 to project an interface image G-M3, which will be described later. The interface image G-M3 can accept input of information related to projection conditions such as the brightness of the location where the projection target SC is installed and the presence or absence of unevenness.

[0110] After step S200, in step S210, the processing device 12 determines whether the location where the projection target SC is installed is a bright place.

[0111] If the location where the projection target SC is installed is not a bright location (step S210: NO), the processing device 12 determines in step S220 whether or not the projection target SC has irregularities.

[0112] If the projection target SC does not have any concaves or convexes (step S220: NO), the processing device 12 executes step S70. Therefore, if the location where the projection target SC is installed is not a bright location (step S210: NO) and the projection target SC does not have any concaves or convexes (step S220: NO), the phase shift method is selected.

[0113] On the other hand, if the location where the projection target SC is installed is bright (step S210: YES) or if the projection target SC has unevenness (step S220: YES), the processing device 12 executes step S80. Therefore, if the location where the projection target SC is installed is bright (step S210: YES) or if the projection target SC has unevenness (step S220: YES), the phase shift method is selected.

[0114] 15 is a diagram showing an example of interface image G-M3. Interface image G-M3 is a projection image G for receiving input of information related to projection conditions. Interface image G-M3 includes a message Rc and buttons B3, B4, and B5.

[0115] The message Rc is a display that prompts the user to input information about the projection conditions. In the example shown in FIG. 15, the message Rc displays the text "Please select the projection conditions." Note that the message Rc is not limited to the example shown in FIG. 15 and may be any display that prompts the user to input information about the projection conditions. Furthermore, the message Rc may be displayed as needed and may be omitted.

[0116] Button B3 is a button for setting, as a projection condition, that the location where the projection target SC is installed is a bright location. Button B4 is a button for setting, as a projection condition, that the projection target SC has unevenness. Button B5 is a button for setting, as a projection condition, that the location where the projection target SC is installed is not a bright location and that the projection target SC does not have unevenness. Note that the information regarding the projection conditions input in interface image G-M3 may be any information regarding the first condition and second condition, etc., of the first embodiment described above, and is not limited to the example shown in FIG. 15.

[0117] The above buttons B3, B4, and B5 can selectively accept operations from the user. When button B3 is operated, in step S210, it is determined that the location where the projection target SC is installed is a bright location (step S210: YES). As a result, the Gray code method is selected, as described above. When button B4 is operated, it is determined that the projection target SC has concaves and convexes (step S220: YES). As a result, the Gray code method is selected, as described above. When button B5 is operated, in step S210, it is determined that the location where the projection target SC is installed is not a bright location (step S210: NO), and in step S220, it is determined that the projection target SC does not have concaves and convexes (step S220: NO). As a result, the phase shift method is selected, as described above.

[0118] The fourth embodiment described above also improves user convenience. As described above, the projection method of this embodiment includes steps S200, S1C, and S91. Step S200 displays an interface image G-M3 for receiving input of information related to projection conditions. Step S1C causes the processing device 12 to determine the type of structured light to be projected from the projection device 10 based on the projection conditions input using the interface image G-M3. Step S91 causes the projection device 10 to project structured light GS based on the type determination. In this way, the user inputs the projection conditions, allowing an appropriate type of structured light GS to be selected, thereby improving user convenience.

[0119] 5. Variations The above-described embodiments can be modified in various ways. Specific modifications that can be applied to the above-described embodiments are exemplified below. Two or more embodiments arbitrarily selected from the following examples can be combined as appropriate within the scope of not mutually contradictory.

[0120] 5-1. Variation 1 16 is a flowchart showing the flow of the projection method according to Modification 1. This projection method is the same as the projection method according to the first embodiment, except that the execution order of steps S40 and S50 is reversed. Modification 1 described above can also improve convenience for users.

[0121] 5-2. Variation 2 In the above-described embodiment, an example is given in which the processing device 12 of the projection device 10 executes the program PR1, but this is not limited to this example. For example, the processing device of a computer communicatively connected to the projection device 10 and the imaging device 20 may execute the program PR1.

[0122] 5-3. Variation 3 In the above-described embodiment, the first pixel number PN1 is the number of pixels in the first portion PP1, which is the upper side of the first reference image G1. However, this is not limited to this example. For example, the first pixel number PN1 may be the number of pixels in any portion of the first reference image G1, such as the lower side, left side, right side, or center of the first reference image G1. As long as the processing device 12 knows the number of pixels in any portion of the first reference image G1, the portion that serves as the basis for the first pixel number is not particularly limited. Furthermore, the first pixel number PN1 may be smaller than 1,280 pixels. Furthermore, the first reference image G1 may be image light formed using a portion of the vertical pixels and a portion of the horizontal pixels of each light modulation element. The first pixel number PN1 may be the pixel number PV or a number of pixels smaller than the pixel number PV.

[0123] 5-4. Variation 4 In the above-described embodiment, the processing device 12 determines whether the ratio PN2 / PN1 is equal to or greater than 2 using the first pixel number PN1, which is the number of pixels in the first portion PP1 of the first reference image G1. However, the present invention is not limited to this. For example, the first pixel number PN1 may be the number of pixels in the first portion PP1' of an image other than the first reference image G1. That is, in the above-described embodiment, the first reference image G1 is used both to detect the projection area RP1 and to calculate the ratio PN2 / PN1. However, for example, the first reference image G1 may be used to detect the projection area RP1, and an image other than the first reference image G1 may be separately projected to calculate the ratio PN2 / PN1. The image other than the first reference image G1 used to calculate the ratio PN2 / PN1 may be projected at any timing before step S40.

[0124] 5-5. Variation 5 In the above-described embodiment, an example is given in which the correspondence information DC is used to adjust the projection image G, but this is not limited to this example. For example, the correspondence information DC may be used to display a uniform grid-like pattern or the like on the projection target SC, or may be used to reflect a three-dimensional model of the projection target SC as seen from the imaging device 20 in three-dimensional image editing software or the like, and then draw a picture on the model, or to display on a PC monitor or the like how the picture will look when viewed from the projection device 10, or to project it on the projection device 10.

[0125] 6. Supplementary Notes A summary of this disclosure is provided below.

[0126] (Supplementary Note 1) A first aspect, which is a preferred example of the projection method of the present disclosure, is a projection method in a projection device equipped with a light modulation element that modulates light from a light source, and includes: projecting a first reference image including a first portion having a first number of pixels onto a projection target; acquiring a first captured image by an imaging device having an imaging element capturing the first reference image; projecting a second reference image onto the projection target; acquiring a second captured image by the imaging device capturing the second reference image; determining by a processing device a type of structured light to be projected from the projection device based on a difference image between the first captured image and the second captured image, the first number of pixels, and a second number of pixels which is the number of pixels of a second portion in the first captured image that corresponds to the first portion; and projecting the structured light from the projection device according to the result of determining the type of structured light.

[0127] In the above aspect, the structured light to be projected from the projection device is automatically determined by the processing device, thereby improving convenience for the user.

[0128] (Appendix 2) A second aspect, which is a preferred example of the projection method of the present disclosure, is a projection method in a projection device equipped with a light modulation element that modulates light from a light source, and includes displaying an interface image for receiving input of information regarding projection conditions, a processing device determining the type of structured light to be projected from the projection device based on the projection conditions input using the interface image, and projecting the structured light from the projection device according to the result of determining the type.

[0129] In the above aspect, the user inputs the projection conditions, and an appropriate type of structured light is selected, thereby improving convenience for the user.

[0130] (Supplementary Note 3) In a third aspect, which is a preferred example of the first aspect, the processing device executes the following: determining whether a ratio of the second number of pixels to the first number of pixels satisfies a first condition; determining whether a gradation value of the difference image satisfies a second condition; determining that the type of structured light to be projected from the projection device is a Gray code pattern if the ratio satisfies the first condition but the gradation value does not satisfy the second condition; and determining that the type of structured light to be projected from the projection device is a phase shift pattern if the ratio does not satisfy the first condition but the gradation value satisfies the second condition. In the above aspect, it is possible to determine the type of structured light that is more accurate and more appropriate, between a Gray code pattern and a phase shift pattern, depending on the projection conditions.

[0131] (Supplementary Note 4) In a fourth aspect, which is a preferred example of the third aspect, the processing device receives from a user a setting of a mode for generating a correspondence between pixels of the light modulation element and pixels of the image sensor based on the structured light, and if the ratio satisfies the first condition and the gradation value satisfies the second condition, determines the type of structured light to be projected from the projection device in accordance with the setting. In the above aspect, if the required measurement accuracy can be ensured for both a Gray code pattern and a phase shift pattern, the type of structured light can be determined in accordance with the user's wishes.

[0132] (Supplementary Note 5) In a fifth aspect, which is a preferred example of the fourth aspect, the mode is either a first mode in which the number of the structured light beams is M (M is a natural number greater than or equal to 2) or a second mode in which the number of the structured light beams is N (N is a natural number greater than M), and the processing device determines that the type of the structured light to be projected from the projection device is a phase shift pattern when the setting is a first setting corresponding to the first mode, and determines that the type of the structured light to be projected from the projection device is a Gray code pattern when the setting is a second setting corresponding to the second mode. In the above aspect, if the required measurement accuracy can be ensured for both the Gray code pattern and the phase shift pattern, the type of structured light can be determined according to the user's preference.

[0133] (Supplementary Note 6) In the sixth aspect, which is a preferred example of the third aspect, if the ratio does not satisfy the first condition and the gradation value does not satisfy the second condition, the processing device causes the projection device to project a message indicating that the accuracy of generating a correspondence between the pixels of the light modulation element and the pixels of the image sensor based on the structured light will decrease. In the above aspect, if the required measurement accuracy cannot be ensured for both the Gray code pattern and the phase shift pattern, it is possible to notify the user to that effect.

[0134] (Supplementary Note 7) In a seventh aspect, which is a preferred example of the sixth aspect, the message includes potential solutions for improving the accuracy. In the above aspect, if the required measurement accuracy cannot be ensured for both the Gray code pattern and the phase shift pattern, the user can be notified of the solutions. [Explanation of symbols]

[0135] 10...projection device, 11...storage device, 12...processing device, 12a...projection control unit, 12b...imaging control unit, 12c...processing unit, 13...communication device, 14...image processing circuit, 15...optical device, 15a...light source, 15b...light modulation element, 15c...optical system, 16...operation device, 20...imaging device, 21...imaging element, 30...terminal device, 31...display device, 100...system, B1...button, B2...button, B3...button, B4...button, B5...button, CPX...camera information, D0...imaging data, D1...first imaging data, D2...second imaging data, D3...differential image information, DC...correspondence information, DG1...first image information, DG2...second image information, DGa...gray code information, DGb...phase shift information, DS...structured light, G...projected image, G-M1...image, G-M2...image, G-M3...interface image, G1...first reference image, G2...second reference image, GS...structured light, IMG...video data, PC...second part, PC1...second part , PP1...first part, PP1'...first part, PPX...panel information, PR1...program, RC...area, RP...area, RP1...projection area, Ra...message, Rb...message, Rc...message, S35...step, S1...step, S1A...step, S1B...step, S1C...step, S10...step, S20...step, S21...step, S22...step, S23...step, S24...step, S25...step, S3 0...step, S40...step, S50...step, S60...step, S70...step, S80...step, S90...step, S91...step, S92...step, S100...step, S110...step, S140...step, S150...step, S160...step, S170...step, S180...step, S190...step, S200...step, S210...step, S220...step, SC...projection target.

Claims

1. A projection method for a projection device including a light modulation element that modulates light from a light source, comprising: projecting a first reference image including a first portion having a first number of pixels onto a projection target; an imaging device having an imaging element captures the first reference image to obtain a first captured image; projecting a second reference image onto the projection target; The imaging device captures the second reference image to obtain a second captured image; a processing device determining a type of structured light to be projected from the projection device based on a difference image between the first captured image and the second captured image, the first number of pixels, and a second number of pixels which is the number of pixels of a second portion of the first captured image corresponding to the first portion; projecting the structured light from the projection device according to a result of determining the type of the structured light. A projection method characterized by:

2. A projection method for a projection device including a light modulation element that modulates light from a light source, comprising: displaying an interface image for receiving input of information regarding projection conditions; a processing device determining a type of structured light to be projected from the projection device based on the projection conditions input using the interface image; projecting the structured light from the projection device in accordance with a result of determining the type. A projection method characterized by:

3. The processing device includes: determining whether a ratio of the second number of pixels to the first number of pixels satisfies a first condition; determining whether the gradation value of the difference image satisfies a second condition; determining that the type of the structured light to be projected from the projection device is a Gray code pattern when the ratio satisfies the first condition and the gradation value does not satisfy the second condition; If the ratio does not satisfy the first condition and the gradation value satisfies the second condition, determining that the type of the structured light to be projected from the projection device is a phase shift pattern. The projection method according to claim 1 .

4. The processing device includes: receiving, from a user, a setting of a mode related to generation of a correspondence relationship between pixels of the light modulation element and pixels of the imaging element based on the structured light; determining the type of the structured light to be projected from the projection device in accordance with the setting when the ratio satisfies the first condition and the gradation value satisfies the second condition; The projection method according to claim 3 .

5. the mode is either a first mode in which the number of the structured light beams is M (M is a natural number equal to or greater than 2) or a second mode in which the number of the structured light beams is N (N is a natural number greater than M); The processing device includes: If the setting is a first setting corresponding to the first mode, determining that the type of the structured light to be projected from the projection device is a phase shift pattern; If the setting is a second setting corresponding to the second mode, it is determined that the type of the structured light to be projected from the projection device is a Gray code pattern. The projection method according to claim 4.

6. The processing device includes: causing the projection device to project a message indicating that accuracy of generating a correspondence between pixels of the light modulation element and pixels of the image sensor based on the structured light will decrease if the ratio does not satisfy the first condition and the gradation value does not satisfy the second condition. The projection method according to claim 3 .

7. the message includes a potential solution for improving the accuracy. The projection method according to claim 6.

Images