Display method and display system

Abstract

To avoid the occurrence of a problem caused by the generation of a plurality of pieces of detection light without deteriorating time resolution about detection of a pointer when a plurality of interactive projectors are arranged side by side.SOLUTION: A projector 10 includes a first acquisition section 110a, a detection section 110b, a second acquisition section 110c, and a display control section 110d. The first acquisition section 110a acquires a captured image of a range containing a display surface by a camera. The detection section 110b detects a first bright point and a second bright point corresponding to pieces of reflected light of first light and second light emitted along the display surface by a pointer from the captured image. When a distance between a position of the first bright point and a position of the second bright point is less than a threshold th1, the second acquisition section 110c acquires a third position determined on the basis of the first bright point and the second bright point. The display control section 110d displays a display image containing a drawn image by the pointer in the third position.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present disclosure relates to a display method and a display system.

Background Art

[0002] When a user performs an operation of drawing characters or figures using an indicator such as a finger on a display surface that is the display destination of an image, an interactive projector displays an image indicating the user's drawn line. As an example of an interactive projector, the position of the indicator is detected by imaging, with a camera, a bright spot generated when the indicator crosses light such as IR (InfraRed) light irradiated along the display surface in order to detect the indicator. The light irradiated along the display surface is called a light curtain, and the irradiation device that irradiates the light curtain is called a light curtain unit or a finger touch unit.

[0003] When the display surface is large and an image can be displayed only on a part of the display surface with one interactive projector, a display system may be constructed by arranging a plurality of interactive projectors side by side. However, when a plurality of interactive projectors that detect the position of an indicator using a light curtain are arranged side by side, a part of the light curtains of adjacent interactive projectors may overlap. When an indicator crosses two overlapping light curtains, two bright spots may be imaged for one indicator in each interactive projector. When two bright spots are imaged for one indicator, problems such as the user's drawn line being double-drawn or the drawn line being jagged may occur. Various techniques for avoiding the occurrence of such problems have been proposed, and as an example, the technique disclosed in Patent Document 1 can be cited. In the technique disclosed in Patent Document 1, by making the irradiation timings of the light curtains of adjacent interactive projectors different on the time axis, the imaging of two bright spots for one indicator is avoided.

Prior Art Documents

Patent Documents

[0004] [Patent Document 1] Japanese Patent Publication No. 2017-138871 [Overview of the initiative] [Problems that the invention aims to solve]

[0005] When using a first interactive projector and a second interactive projector side by side, if the illumination timing of the corresponding light curtains for each is different on the time axis, the other interactive projector cannot detect the indicator object while the light curtain corresponding to one interactive projector is illuminating, resulting in a deterioration of the time resolution for detecting the position of the indicator object. When the time resolution for detecting the indicator object deteriorates, it becomes impossible to continuously detect the position of the indicator object, and problems such as stuttering occur in the displayed lines occur. [Means for solving the problem]

[0006] One aspect of the display method of the present disclosure includes: acquiring an image captured by a camera of an area including a display surface; detecting a first bright spot corresponding to the reflected light of a first light irradiated along the display surface and a second bright spot corresponding to the reflected light of a second light irradiated along the display surface from the image; acquiring a third position in the image determined based on the first bright spot and the second bright spot when the distance between the first bright spot and the second bright spot in the image is less than a first threshold; and displaying a display image including a drawing image at the third position on the display surface.

[0007] Furthermore, one embodiment of the display system of the present disclosure includes a display device that displays an image on a display surface, and a camera that captures an image of an area including the display surface. The display device performs the following: acquires an image captured by the camera; detects a first bright spot corresponding to the reflected light of a first light irradiated along the display surface and a second bright spot corresponding to the reflected light of a second light irradiated along the display surface from the captured image; acquires a third position in the captured image determined based on the first bright spot and the second bright spot if the distance between the first bright spot and the second bright spot in the captured image is less than a first threshold; and displays a display image including a drawing image at the third position on the display surface. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example configuration of the display system 1 according to an embodiment of the present disclosure. [Figure 2] This diagram illustrates the IR light emitted from the irradiating device 20A and the irradiating device 20B, respectively, in order to detect the position of the indicator 30 in the display system 1. [Figure 3] This diagram illustrates the IR light emitted from the irradiating device 20A and the irradiating device 20B, respectively, in order to detect the position of the indicator 30 in the display system 1. [Figure 4] This diagram illustrates the problems that arise when the indicator 30 crosses both IR light LA ​​and IR light LB. [Figure 5] This diagram shows an example configuration of the projector 10 included in the display system 1. [Figure 6] This is a diagram illustrating the effects of this embodiment. [Figure 7] This is a flowchart showing the flow of the display method executed by the processing unit 110 according to the program PRA. [Figure 8] This flowchart shows the processing details of the second acquisition process SA130, which is included in the display method executed by the processing device 110. [Figure 9] This is a diagram to illustrate variation (2). [Modes for carrying out the invention]

[0009] The embodiments described below are subject to various technically preferred limitations. However, the embodiments of this disclosure are not limited to those described below. 1. Embodiment Figure 1 shows an example configuration of a display system 1 according to one embodiment of the present disclosure. The display system 1 includes projectors 10A and 10B, and illumination devices 20A and 20B. An image supply device (not shown) is connected to projectors 10A and 10B by wire or wireless. Projector 10A displays a display image PGA corresponding to image data shared from the image supply device (not shown) on a display surface SS such as a projection screen. Similarly, projector 10B displays a display image PGB corresponding to image data shared from the image supply device (not shown) on the display surface SS. Projectors 10A and 10B are installed on the ceiling of the room where the display surface SS is installed. Examples of image data supplied from the image supply device to each of projectors 10A and 10B include image data representing materials for meetings, etc., or image data representing teaching materials for schools, etc. A specific example of the image supply device is, for example, a personal computer.

[0010] When a user draws characters or figures on the display surface SS using the indicator 30, Projector 10A displays a display image PGA on the display surface SS, which includes a drawing image showing the user's lines. The display image PGA is an image in which a drawing image showing the user's lines is superimposed on an image represented by image data shared from an image supply device. In this embodiment, the indicator 30 is the user's finger, but it may also be a stylus. Similarly, when a user draws characters or figures on the display surface SS using the indicator 30, Projector 10B displays a display image PGB on the display surface SS, which includes a drawing image showing the user's lines. The display image PGB is also an image in which a drawing image showing the user's lines is superimposed on an image represented by image data shared from an image supply device. In other words, Projector 10A and Projector 10B are interactive projectors. Hereafter, when it is not necessary to distinguish between Projector 10A and Projector 10B, Projector 10A and Projector 10B will be referred to as Projector 10. In the following, as shown in Figure 1, the coordinate axis along the normal of the display surface SS will be referred to as the Z-axis, the coordinate axis along the vertical direction will be referred to as the Y-axis, and the coordinate axis perpendicular to the Y-axis and Z-axis will be referred to as the X-axis.

[0011] Irradiation device 20A is a finger touch unit paired with projector 10A. Irradiation device 20B is a finger touch unit paired with projector 10B. As shown in Figure 1, illumination devices 20A and 20B are arranged above the display surface SS so as to be aligned in the X-axis direction. Irradiation devices 20A and 20B are equipped with IR light sources that emit IR light. Each of illumination devices 20A and 20B irradiates the display surface SS with IR light that forms a light curtain. Figures 2 and 3 are diagrams illustrating the IR light emitted from each of illumination devices 20A and 20B. Figure 2 is a view of the IR light emitted from each of illumination devices 20A and 20B from the Z-axis direction. Figure 3 is a view of the IR light emitted from each of illumination devices 20A and 20B from the Y-axis direction. As shown in Figures 2 and 3, the irradiation device 20A irradiates IR light LA ​​in a fan shape along the display surface SS at a distance HA in the Z-axis direction from the display surface SS. IR light LA ​​is an example of the first light in this disclosure. Similarly, the irradiation device 20B irradiates IR light LB in a fan shape along the display surface SS at a distance HB in the Y-axis direction from the display surface SS. IR light LB is an example of the second light in this disclosure.

[0012] As shown in Figures 2 and 3, the light curtain formed by IR light LA ​​and the light curtain formed by IR light LB partially overlap when viewed from the Z-axis direction. It is preferable that there is no gap in the Z-axis direction between the light curtain formed by IR light LA ​​and the light curtain formed by IR light LB, that is, that distance HA and distance HB are equal. However, distances HA and HB may not coincide due to differences in the initial settings of the irradiation devices 20A and 20B. In addition, curvature is likely to occur at the ends of the fan-shaped light curtain, and distances HA and HB may not coincide due to this curvature. When distances HA and HB do not coincide, as shown in Figure 3, two bright spots, bright spot PA and bright spot PB, are generated for one indicator 30. In a situation where two bright spots are generated for one indicator 30, suppose the user performs an action of drawing a line segment on the display surface SS using the indicator 30. In this case, conventional interactive projectors suffer from a problem where a display image PGC, which includes a drawing image representing the line segment TA corresponding to the trajectory of the bright point PA and the line segment TB corresponding to the trajectory of the bright point PB, is displayed on the display surface SS, as shown in Figure 4. The projector 10 in this embodiment is configured to avoid the occurrence of the above problem without degrading the time resolution for detecting the position of the indicator 30. The projector 10 is an example of a display device in this disclosure.

[0013] Figure 5 shows an example configuration of projector 10. As shown in Figure 5, projector 10 includes a processing unit 110, a projection device 120, a camera 130, and a storage device 140. In addition to the processing unit 110, projection device 120, camera 130, and storage device 140, projector 10 also includes a communication circuit for communicating with an image supply device by wire or wireless, and an input device such as a keyboard for receiving various user input operations. However, the communication device and input device are not shown in Figure 5 because they are not very relevant to this disclosure.

[0014] The processing device 110 includes a processor such as a CPU (Central Processing Unit), that is, a computer. The processing device 110 may be composed of a single processor or a plurality of processors. The processing device 110 functions as a control center of the projector 10 by operating according to the program PRA stored in the storage device 140.

[0015] The projection device 120 includes a projection lens, a liquid crystal driving unit, a liquid crystal panel, and a light source unit. In FIG. 5, illustration of the projection lens, the liquid crystal driving unit, the liquid crystal panel, and the light source unit is omitted. The liquid crystal driving unit drives the liquid crystal panel according to the image data supplied from the processing device 110, thereby drawing the display image represented by this image data on the liquid crystal panel. The light source unit includes a light source such as a halogen lamp or a laser diode, for example. The light from the light source unit is modulated for each pixel in the liquid crystal panel and projected as image light through the projection lens. When this image light is projected onto the display surface SS, the display image is displayed on the display surface SS. Note that the projection device 120 may have any configuration as long as it can project image light onto the display surface SS, and may be configured to include a digital micromirror device (DMD) instead of the liquid crystal driving unit and the liquid crystal panel.

[0016] The camera 130 is, for example, an IR camera, and its posture has been adjusted so that the optical axis faces the center of the display surface SS. The camera 130 includes, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), which is an image pickup device that converts the light condensed by an optical system such as a lens into an electrical signal, and an IR light filter for receiving IR light is attached thereto. The camera 130 images an image of a predetermined range including the display surface SS at a predetermined period under the control of the processing device 110. Each time the camera 130 images an image of a predetermined range including the display surface SS, the camera 130 outputs the image data representing the captured image to the processing device 110.

[0017] The memory device 140 is a recording medium readable by the processing device 110. The memory device 140 includes, for example, a non-volatile memory and a volatile memory. The non-volatile memory is, for example, a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read Only Memory). The volatile memory is, for example, a RAM (Random Access Memory).

[0018] In the non-volatile memory of the memory device 140, a program PRA for causing the processing device 110 to execute the display method of the present disclosure is stored in advance. Also, in the non-volatile memory, conversion data for mutually converting the position on the captured image captured by the camera 130 and the position on the display image displayed by the projection device 120 is stored in advance. This conversion data is appropriately generated by performing well-known calibration and stored in the non-volatile memory. The volatile memory of the memory device 140 is used by the processing device 110 as a work area when executing the program PRA.

[0019] When the projector 10 is powered on, the processing device 110 reads the program PRA from the non-volatile memory to the volatile memory and starts executing the read program PRA. The processing device 110 operating according to the program PRA functions as a first acquisition unit 110a, a detection unit 110b, a second acquisition unit 110c, and a display control unit 110d shown in FIG. 5. The first acquisition unit 110a, the detection unit 110b, the second acquisition unit 110c, and the display control unit 110d shown in FIG. 5 are software modules realized by operating the processing device 110 according to the program PRA. The functions of each of the first acquisition unit 110a, the detection unit 110b, the second acquisition unit 110c, and the display control unit 110d shown in FIG. 5 are as follows.

[0020] The first acquisition unit 110a acquires image data output from the camera 130. As described above, in this embodiment, the camera 130 captures an image of a predetermined range including the display surface SS. In other words, the first acquisition unit 110a acquires image data representing the captured image of a predetermined range including the display surface SS.

[0021] The detection unit 110b analyzes the image data acquired by the first acquisition unit 110a to detect bright spots corresponding to the reflected light of the light curtain by the indicator 30 from the captured image data. For example, the detection unit 110b detects a region in the captured image that contains a predetermined number of pixels, such as 3 pixels or more, and in which the brightness of each pixel is greater than or equal to a predetermined value, as a region corresponding to the bright spots. In order to improve the stability of bright spot detection, the detection unit 110b may perform bright spot detection after removing the background image obtained by imaging the display surface SS when the indicator 30 is not close to the image.

[0022] As described above, in this embodiment, IR light LA ​​is irradiated from illumination device 20A along the display surface SS, and IR light LB is irradiated from illumination device 20B along the display surface SS. When a user brings the indicator 30 close to the display surface SS to perform some kind of writing, the indicator 30 crosses either one or both of the IR light LA ​​and IR light LB. When the indicator 30 crosses either the IR light LA ​​or IR light LB, a single bright spot corresponding to the reflected light generated when the indicator 30 crosses the IR light is captured in the image. Therefore, when the indicator 30 crosses either the IR light LA ​​or IR light LB, the detection unit 110b detects a single bright spot corresponding to the reflected light of the IR light LA ​​or IR light LB by the indicator 30.

[0023] When one indicator 30 crosses both IR light LA ​​and IR light LB, the captured image shows a bright spot PA corresponding to the reflected light of IR light LA ​​by the indicator 30, and a bright spot PB corresponding to the reflected light of IR light LB by the indicator 30. Therefore, when one indicator 30 crosses both IR light LA ​​and IR light LB, the detection unit 110b detects a bright spot PA corresponding to the reflected light of IR light LA ​​by the indicator 30, and a bright spot PB corresponding to the reflected light of IR light LB by the indicator 30. Bright spot PA is an example of a first bright spot in this disclosure. Bright spot PB is an example of a second bright spot in this disclosure.

[0024] When indicator 30 crosses only the IR light LA ​​and another indicator different from indicator 30 crosses only the IR light LB, or when indicator 30 crosses only the IR light LB and another indicator crosses only the IR light LA, two bright spots appear in the captured image. In these cases as well, the detection unit 110b detects two bright spots. Furthermore, when indicator 30 does not cross either the IR light LA ​​or the IR light LB, and the other indicator also does not cross either the IR light LA ​​or the IR light LB, no bright spots appear in the captured image. In this case, the detection unit 110b does not detect any bright spots.

[0025] The second acquisition unit 110c acquires the placement position of the drawing image, which represents the lines drawn using the indicator 30, on the display surface SS, based on the detection result by the detection unit 110b. The placement position is an example of a third position in this disclosure. If no bright spot is detected by the detection unit 110b, the second acquisition unit 110c does not acquire the placement position. This is because if no bright spot is detected by the detection unit 110b, it is considered that no writing has been performed using the indicator 30.

[0026] When the detection unit 110b detects one bright spot, the second acquisition unit 110c calculates the centroid position of the brightness of the region corresponding to the bright spot in the captured image, and obtains the placement position by converting this centroid position to a position on the display image using conversion data. The centroid position of brightness is a value obtained by weighting the X and Y coordinate values ​​of the pixels belonging to the region with the brightness value of the pixel as the weight.

[0027] When the detection unit 110b detects two bright spots, the second acquisition unit 110c first determines whether the distance between the two bright spots is less than the threshold th1. The threshold th1 is an example of a first threshold in this disclosure. If the distance between the two bright spots is less than the threshold th1, the second acquisition unit 110c considers that two bright spots were generated by one indicator 30 crossing the IR light LA ​​and the IR light LB, calculates the centroid position of the luminance of a predetermined region including the two bright spots in the captured image, and obtains the placement position by converting this centroid position to a position on the display image using conversion data. In other words, if the distance between the two bright spots is less than the threshold th1, one placement position is obtained for these two bright spots.

[0028] The reason for determining the placement of the two bright spots by considering their brightness in addition to their respective positions is as follows: When the indicator 30 reflects IR light LA, causing the bright spot PA to appear in the captured image, and the indicator 30 reflects IR light LB, causing the bright spot PB to appear in the captured image, the bright spot PA is often brighter than the bright spot PB in the region overlapping with the display image PGA, and it is thought that the brighter bright spot represents the more accurate position. Furthermore, a specific example of a predetermined region containing the two bright spots in the captured image is the region enclosed by the smallest circle containing the two bright spots.

[0029] If the distance between two bright spots is greater than or equal to the threshold th1, the second acquisition unit 110c assumes that indicator 30 crosses only the IR light LA ​​and the other indicator crosses only the IR light LB, or that indicator 30 crosses only the IR light LB and the other indicator crosses only the IR light LA. The second acquisition unit 110c calculates the centroid position of the brightness for each bright spot captured in the image and obtains the placement position by converting this centroid position to a position on the display image using conversion data. In other words, if the distance between two bright spots is greater than or equal to the threshold th1, a total of two placement positions are obtained, one for each of these two bright spots.

[0030] The display control unit 110d generates image data representing a display image in which a drawing image representing a line drawn by the indicator 30 is placed at the placement position acquired by the second acquisition unit 110c, and provides it to the projection device 120.

[0031] In this embodiment, as shown in Figure 3, when a single indicator 30 crosses the IR light LA ​​and IR light LB, and bright spots PA and PB are captured in the image, a display image PGD is displayed on the display surface SS, as shown in Figure 6, in which a drawing image representing the line TC is placed at a single position determined based on the bright spots PA and PB. In Figure 6, the line segment corresponding to the trajectory of bright spot PA is drawn as a dotted line, and the line segment corresponding to the trajectory of bright spot PB is drawn as a dashed line, but the line segments corresponding to the trajectories of bright spot PA and bright spot PB are not actually displayed. Therefore, even if two bright spots are captured for a single indicator 30, the problem of the user's drawn line being drawn twice does not occur. The applicant has experimentally confirmed that the above problem does not occur even when the difference between distance HB and distance HA is approximately 20 mm.

[0032] Furthermore, the processing unit 110, which operates according to the program PRA, displays the display method shown in Figure 7 at regular intervals. repetition Execute. As shown in Figure 7, the display method of this embodiment includes a first acquisition process SA110, a detection process SA120, a second acquisition process SA130, and a display control process SA140.

[0033] In the first acquisition process SA110, the processing unit 110 functions as the first acquisition unit 110a. In the first acquisition process SA110, the processing unit 110 acquires image data representing an image captured within a predetermined range including the display surface SS.

[0034] In detection process SA120, the processing unit 110 functions as a detection unit 110b. In detection process SA120, the processing unit 110 analyzes the image data acquired in the first acquisition process SA110 to detect bright spots corresponding to the reflected light of the light curtain by the indicator 30 from the captured image data.

[0035] In the second acquisition process SA130, the processing unit 110 functions as the second acquisition unit 110c. In the second acquisition process SA130, the processing unit 110 acquires the placement position based on the bright spot detected in the detection process SA120. Figure 8 is a flowchart showing the processing contents of the second acquisition process SA130. As shown in Figure 8, the second acquisition process SA130 includes the first determination process SA1310, the second determination process SA1320, the third determination process SA1330, the first position calculation process SA1340, and the second position calculation process SA1350.

[0036] In the first determination process SA1310, the processing unit 110 determines whether or not bright spots were detected in the detection process SA120. If no bright spots were detected in the detection process SA120, the result of the first determination process SA1310 is "No", i.e., negative. If the result of the first determination process SA1310 is negative, the processing unit 110 terminates the second acquisition process SA130 without executing the processes from the second determination process SA1320 onward. In this case, the placement position is not acquired. If one or two bright spots are detected in the detection process SA120, the result of the first determination process SA1310 is "Yes", i.e., positive. If the result of the first determination process SA1310 is positive, the processing unit 110 executes the second determination process SA1320.

[0037] In the second determination process SA1320, the processing unit 110 determines whether the number of bright spots detected in the detection process SA120 is 1. If the number of bright spots detected in the detection process SA120 is 1, the second determination process SA 1320 The result of the determination is "Yes", that is, affirmative. If the result of the second determination process SA1320 is affirmative, the processing unit 110 executes the first position calculation process SA1340 and terminates the second acquisition process SA130. If the number of bright spots detected in the detection process SA120 is not 1, the second determination process SA 1320 The result of the determination is "No," that is, negative. If the result of the second determination process SA1320 is negative, the processing unit 110 executes the third determination process SA1330.

[0038] In the third determination process SA1330, the processing unit 110 determines whether the distance between the two bright spots detected in the detection process SA120 is less than the threshold th1. As mentioned above, if two bright spots are generated by one indicator 30 crossing the IR light LA ​​and the IR light LB, the distance between these two bright spots will be less than the threshold th1, and the determination result of the third determination process SA1330 will be "Yes," that is, affirmative. On the other hand, if the first bright spot is generated by the indicator 30 crossing the IR light LA ​​and the second bright spot is generated by another indicator crossing the IR light LB, the distance between the first bright spot and the second bright spot will be greater than or equal to the threshold th1, and the determination result of the third determination process SA1330 will be "No," that is, negative. If the determination result of the third determination process SA1330 is affirmative, the processing unit 110 executes the second position calculation process SA1350. If the result of the third determination process SA1330 is negative, the processing unit 110 executes the first position calculation process SA1340.

[0039] In the first position calculation process SA1340, the processing unit 110 obtains the placement position of the drawn image by calculating the luminance centroid for each bright spot detected in the detection process SA120. Therefore, if there is one bright spot detected in the detection process SA120, one placement position is obtained in the first position calculation process SA1340. Conversely, if there are two bright spots detected in the detection process SA120, two placement positions are obtained in the first position calculation process SA1340.

[0040] In the second position calculation process SA1350, the processing unit 110 obtains one placement position based on the two bright spots detected in the detection process SA120. Specifically, the processing unit 110 calculates the centroid position of the brightness in a predetermined region containing the two bright spots in the captured image, and obtains one placement position by converting this centroid position to a position on the display image using conversion data.

[0041] In the display control process SA140, the processing unit 110 functions as a display control unit 110d. In the display control process SA140, if the placement position is acquired in the second acquisition process SA130, the processing unit 110 generates image data representing a display image including a drawing image at this placement position and provides it to the projection device 120. If the placement position is not acquired in the second acquisition process SA130, the processing unit 110 generates image data representing a display image without a drawing image and provides it to the projection device 120.

[0042] As explained above, with the projector 10 of this embodiment, even if two bright spots are captured in the image taken by the camera 130, if the distance between the two bright spots is less than the threshold th1, one placement position is determined based on the two bright spots, and an image representing the drawn line is displayed at this placement position. Therefore, even if two bright spots are captured for one indicator 30, problems such as the user's drawn line being drawn twice will not occur. In addition, in the display system 1 of this embodiment, it is not necessary to make the irradiation timing of the IR light LA ​​by the irradiation device 20A and the irradiation timing of the IR light LB by the irradiation device 20B different, and the time resolution for detecting the indicator 30 does not deteriorate as in the technology disclosed in Patent Document 1.

[0043] 2. Transformation The above embodiment can be modified as follows. (1) In the above embodiment, when two bright spots are captured in the image obtained by imaging the display surface SS with the camera 130, and the distance between the two bright spots is less than the threshold th1, the placement position is obtained based on the centroid position of the luminance in the region containing the two bright spots. However, the placement position only needs to be determined based on the two bright spots in the image, and the placement position may be determined based only on the positions of the two bright spots without using information about luminance. For example, the placement position may be obtained based on a position determined by weighting the positions of the bright spot PA and the bright spot PB using a weight determined independently of luminance. A specific example of this method of determining the placement position by weighting is an embodiment in which the position of the midpoint of a line segment, where the bright spot PA in the image is one endpoint and the bright spot PB in the same image is the other endpoint, is converted using conversion data to obtain the placement position.

[0044] (2) In the second position calculation process SA1350 performed in the projector 10A, the processing unit 110 of the projector 10A may obtain the placement position based on the position of the brighter bright point if the difference in brightness between the two bright points in the captured image is greater than or equal to a threshold th2. The threshold th2 is an example of a second threshold in this disclosure. As described above, when the bright point PA is captured in the captured image due to the reflection of IR light LA ​​by the indicator 30, and the bright point PB is captured in the captured image due to the reflection of IR light LB by the indicator 30, the bright point PA is often brighter than the bright point PB in the region that overlaps with the display image PGA. According to this embodiment, compared to the embodiment in which the placement position is obtained based on the centroid position of the brightness in the region including the bright points PA and PB, the calculation of a weighted average is not required, and therefore the placement position can be obtained in an easy manner.

[0045] Furthermore, in the captured image captured by the camera 130 of the projector 10A, the position of the bright spot PB corresponding to the reflected light of the IR light LB from the indicator 30 is generally on a straight line connecting the reference position P0 on the display surface SS corresponding to the position of the camera 130 and the position of the bright spot PA corresponding to the reflected light of the IR light LA ​​from the indicator 30, as shown in Figure 9. The position of the bright spot PA is an example of a first position in this disclosure. The position of the bright spot PB is an example of a second position in this disclosure. The reference position P0 is a position determined by projecting the camera 130 onto a plane including the display surface SS. Therefore, in the second position calculation process SA1350 executed in the projector 10A, the processing unit 110 of the projector 10A may obtain the placement position based on the position of the bright spot that is closer to the reference position P0 among the two bright spots captured in the image. This embodiment also makes it possible to obtain the placement position simply, as it eliminates the need for weighted averaging calculations compared to the embodiment that obtains the placement position based on the centroid position of the luminance in the region including the bright spots PA and PB.

[0046] (3) In the above embodiment, the camera 130 was included in the projector 10, but the camera 130 may not be included in the projector 10, and the camera 130 and the projector 10 may be separate components. Although the above embodiment was an example of applying the present disclosure to a projector, the scope of application of the present disclosure is not limited to projectors, and may also be a flat panel display. In short, the present disclosure can be applied to any display device that displays an image corresponding to the user's writing action on the display surface.

[0047] (4) In the above embodiment, the first acquisition unit 110a, the detection unit 110b, the second acquisition unit 110c, and the display control unit 110d were software modules. However, one or more of the first acquisition unit 110a, the detection unit 110b, the second acquisition unit 110c, and the display control unit 110d, or all of them, may be hardware modules such as ASICs (Application Specific Integrated Circuits). Even if one or more of the first acquisition unit 110a, the detection unit 110b, the second acquisition unit 110c, and the display control unit 110d, or all of them, are hardware modules, the same effects as in the above embodiment will be achieved.

[0048] (5) Program PRA may be manufactured as a standalone product and may be provided for a fee or free of charge. Specific ways of providing Program PRA include providing it by writing it to a computer-readable recording medium such as flash ROM, or providing it by downloading it via a telecommunications line such as the Internet. By operating a general computer in accordance with Program PRA provided in these ways, it becomes possible to have the computer execute the display method of the Disclosure.

[0049] 3. Summary of this disclosure This disclosure is not limited to the embodiments and modifications described above, and can be implemented in various forms without departing from its spirit. For example, this disclosure can also be implemented in the following forms. The technical features in the embodiments described above that correspond to the technical features in each of the forms described below can be replaced or combined as appropriate in order to solve some or all of the problems of this disclosure or to achieve some or all of the effects of this disclosure. Furthermore, if such technical features are not described as essential in this specification, they can be deleted as appropriate. A summary of this disclosure is provided below.

[0050] (Note 1) A display method according to one aspect of the present disclosure includes: acquiring an image captured by a camera of an area including a display surface; detecting a first bright spot corresponding to the reflected light of a first light irradiated along the display surface and a second bright spot corresponding to the reflected light of a second light irradiated along the display surface from the image; acquiring a third position determined based on the first bright spot and the second bright spot in the image when the distance between the first bright spot and the second bright spot in the image is less than a first threshold; and displaying a display image including a drawing image at the third position on the display surface.

[0051] According to the display method described in (Note 1), even if a first bright spot and a second bright spot are captured in the image due to a single indicator reflecting the first and second bright spots, a single drawing image is displayed at a third position determined based on the first and second bright spots, and the drawing image corresponding to the first bright spot and the drawing image corresponding to the second bright spot are not displayed separately. Therefore, problems such as lines drawn by the user using the indicator being drawn twice do not occur. In addition, in the display method of this embodiment, it is not necessary to have different irradiation timings for the first light and the second light, and the time resolution for detecting the position of the indicator does not deteriorate.

[0052] (Note 2) The display method described in (Appendix 1), wherein the third position corresponds to the centroid position of the luminance in the region including the first bright spot and the second bright spot in the captured image.

[0053] According to the display method described in (Appendix 2), the drawn image can be displayed at a position on the display surface corresponding to the centroid position of the luminance in the region including the first bright spot and the second bright spot.

[0054] (Note 3) The display method according to (Appendix 1), wherein obtaining the third position includes, if the first bright spot is brighter than the second bright spot, obtaining the third position based on the position of the first bright spot.

[0055] According to the display method described in (Appendix 3), the third position can be obtained more easily compared to the method of obtaining the third position based on the centroid position of the luminance in the region including the first bright spot and the second bright spot.

[0056] (Note 4) The display method according to (Note 1), wherein obtaining the third position includes obtaining the third position based on the position of the first bright spot when the position on the display surface corresponding to the camera is taken as the reference position, and the first position on the display surface corresponding to the first bright spot is closer to the reference position than the second position on the display surface corresponding to the second bright spot.

[0057] According to the display method described in (Appendix 4), the third position can be obtained more easily compared to the method of obtaining the third position based on the centroid position of the luminance in the region including the first bright spot and the second bright spot.

[0058] (Note 5) A display system according to one aspect of the present disclosure includes a display device that displays an image on a display surface, and a camera that captures an image of an area including the display surface, wherein the display device performs the following: acquires an image captured by the camera; detects a first bright spot corresponding to the reflected light of a first light irradiated along the display surface and a second bright spot corresponding to the reflected light of a second light irradiated along the display surface from the captured image; acquires a third position in the captured image determined based on the first bright spot and the second bright spot if the distance between the first bright spot and the second bright spot in the captured image is less than a first threshold; and displays a display image including a drawing image at the third position on the display surface.

[0059] According to the display system described in (Appendix 5), similar to the display method described in (Appendix 1), it is possible to avoid problems such as lines drawn by the user using the indicator being drawn twice without degrading the time resolution for detecting the position of the indicator. [Explanation of symbols]

[0060] 1...Display system, 10, 10A, 10B...Projector, 20A, 20B...Irradiation device, 30...Indicator, 110...Processing device, 110a...First acquisition unit, 110b...Detection unit, 110c...Second acquisition unit, 110d...Display control unit, 120...Projection device, 130...Camera, 140...Storage device, PRA...Program, SS...Display surface.

Claims

1. The camera captures an image of the area including the display surface, and To detect from the captured image a first bright spot corresponding to the reflected light of a first light irradiated along the display surface and a second bright spot corresponding to the reflected light of a second light irradiated along the display surface, If the distance between the first bright spot and the second bright spot in the captured image is less than the first threshold, Obtaining a third position determined based on the first bright spot and the second bright spot, Displaying a display image including a drawing image at the third position on the display surface, Includes, The third position corresponds to the centroid of the brightness in the region including the first bright spot and the second bright spot in the captured image. Display method.

2. Acquiring an image captured by a camera that includes the display surface, To detect from the captured image a first bright spot corresponding to the reflected light of a first light irradiated along the display surface and a second bright spot corresponding to the reflected light of a second light irradiated along the display surface, If the distance between the first bright spot and the second bright spot in the captured image is less than the first threshold, Obtaining a third position determined based on the first bright spot and the second bright spot, Displaying a display image including a drawing image at the third position on the display surface, Includes, Obtaining the third position means If the first bright spot is brighter than the second bright spot, the third position is obtained based on the position of the first bright spot, Display method.

3. Acquiring an image captured by a camera that includes the display surface, To detect from the captured image a first bright spot corresponding to the reflected light of a first light irradiated along the display surface and a second bright spot corresponding to the reflected light of a second light irradiated along the display surface, If the distance between the first bright spot and the second bright spot in the captured image is less than the first threshold, Obtaining a third position determined based on the first bright spot and the second bright spot, Displaying a display image including a drawing image at the third position on the display surface, Includes, Obtaining the third position means The method includes obtaining the third position based on the position of the first bright spot when the position on the display surface corresponding to the camera is taken as the reference position, and the first position on the display surface corresponding to the first bright spot is closer to the reference position than the second position on the display surface corresponding to the second bright spot. Display method.

4. A display device that displays an image on a display surface, A camera that captures an image of the area including the display surface, The aforementioned display device is Acquiring images captured by the aforementioned camera, To detect from the captured image a first bright spot corresponding to the reflected light of a first light irradiated along the display surface and a second bright spot corresponding to the reflected light of a second light irradiated along the display surface, If the distance between the first bright spot and the second bright spot in the captured image is less than the first threshold, To obtain a third position in the captured image, which is determined based on the first bright spot and the second bright spot, The following actions are performed: display a display image including a drawing image at the third position on the display surface: The third position corresponds to the centroid of the brightness in the region including the first bright spot and the second bright spot in the captured image. Display system.

5. A display device that displays an image on a display surface, A camera that captures an image of the area including the display surface, The aforementioned display device is Acquiring images captured by the aforementioned camera, To detect from the captured image a first bright spot corresponding to the reflected light of a first light irradiated along the display surface and a second bright spot corresponding to the reflected light of a second light irradiated along the display surface, If the distance between the first bright spot and the second bright spot in the captured image is less than the first threshold, To obtain a third position in the captured image, which is determined based on the first bright spot and the second bright spot, The following actions are performed: display a display image including a drawing image at the third position on the display surface: Obtaining the third position means If the first bright spot is brighter than the second bright spot, the third position is obtained based on the position of the first bright spot, Display system.

6. A display device that displays an image on a display surface, A camera that captures an image of the area including the display surface, The aforementioned display device is Acquiring images captured by the aforementioned camera, To detect from the captured image a first bright spot corresponding to the reflected light of a first light irradiated along the display surface and a second bright spot corresponding to the reflected light of a second light irradiated along the display surface, If the distance between the first bright spot and the second bright spot in the captured image is less than the first threshold, To obtain a third position in the captured image, which is determined based on the first bright spot and the second bright spot, The following actions are performed: display a display image including a drawing image at the third position on the display surface: Obtaining the third position means The method includes obtaining the third position based on the position of the first bright spot when the position on the display surface corresponding to the camera is taken as the reference position, and the first position on the display surface corresponding to the first bright spot is closer to the reference position than the second position on the display surface corresponding to the second bright spot. Display system.

Images