Display method and display device

Abstract

The drawing system in the past only displays a wavy line having a specified waveform when the tip of the pen functioning as the pointing device is tilted to the right. There is room for improvement from a viewpoint of usability of the user.SOLUTION: A display method includes acquiring a coordinate data set including a plurality of coordinates indicating a hand-drawn wave line drawn by a user, calculating a calculated coordinate data set including a plurality of calculated coordinates indicating a shaped wave line based on the coordinate data set, and displaying the shaped wave line based on the calculated coordinate data set.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to a display method and a display device. [Background technology]

[0002] A display method for displaying a line drawn by a pointing device or the like is known. In a drawing system described in Patent Document 1, signals related to coordinates indicating the input position of the line to be drawn are gradually recorded and saved. A drawing unit of the drawing system forms a primary coordinate group consisting of a plurality of coordinates. A correction processing unit of the drawing system corrects the signals related to the primary coordinate group and forms a signal related to a secondary coordinate group. The secondary coordinate group forms a straight line. The drawing system displays the straight line as a wavy line depending on the state when the input position is specified. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-185694 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional drawing systems simply display a wavy line with a specified waveform when the tip of a pen serving as a pointing device is tilted to the right, and there is room for improvement in terms of user convenience. [Means for solving the problem]

[0005] The display method of the present disclosure includes acquiring a group of coordinate data including a plurality of coordinates indicating a handwritten wavy line drawn by a user, calculating a group of calculated coordinate data including a plurality of calculated coordinates indicating a shaped wavy line based on the group of coordinate data, and displaying the shaped wavy line based on the group of calculated coordinate data.

[0006] The display device of the present disclosure comprises a display unit and a control unit, and the control unit acquires a group of coordinate data including a plurality of coordinates indicating a handwritten wavy line drawn by a user, calculates a group of calculated coordinate data including a plurality of calculated coordinates indicating a shaped wavy line based on the group of coordinate data, and displays the shaped wavy line on the display unit based on the group of calculated coordinate data. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a display system. [Figure 2] FIG. 1 is a diagram showing a schematic configuration of a display system. [Figure 3] FIG. 1 is a block diagram showing the configuration of a display system. [Figure 4] FIG. 4 is a diagram showing an example of a control flow executed by the projector. [Figure 5] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 6] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 7] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 8] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 9] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 10] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 11] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 12] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 13] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 14] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 15] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 16] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 17]FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 18] FIG. 4 is a diagram showing an example of a set waveform. [Figure 19] FIG. 4 is a diagram showing an example of a set waveform. [Figure 20] FIG. 4 is a diagram showing an example of a set waveform. [Figure 21] FIG. 4 is a diagram showing an example of a control flow executed by the projector. [Figure 22] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 23] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 24] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 25] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 26] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. [Figure 27] FIG. 4 is a diagram showing an example of a projected image displayed on a display surface. DETAILED DESCRIPTION OF THE INVENTION

[0008] 1 shows a schematic configuration of a display system 100. FIG. 1 is a perspective view showing an installation state of each unit constituting the display system 100. The display system 100 includes a projector 1, a display screen 2, and an operation pen 5.

[0009] Several figures, including FIG. 1, show an XYZ coordinate system. The Z axis is an axis perpendicular to the display screen 2. The +Z direction is the direction toward the front of the display screen 2. The -Z direction is the direction toward the back of the display screen 2. The X axis is parallel to the display screen 2. The X axis is an axis perpendicular to the Y axis. The +X direction is the direction from left to right of the display screen 2. The -X direction is the direction from right to left of the display screen 2. The Y axis is parallel to the display screen 2. The Y axis is an axis perpendicular to the X and Z axes. The +Y direction is the direction from bottom to top of the display screen 2. The -Y direction is the direction from top to bottom of the display screen 2. The plane formed by the X and Y axes is called the XY plane.

[0010] The projector 1 generates image light corresponding to image data. The projector 1 emits the generated image light onto a display surface 2S of a display screen 2. The projector 1 displays a projection image PG on the display surface 2S by emitting the image light onto the display surface 2S. The projector 1 corresponds to an example of a display device.

[0011] The projector 1 has an interactive function. The interactive function includes a function for writing a drawn image onto the projection image PG, a function for performing gesture operations on the drawn image, etc. The projector 1 detects an indication position designated by the operation pen 5 or a finger, the movement trajectory of the operation pen 5, the operation of the operation pen 5, etc. The projector 1 displays a handwritten image 110 corresponding to the movement trajectory of the operation pen 5, etc. on the display surface 2S. The handwritten image 110 is an example of a drawn image. The projector 1 changes the position, shape, etc. of the drawn image included in the projection image PG based on the operation of the operation pen 5.

[0012] The display screen 2 has a display surface 2S. The display screen 2 shown in FIG. 1 is placed at a position in the -Z direction and the -Y direction of the projector 1. The display screen 2 shown in FIG. 1 is made of a plate-like member, but is not limited to this. The display screen 2 may be made of cloth or the like. The display screen 2 may also be a wall or floor of a building.

[0013] The display surface 2S displays a projected image PG including a drawn image. The display surface 2S shown in Fig. 1 is flat, but is not limited to this. The display surface 2S may also be a curved surface or a surface with irregularities.

[0014] FIG. 1 shows an operation pen 5 operated by a user. The operation pen 5 indicates a position on the display surface 2S or a spatial area in the +Z direction of the display surface 2S. Although FIG. 1 shows an operation pen 5, this is not limiting. A finger or the like may be used instead of the operation pen 5. As long as the configuration allows for designation of a position on the display surface 2S or a spatial area in the +Z direction of the display surface 2S, the form of the operation pen is not limited.

[0015] Fig. 2 shows a schematic configuration of the display system 100. Fig. 2 shows a plan view of the display system 100 from the +X direction. Fig. 2 shows a projector 1 that emits detection light LT. The projector 1 has a signal light transmission unit 13.

[0016] The signal light transmitter 13 emits the detection light LT toward the display surface 2S. The signal light transmitter 13 emits the detection light LT to an area including a projection area where the projector 1 projects the image light onto the display surface 2S. The detection light LT is light used to detect the operation pen 5 and the like. The signal light transmitter 13 emits infrared light as the detection light LT.

[0017] When the operation pen 5 is positioned within a detection area LA of the detection light LT, it blocks the detection light LT. The detection light LT hits the operation pen 5 and is reflected. A portion of the reflected light travels from the operation pen 5 toward the projector 1. The projector 1 receives the light reflected by the operation pen 5 positioned within the detection area LA. The detection area LA is the area between the display surface 2S and a virtual surface VS. The virtual surface VS is a surface that is parallel to the display surface 2S and is separated by a detection distance dt in the +Z direction. The detection distance dt is set in advance depending on the installation position of the projector 1 or the image capture unit 15, which will be described later. The projector 1 detects the position indicated by the operation pen 5, the movement trajectory of the operation pen 5, and the operation of the operation pen 5 based on the reflected light.

[0018] 2 shows an operation pen 5, but a finger may be used instead of the operation pen 5. The signal light transmission unit 13 shown in FIG. 2 is disposed in the projector 1, but is not limited to this. As an example, the signal light transmission unit 13 may be disposed at a position along the display surface 2S.

[0019] Fig. 3 shows a block configuration of the display system 100. The display system 100 includes a projector 1, a display screen 2, an operation pen 5, and a remote control 17. Although the display system 100 shown in Fig. 3 includes the remote control 17, it does not necessarily have to include the remote control 17.

[0020] The operation pen 5 indicates a position specified by the user, a movement trajectory based on the user's operation, and an action by the user. The position of the pen tip of the operation pen 5 indicates a position specified by the user. The displacement of the pen tip of the operation pen 5 indicates a movement trajectory based on the user's operation and an action by the user. The pen tip of the operation pen 5 may emit pattern light of a predetermined light emission pattern. As an example, the pattern light indicates that the pen tip is in contact with the display surface 2S, or that the pen tip is separated from the display surface 2S. The pattern light may also indicate that the pen tip is separated from the display surface 2S by a distance equal to or less than a predetermined distance.

[0021] The remote control 17 accepts user operations performed by the user. The remote control 17 generates various operation signals based on operation inputs by the user. The remote control 17 transmits the operation signals to the projector 1. The remote control 17 has, as an example, a plurality of operation buttons. The remote control 17 may also have a touch panel with a touch input function.

[0022] The projector 1 can operate in a whiteboard mode. The whiteboard mode is a mode in which drawing operations using an operation pen 5 are accepted. The whiteboard mode is an example of an interactive function. The projector 1 includes a remote control light receiving unit 11, a signal light transmitting unit 13, an imaging unit 15, a wireless communication unit 20, an image projection device 30, an image generation device 40, and a control device 50.

[0023] The remote control light receiving unit 11 receives an operation signal transmitted from the remote control 17. The remote control light receiving unit 11 accepts a user operation by receiving the operation signal transmitted from the remote control 17. The operation signal is a signal corresponding to a user operation on the remote control 17. The received operation signal may be an infrared signal, a Bluetooth signal, or the like. Bluetooth is a registered trademark. The remote control light receiving unit 11 may generate a signal corresponding to the received operation signal. The remote control light receiving unit 11 transmits the operation signal to the control device 50.

[0024] The signal light transmitting unit 13 emits detection light LT. When the projector 1 detects the position of the operation pen 5, the signal light transmitting unit 13 may transmit signal light that synchronizes the light emission timing of the operation pen 5 with the capture timing of the capture unit 15. The signal light is near-infrared light having a predetermined light emission pattern. As an example, the pen tip of the operation pen 5 emits pattern light of a predetermined light emission pattern in synchronization with the timing of receiving the signal light. The light emission pattern of the pattern light differs depending on whether the pen tip of the operation pen 5 is in contact with the display surface 2S or not.

[0025] The photographing unit 15 photographs the operation pen 5 etc. The photographing unit 15 may receive pattern light emitted by the pen tip of the operation pen 5. The photographing unit 15 has an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device). The photographing unit 15 photographs an area including the display surface 2S at regular time intervals based on the control of the control device 50, and generates photographed data. When the user operates the operation pen 5 etc., the photographing unit 15 generates one or more pieces of photographed data as information related to the operation. The photographing unit 15 transmits the photographed data to the control device 50.

[0026] The image capturing unit 15 is used to detect the distance between the operation pen 5 and the display surface 2S in the +Z direction. The projector 1 detects the distance between the operation pen 5 and the display surface 2S in the +Z direction, for example, by triangulation using a known stereo camera. Specifically, the projector 1 includes a first camera and a second camera (not shown) arranged at different positions as the image capturing unit 15. The projector 1 calculates a first distance from the first camera and the second camera to the operation pen 5 using a first captured image captured by the first camera, a second captured image captured by the second camera, the distance between the first camera and the second camera, and internal parameters of the first camera and the second camera. The first captured image and the second captured image are images of infrared light of the operation pen 5 that reflects the detection light LT. The second distance from the first camera and the second camera to the display surface 2S is measured in advance by a user or the like and stored. The projector 1 calculates the difference between the first distance and the second distance as the distance between the operation pen 5 and the display surface 2S. The first camera and the second camera are, for example, infrared cameras. The first camera and the second camera may be visible light cameras. One of the first camera and the second camera may be a visible light camera, and the other of the first camera and the second camera may be an infrared camera. When at least one of the first camera and the second camera is a visible light camera, the projector 1 may be equipped with a visible light imaging light source, or illumination light arranged in the installation space of the projector 1 may be used as the imaging light source. The projector 1 may detect the distance between the operation pen 5 and the display surface 2S by triangulation using the stereo camera described above.

[0027] The wireless communication unit 20 is an interface circuit that wirelessly connects to an external device for communication. The wireless communication unit 20 connects to the external device in accordance with a predetermined communication protocol. The wireless communication unit 20 performs wireless communication such as wireless LAN (Local Area Network) including Wi-Fi, Bluetooth, and NFC (Near Field Communication). Wi-Fi is a registered trademark. The wireless communication unit 20 receives a communication signal. The wireless communication unit 20 demodulates packet data from the communication signal. The wireless communication unit 20 extracts image data and various signals from the packet data. The wireless communication unit 20 transmits the image data to the image processing unit 41.

[0028] The image projection device 30 projects image light onto the display surface 2S. The image projection device 30 displays a projection image PG on the display surface 2S by projecting the image light. The image projection device 30 corresponds to an example of a display unit. The image projection device 30 has a light source 31, a light modulation device 33, and an optical unit 35.

[0029] The light source 31 emits light to the light modulation device 33. The light source 31 is configured as a lamp light source such as a halogen lamp, a xenon lamp, or an ultra-high pressure mercury lamp. The light source 31 may also be configured as a solid-state light source such as an LED (Light Emitting Diode) or a laser light source.

[0030] The light modulation device 33 has a light modulation element. The light modulation element modulates the light emitted from the light source 31. The light modulation element is composed of a transmissive liquid crystal light valve, a reflective liquid crystal light valve, a digital mirror device, etc. The light modulation device 33 emits the image light modulated by the light modulation element to the optical unit 35.

[0031] The optical unit 35 has optical elements such as lenses and mirrors. The optical unit 35 emits the image light modulated by the light modulation device 33 toward the display surface 2S. The optical unit 35 emits the image light toward the display surface 2S, thereby projecting a projection image PG onto the display surface 2S.

[0032] The image generation device 40 generates various images to be projected onto the display surface 2S by the image projection device 30. The image generation device 40 has an image processing unit 41, an OSD synthesis unit 43, and a frame memory 45. Although the projector 1 shown in FIG. 3 shows the image generation device 40 and the control device 50 as separate entities, they may also be configured as an integrated unit.

[0033] The image processing unit 41 expands image data received from the wireless communication unit 20, drawing data received from the control device 50, etc. into the frame memory 45. The image processing unit 41 performs image processing on the expanded image data, etc. The image processing includes at least one of resolution conversion processing, resizing processing, correction processing for distortion aberration, shape, etc., digital zoom processing, adjustment processing for image color, brightness, etc. The image processing unit 41 is an arithmetic processing device configured from a CPU (Central Processing Unit), GPU (Graphics Processing Unit), microcomputer, etc.

[0034] The OSD synthesis unit 43 synthesizes OSD data with image data and the like expanded in the frame memory 45. OSD stands for On Screen Display. The OSD data includes data for displaying various icons, display screens, and the like within the projection image PG. The OSD synthesis unit 43 is a processing unit configured with a CPU, a GPU, a microcomputer, and the like. The OSD synthesis unit 43 may be configured with the same processing unit as the image processing unit 41.

[0035] The frame memory 45 expands image data, OSD data, etc. The frame memory 45 is connected to the image processing unit 41 and the OSD synthesis unit 43. The frame memory 45 expands image data based on the control of the image processing unit 41 and the OSD synthesis unit 43. The frame memory 45 is configured with a semiconductor memory such as a RAM (Random Access Memory).

[0036] The control device 50 is a controller that controls each part of the projector 1. The control device 50 controls the projection image PG that is projected onto the display surface 2S by the image projection device 30. The control device 50 controls the image generation device 40 and controls various images that are displayed within the projection image PG. The control device 50 detects the operation pen 5, fingers, etc., and performs various controls based on the detection results. The control device 50 has a memory unit 60 and one or more control processors 70.

[0037] The storage unit 60 stores various data such as OSD data, various programs, etc. The storage unit 60 stores the captured data transmitted from the image capture unit 15. The storage unit 60 may store image data received by the wireless communication unit 20. The storage unit 60 may store various data as a work area for the control processor 70. The storage unit 60 is composed of a volatile semiconductor memory such as RAM, and a non-volatile memory such as a ROM (Read Only Memory) or a flash memory. The storage unit 60 may also have a magnetic memory such as an HDD (Hard Disk Drive). The storage unit 60 stores the control program CP.

[0038] The control program CP is firmware that causes the control processor 70 to operate as various functional units. The control program CP is executed by the control processor 70 to cause the control processor 70 to operate as various functional units.

[0039] One or more control processors 70 are configured by a CPU, a microcomputer, etc. The control processor 70 operates as various functional units by executing a control program CP. The control processor 70 functions as a communication control unit 71, a drawing control unit 73, and a display control unit 75. The control processor 70 may also function as a functional unit other than the communication control unit 71, the drawing control unit 73, and the display control unit 75. The control processor 70 corresponds to an example of a control unit.

[0040] The communication control unit 71 controls communication with an external device. The communication control unit 71 controls the wireless communication unit 20 to receive various data from the external device. The communication control unit 71 transmits various data to the external device via the wireless communication unit 20.

[0041] The drawing control unit 73 receives the captured image data. The drawing control unit 73 performs various drawing processes based on the captured image data. The drawing processes include generating, correcting, editing, and erasing drawn images including the handwritten image 110. The drawing control unit 73 controls the display of various icons and various display images within the projection image PG.

[0042] The drawing control unit 73 determines whether or not the operation pen 5 is in contact with the display surface 2S based on the imaging data. If the drawing control unit 73 determines that the operation pen 5 is in contact with the display surface 2S, it acquires the position of the operation pen 5 as pointed coordinate data PD based on the imaging data. The drawing control unit 73 determines a predetermined position within the operation pen 5 included in the imaging data. One example of the predetermined position is the position of the pen tip of the operation pen 5. The drawing control unit 73 acquires the predetermined position within the operation pen 5 as pointed coordinate data PD. The pointed coordinate data PD corresponds to an example of coordinates. In addition, when the drawing control unit 73 determines that the operation pen 5 is in contact with the display surface 2S, it is not limited to acquiring the position of the operation pen 5 as the indication coordinate data PD based on the shooting data, but may also acquire it as the indication coordinate data PD when it determines that the user is performing a drawing operation. The drawing control unit 73 only needs to acquire the designated coordinate data PD when the user is performing a drawing operation, and therefore does not need to determine whether the operation pen 5 is in contact with the display surface 2S.

[0043] The drawing control unit 73 acquires pointed coordinate data PD at predetermined time intervals until it determines that the operation pen 5 is not in contact with the display surface 2S. The drawing control unit 73 stores the multiple pointed coordinate data PD acquired at predetermined time intervals as a pointed coordinate data group PDs in the storage unit 60. The pointed coordinate data group PDs is a data group representing a handwritten image 110 handwritten by the user using the operation pen 5. The pointed coordinate data group PDs corresponds to an example of a coordinate data group.

[0044] The drawing control unit 73 determines the graphic shape of the handwritten image 110 based on the entirety of the designated coordinate data group PDs or a part of the designated coordinate data group PDs. The drawing control unit 73 determines whether the graphic shape is a predetermined shape. Predetermined shapes include a circle, an ellipse, a rectangle, a triangle, a trapezoid, a straight line, a wavy line, and a curved line. If the drawing control unit 73 determines that the graphic shape is a predetermined shape, it generates a corrected coordinate data group RDs for displaying a corrected drawn image 120 obtained by correcting the handwritten image 110. The corrected coordinate data group RDs corresponds to an example of a calculated coordinate data group. The corrected coordinate data group RDs includes a plurality of corrected coordinate data RD. The corrected coordinate data RD corresponds to an example of calculated coordinates.

[0045] The drawing control unit 73 outputs the pointed coordinate data PD and the corrected coordinate data RD to the display control unit 75. After generating the pointed coordinate data group PDs and the corrected coordinate data group RDs, the drawing control unit 73 may output the pointed coordinate data group PDs and the corrected coordinate data group RDs to the display control unit 75.

[0046] The display control unit 75 outputs at least one of the pointed coordinate data PD and the corrected coordinate data RD to the image generation device 40 at predetermined time intervals. The display control unit 75 may output at least one of the pointed coordinate data group PDs and the corrected coordinate data group RDs to the image generation device 40. The display control unit 75 outputs the pointed coordinate data PD or the pointed coordinate data group PDs to the image generation device 40, thereby causing the image projection device 30 to project a handwritten image 110. The image projection device 30 displays the handwritten image 110 on the display surface 2S. The display control unit 75 outputs the corrected coordinate data RD or the corrected coordinate data group RDs to the image generation device 40, thereby causing the image projection device 30 to project a corrected drawn image 120. The image projection device 30 displays the corrected drawn image 120 on the display surface 2S.

[0047] FIG. 4 shows an example of a control flow executed by the projector 1. FIG. 4 shows the control flow in the form of a flowchart. The control flow corresponds to an example of a display method. FIG. 4 shows the control flow when a user uses the operation pen 5 to draw a handwritten wavy line on the display surface 2S. FIG. 4 shows the control flow when the handwritten wavy line is corrected into a shaped wavy line and the shaped wavy line is displayed on the display surface 2S.

[0048] In step S101, the projector 1 generates captured data. When a user is drawing on the display surface 2S with the operation pen 5, the image capturing unit 15 captures an image of the operation pen 5 on the display surface 2S. The image capturing unit 15 captures an image of the operation pen 5 at predetermined time intervals and generates multiple pieces of captured data. The image capturing unit 15 outputs the captured data to the control processor 70.

[0049] After generating the captured image data, the projector 1 acquires pointed coordinate data PD in step S103. The image capture data is input from the image capture unit 15 to the drawing control unit 73, which operates in the control processor 70. The drawing control unit 73 acquires pointed coordinate data PD based on the captured image data. The drawing control unit 73 acquires pointed coordinate data PD based on each piece of captured image data captured at a predetermined time interval. The drawing control unit 73 stores the multiple pointed coordinate data PD in the memory unit 60 as a pointed coordinate data group PDs.

[0050] After acquiring the pointed coordinate data PD, in step S105, the projector 1 determines the graphic shape of the handwritten image 110. The drawing control unit 73 determines the graphic shape of the handwritten image 110 from the multiple pointed coordinate data PD included in the pointed coordinate data group PDs or from part of the multiple pointed coordinate data PD included in the pointed coordinate data group PDs.

[0051] After determining the graphic shape of the handwritten image 110, the projector 1 determines in step S107 whether the graphic shape is a wavy line. If the drawing control unit 73 determines that the graphic shape is a wavy line, the projector 1 proceeds to step S109 (step S107: YES). If the drawing control unit 73 determines that the graphic shape is not a wavy line, the projector 1 proceeds to step S115 (step S107: NO).

[0052] After determining that the graphic shape is a wavy line, the projector 1 calculates a corrected coordinate data set RDs in step S109. The drawing control unit 73 acquires the designated coordinate data set PDs or a portion of the designated coordinate data PD included in the designated coordinate data set PDs. Based on the designated coordinate data set PDs, the drawing control unit 73 calculates wavy line dimensions, including the amplitude, wavelength, and width, of the handwritten wavy line that constitutes the handwritten image 110. The amplitude and wavelength are calculated using a predetermined algorithm. For example, the amplitude and wavelength are determined by calculating the average value or median value of the amplitudes and wavelengths of multiple wavy lines included in the handwritten image 110. The width is calculated from the start and end positions of the handwritten image 110. Known calculation methods are used to calculate the amplitude, wavelength, and width of the handwritten wavy line. The drawing control unit 73 determines the waveform of the shaped wavy line that constitutes the corrected drawn image 120 by calculating the wavy line dimensions, including the amplitude, wavelength, and width, of the handwritten wavy line.

[0053] The drawing control unit 73 calculates corrected coordinate data RD using the designated coordinate data PD and the calculated wavy line dimensions of the handwritten wavy line. The drawing control unit 73 calculates corrected coordinate data RD corresponding to each designated coordinate data PD to calculate a corrected coordinate data group RDs including multiple corrected coordinate data RD. The corrected coordinate data group RDs represents a shaped wavy line. The drawing control unit 73 outputs the corrected coordinate data group RDs to the display control unit 75.

[0054] After calculating the corrected coordinate data group RDs, the projector 1 displays the corrected drawn image 120 in step S111. The display control unit 75 causes the image projection device 30 to display the corrected drawn image 120 based on the corrected coordinate data group RDs. The corrected drawn image 120 is a shaped wavy line obtained by shaping a handwritten wavy line in the drawing control unit 73. The shaped wavy line is a wavy line having a constant amplitude and a constant wavelength. The display control unit 75 outputs drawing data including the corrected coordinate data group RDs to the image generation device 40. The image generation device 40 performs various image processing on the drawing data. After performing the various image processing, the image generation device 40 outputs the drawing data including the corrected coordinate data group RDs to the image projection device 30. The image generation device 40 performs processing to combine the corrected drawn image 120 with an OSD image, and then outputs the drawing data including the corrected coordinate data group RDs to the image projection device 30. The image projection device 30 projects the corrected drawn image 120 based on the corrected coordinate data group RDs onto the display surface 2S. The image projection device 30 projects the corrected drawn image 120 onto the display surface 2S, thereby displaying the corrected drawn image 120 on the display surface 2S. The display surface 2S displays a projection image PG including the shaped wavy line.

[0055] In step S115, the projector 1 executes a correction process according to the graphic shape. After determining in step S107 that the graphic shape is not a wavy line, the rendering control unit 73 further determines the graphic shape. The determined graphic shape is set in advance. The determined graphic shapes include a circle, a square, a triangle, a straight line, and the like. The determined graphic shape does not have to be set other than a wavy line. If the rendering control unit 73 determines that the graphic shape is a predetermined graphic shape, it executes a correction process according to the graphic shape and determines a correction graphic. The correction process includes a shaping process, a deformation process, a replacement process, and the like. The shaping process is a process of shaping a graphic shape to create a correction graphic. The deformation process is a process of transforming a graphic shape into a predetermined shape to create a correction graphic. The replacement process is a process of replacing a graphic shape with another shape to create a correction graphic. The rendering control unit 73 calculates a correction coordinate data group RDs corresponding to the correction graphic and outputs it to the display control unit 75.

[0056] After performing correction processing according to the shape of the figure, the projector 1 displays a corrected drawn image 120 of the corrected figure in step S117. The display control unit 75 acquires drawing data including the corrected coordinate data group RDs. The display control unit 75 displays the corrected figure on the display surface 2S based on the corrected coordinate data group RDs. The display control unit 75 outputs the drawing data including the corrected coordinate data group RDs to the image generation device 40. The image generation device 40 performs various image processes on the drawing data. After performing the various image processes, the image generation device 40 outputs the drawing data including the corrected coordinate data group RDs to the image projection device 30. The image generation device 40 performs processing to combine the corrected figure with an OSD image, and then outputs the drawing data including the corrected coordinate data group RDs to the image projection device 30. The image projection device 30 projects the corrected figure based on the corrected coordinate data group RDs onto the display surface 2S as a corrected drawn image 120. The image projection device 30 projects the correction figure onto the display surface 2S, thereby displaying the correction figure on the display surface 2S. The display surface 2S displays the correction figure as a corrected drawn image 120.

[0057] FIG. 5 shows an example of a projected image PG displayed on the display surface 2S. FIG. 5 shows the projected image PG when a user draws a handwritten image 110. FIG. 5 shows a first handwritten image 111, which is an example of the handwritten image 110. The first handwritten image 111 is an example of a handwritten wavy line. The first handwritten image 111 is drawn by the user in a drawing area DA. The drawing area DA is an area in which the handwritten image 110 can be drawn by the user's operation. The drawing area DA is configured with a plain background, for example.

[0058] The drawing control unit 73 acquires the designated coordinate data PD included in the captured data, and displays the handwritten image 110 in the drawing area DA based on the designated coordinate data group PDs including the plurality of designated coordinate data PD.

[0059] The projection image PG shown in FIG. 5 includes an icon display area AD, an operation area OA, and a first handwritten image 111. A color specification icon group CA is displayed in the operation area OA. The icon display area AD and the operation area OA are examples of an OSD image. The projection image PG shown in FIG. 5 includes, but is not limited to, the first handwritten image 111, the icon display area AD, and the operation area OA. The projection image PG may or may not include the icon display area AD and the operation area OA. The projection image PG may include an OSD image different from the icon display area AD and the operation area OA.

[0060] The icon display area AD is an area for displaying one or more icons. The types and number of icons displayed in the icon display area AD are set as appropriate. The position of the icon display area AD is set in advance. The icon display area AD shown in FIG. 5 is positioned in the -X direction and the -Y direction of the display surface 2S, but is not limited to this. The position of the icon display area AD is set as appropriate.

[0061] The operation area OA is an area in which a plurality of icons are fixedly displayed. The operation area OA shown in Fig. 5 displays a plurality of icons including a color-designation icon group CA. The operation area OA shown in Fig. 5 is positioned in the -Y direction of the display surface 2S, but is not limited to this. The operation area OA may also be positioned in the +Y direction of the display surface 2S, for example.

[0062] The color designation icon group CA is input when the user designates the color of the handwritten image 110 to be drawn in the drawing area DA. The color designation icon group CA includes multiple icons of different colors. When the user performs a designation operation to designate one icon from multiple icons included in the multiple color designation icon groups CA, the projector 1 accepts the color of the icon for which the designation operation was performed as the designated color.

[0063] FIG. 6 shows an example of a projection image PG displayed on the display surface 2S. FIG. 6 shows the projection image PG when a user draws a handwritten image 110. FIG. 6 shows a first handwritten image 111, which is an example of the handwritten image 110. The first handwritten image 111 is an example of a handwritten wavy line. FIG. 6 shows a plurality of pieces of designated coordinate data PD corresponding to the first handwritten image 111. FIG. 6 schematically shows a designated coordinate data group PDs including a plurality of pieces of designated coordinate data PD. A mark indicating the designated coordinate data PD is not displayed in the projection image PG.

[0064] The plurality of designated coordinate data PD are acquired by the drawing control unit 73 based on the captured image data. The plurality of designated coordinate data PD indicate coordinate points on the handwritten image 110. The plurality of designated coordinate data PD include first designated coordinate data PD1, second designated coordinate data PD2, third designated coordinate data PD3, ..., and nth designated coordinate data PDn, where n is an integer equal to or greater than 4. The designated coordinate data group PDs is composed of the first designated coordinate data PD1, second designated coordinate data PD2, third designated coordinate data PD3, ..., and nth designated coordinate data PDn. The drawing control unit 73 calculates a corrected coordinate data group RDs based on the designated coordinate data group PDs including the first designated coordinate data PD1, second designated coordinate data PD2, third designated coordinate data PD3, ..., and nth designated coordinate data PDn.

[0065] FIG. 7 shows an example of a projected image PG displayed on the display surface 2S. FIG. 7 shows the projected image PG when a user draws a handwritten image 110. FIG. 7 shows a first handwritten image 111, which is an example of the handwritten image 110. The first handwritten image 111 is an example of a handwritten wavy line. FIG. 7 shows a plurality of corrected coordinate data RD calculated based on the pointed coordinate data group PDs representing the first handwritten image 111. FIG. 7 schematically shows the plurality of corrected coordinate data RD. Marks representing the corrected coordinate data RD are not displayed on the projected image PG. FIG. 7 virtually shows a first corrected-drawn image 121, which is an example of the corrected-drawn image 120.

[0066] The plurality of corrected coordinate data RD are calculated by the drawing control unit 73 based on the pointed coordinate data PD included in the pointed coordinate data group PDs. The plurality of corrected coordinate data RD indicate coordinate points on the first corrected drawn image 121. The plurality of corrected coordinate data RD are first corrected coordinate data RD1, second corrected coordinate data RD2, third corrected coordinate data RD3, ..., m-th corrected coordinate data RDm. m indicates an integer equal to or greater than 4. m may be the same as or different from n. The corrected coordinate data group RDs is composed of the first corrected coordinate data RD1, second corrected coordinate data RD2, third corrected coordinate data RD3, ..., m-th corrected coordinate data RDm. The drawing control unit 73 calculates the corrected coordinate data group RDs including the first corrected coordinate data RD1, second corrected coordinate data RD2, third corrected coordinate data RD3, ..., m-th corrected coordinate data RDm based on the pointed coordinate data group PDs.

[0067] The drawing control unit 73 calculates the average values ​​of the amplitude and wavelength of the handwritten wavy line based on the pointed coordinate data group PDs. The drawing control unit 73 determines a shaped wavy line having the calculated average values ​​as its amplitude and wavelength as the graphic shape. The drawing control unit 73 determines the width of the shaped wavy line based on the first pointed coordinate data PD1 and the nth pointed coordinate data PDn. The drawing control unit 73 calculates a corrected coordinate data group RDs including multiple corrected coordinate data RD based on the calculated amplitude, wavelength, and width. FIG. 7 shows multiple corrected coordinate data RD included in the calculated corrected coordinate data group RDs. The first corrected drawn image 121 represented by the corrected coordinate data group RDs becomes a shaped wavy line.

[0068] FIG. 8 shows an example of a projection image PG displayed on the display surface 2S. FIG. 8 shows the projection image PG when a user draws a handwritten image 110. FIG. 8 shows the projection image PG displaying a first corrected drawn image 121 converted based on a first handwritten image 111. The first corrected drawn image 121 is an example of a corrected drawn image 120. The first corrected drawn image 121 is an example of a shaped wavy line. FIG. 8 shows an example of the corrected drawn image 120 when it is displayed.

[0069] After calculating the corrected coordinate data group RDs, the drawing control unit 73 outputs the corrected coordinate data group RDs to the display control unit 75. The display control unit 75 causes the image projection device 30 to project a first corrected drawn image 121 based on the corrected coordinate data group RDs. The first corrected drawn image 121 is a shaped wavy line based on the designated coordinate data group PDs of the first handwritten drawn image 111. The image projection device 30 displays the first corrected drawn image 121, which is a shaped wavy line, on the display surface 2S. As an example, when displaying the first corrected drawn image 121 on the display surface 2S, the display control unit 75 erases the first handwritten drawn image 111. The image projection device 30 displays the first corrected drawn image 121 on the display surface 2S and erases the first handwritten drawn image 111.

[0070] The control flow includes acquiring a pointed coordinate data group PDs including a plurality of pointed coordinate data PD indicating a first handwritten drawing image 111 drawn by a user, calculating a corrected coordinate data group RDs including a plurality of corrected coordinate data RD indicating a first corrected drawing image 121 based on the pointed coordinate data group PDs, and displaying the first corrected drawing image 121 based on the corrected coordinate data group RDs. The projector 1 can display a shaped wavy line corresponding to a handwritten wavy line. The projector 1 can display a plurality of types of shaped wavy lines.

[0071] The projector 1 includes an image projection device 30 and a control processor 70. The control processor 70 acquires a pointed coordinate data group PDs including a plurality of pointed coordinate data PD representing a first handwritten image 111 drawn by a user, calculates a corrected coordinate data group RDs including a plurality of corrected coordinate data RD representing a first corrected drawn image 121 based on the pointed coordinate data group PDs, and causes the image projection device 30 to display the first corrected drawn image 121 based on the corrected coordinate data group RDs. The projector 1 can display a shaped wavy line corresponding to a handwritten wavy line. The projector 1 can display a plurality of types of shaped wavy lines.

[0072] Fig. 9 shows an example of a projection image PG displayed on the display surface 2S. Fig. 9 shows the projection image PG when a user draws a handwritten image 110. Fig. 9 shows the projection image PG including a first handwritten image 111 and a first corrected drawn image 121. Fig. 9 shows an example when a corrected drawn image 120 is displayed.

[0073] After calculating the corrected coordinate data group RDs, the drawing control unit 73 outputs the corrected coordinate data group RDs to the display control unit 75. The display control unit 75 causes the image projection device 30 to project a first corrected drawn image 121 based on the corrected coordinate data group RDs. The image projection device 30 displays the first corrected drawn image 121, which is a shaped wavy line, on the display surface 2S. When displaying the first corrected drawn image 121 on the display surface 2S, the display control unit 75 displays the first handwritten drawn image 111 so as to be superimposed thereon. The image projection device 30 displays the first handwritten drawn image 111 and the first corrected drawn image 121 on the display surface 2S. Displaying the first handwritten drawn image 111 and the first corrected drawn image 121 on the display surface 2S makes the difference between the first handwritten drawn image 111 and the first corrected drawn image 121 clear. This makes it easier to see the difference between the handwritten wavy line and the shaped wavy line.

[0074] When displaying the first handwritten image 111 and the first corrected-drawn image 121 on the display surface 2S, the display control unit 75, for example, differentiates the display mode of the first handwritten image 111 from the display mode of the first corrected-drawn image 121. The display mode includes at least one of the line color, the line shape, and the line thickness. By differentiating the display mode of the first handwritten image 111 from the display mode of the first corrected-drawn image 121, the user can more easily distinguish between the first handwritten image 111 and the first corrected-drawn image 121.

[0075] After the first handwritten image 111 and the first corrected drawn image 121 are displayed on the display surface 2S, the user can erase at least one of the first handwritten image 111 and the first corrected drawn image 121. The user can select a desired drawn image and display it on the display surface 2S.

[0076] It is preferable that the first corrected-drawn image 121 be displayed in a different display mode from that of the first handwritten image 111 . Users can more easily distinguish between shaped wavy lines and handwritten wavy lines.

[0077] Fig. 10 shows an example of a projected image PG displayed on display surface 2S. Fig. 10 shows a projected image PG when a user draws a handwritten image 110. Fig. 10 shows a second handwritten image 112, which is an example of handwritten image 110. Second handwritten image 112 is an example of a handwritten wavy line. Second handwritten image 112 is drawn by the user in drawing area DA. Second handwritten image 112 is drawn, for example, from the -X direction to the +X direction.

[0078] Fig. 11 shows an example of a projection image PG displayed on the display surface 2S. Fig. 11 shows the projection image PG when a user draws a handwritten image 110. Fig. 11 shows a second handwritten image 112, which is an example of the handwritten image 110. Fig. 11 shows a plurality of pieces of designated coordinate data PD corresponding to the second handwritten image 112. Marks indicating the designated coordinate data PD are not displayed in the projection image PG.

[0079] The plurality of designated coordinate data PD are acquired by the drawing control unit 73 based on the captured image data. The plurality of designated coordinate data PD indicate coordinate points on the second handwritten image 112. The plurality of designated coordinate data PD are first designated coordinate data PD1, second designated coordinate data PD2, third designated coordinate data PD3, ..., kth designated coordinate data PDk, ..., nth designated coordinate data PDn. n is an integer equal to or greater than 4. k is an arbitrary integer smaller than n. The first designated coordinate data PD1, second designated coordinate data PD2, third designated coordinate data PD3, ..., kth designated coordinate data PDk, ..., nth designated coordinate data PDn constitute a designated coordinate data group PDs.

[0080] The designated coordinate data group PDs includes a partial designated coordinate data group pPDs. For example, the partial designated coordinate data group pPDs includes first designated coordinate data PD1, second designated coordinate data PD2, third designated coordinate data PD3, ..., and k-th designated coordinate data PDk. The partial designated coordinate data group pPDs corresponds to an example of a partial coordinate data group. The first designated coordinate data PD1, second designated coordinate data PD2, third designated coordinate data PD3, ..., and k-th designated coordinate data PDk each correspond to an example of partial image coordinates. The partial designated coordinate data group pPDs represents a partial handwritten image 112a. The partial handwritten image 112a corresponds to a partial image that is a part of a handwritten wavy line. The drawing control unit 73 may calculate a corrected coordinate data group RDs based on the partial designated coordinate data group pPDs.

[0081] FIG. 12 shows an example of a projected image PG displayed on the display surface 2S. FIG. 12 shows the projected image PG when a user draws a handwritten image 110. FIG. 12 shows a second handwritten image 112, which is an example of the handwritten image 110. FIG. 12 shows a plurality of corrected coordinate data RD corresponding to the second handwritten image 112. FIG. 12 schematically shows the plurality of corrected coordinate data RD. Marks indicating the corrected coordinate data RD are not displayed on the projected image PG. FIG. 12 virtually shows a second corrected-drawn image 122, which is an example of the corrected-drawn image 120.

[0082] The drawing control unit 73 acquires the designated coordinate data PD in the order of first designated coordinate data PD1, second designated coordinate data PD2, third designated coordinate data PD3, ..., and kth designated coordinate data PDk. The drawing control unit 73 calculates a corrected coordinate data group RDs based on a partial designated coordinate data group pPDs including the first designated coordinate data PD1, second designated coordinate data PD2, third designated coordinate data PD3, ..., and kth designated coordinate data PDk. The drawing control unit 73 determines the shape of the second handwritten image 112 from the partial designated coordinate data group pPDs. The drawing control unit 73 determines that the second handwritten image 112 is a handwritten wavy line from the partial designated coordinate data group pPDs. If the drawing control unit 73 determines that the second handwritten image 112 is a handwritten wavy line, it calculates the amplitude and wavelength of the handwritten wavy line based on the partial designated coordinate data group pPDs. The drawing control unit 73 determines the waveform of the shaped wavy line by calculating the amplitude and wavelength of the handwritten wavy line.

[0083] The drawing control unit 73 acquires the first pointed coordinate data PD1 and the nth pointed coordinate data PDn, and calculates the width of the second handwritten image 112 based on the first pointed coordinate data PD1 and the nth pointed coordinate data PDn.

[0084] The drawing control unit 73 calculates a plurality of corrected coordinate data RD based on each of the designated coordinate data PD included in the partial designated coordinate data group pPDs. The plurality of corrected coordinate data RD includes first corrected coordinate data RD1, second corrected coordinate data RD2, third corrected coordinate data RD3, ..., k-th corrected coordinate data RDk. The first corrected coordinate data RD1 to the k-th corrected coordinate data RDk are converted coordinate data calculated based on the designated coordinate data PD included in the partial designated coordinate data group pPDs.

[0085] After calculating the kth correction coordinate data RDk, the drawing control unit 73 calculates the k+1th correction coordinate data RDk+1 through the mth correction coordinate data RDm. The drawing control unit 73 calculates each of the k+1th correction coordinate data RDk+1 through the mth correction coordinate data RDm using the amplitude and wavelength of the handwritten wavy line calculated based on the partial indication coordinate data group pPDs. Each of the k+1th correction coordinate data RDk+1 through the mth correction coordinate data RDm is interpolated coordinate data interpolated using the calculated amplitude and wavelength of the handwritten wavy line. The drawing control unit 73 generates a correction coordinate data group RDs including the first correction coordinate data RD1, the second correction coordinate data RD2, the third correction coordinate data RD3, ..., the kth correction coordinate data RDk, the k+1st correction coordinate data RDk+1, ..., and the mth correction coordinate data RDm. The drawing control unit 73 outputs the correction coordinate data group RDs to the display control unit 75.

[0086] Fig. 13 shows an example of a projected image PG displayed on the display surface 2S. Fig. 13 shows the projected image PG when a user draws a handwritten image 110. Fig. 13 shows the projected image PG displaying a second corrected drawn image 122 converted based on a second handwritten image 112. The second corrected drawn image 122 is an example of a corrected drawn image 120. Fig. 13 shows an example of the corrected drawn image 120 when it is displayed.

[0087] The display control unit 75 acquires the corrected coordinate data group RDs output from the drawing control unit 73. The display control unit 75 causes the image projection device 30 to project the second corrected drawn image 122 based on the corrected coordinate data group RDs. The second corrected drawn image 122 is a shaped wavy line based on the designated coordinate data group PDs of the second handwritten drawn image 112. The image projection device 30 displays the second corrected drawn image 122, which is a shaped wavy line, on the display surface 2S. As an example, when displaying the second corrected drawn image 122 on the display surface 2S, the display control unit 75 erases the second handwritten drawn image 112. The image projection device 30 displays the second corrected drawn image 122 on the display surface 2S and erases the second handwritten drawn image 112.

[0088] The second corrected-drawn image 122 displayed on the display surface 2S is composed of a first region R1 and a second region R2. The first region R1 is a region drawn based on the converted coordinate data. The second region R2 is a region drawn based on the interpolated coordinate data. The portion of the second handwritten image 112 different from the partial handwritten image 112a is drawn based on the designated coordinate data PD included in the partial designated coordinate data group pPDs. The handwritten wavy lines corresponding to the portion of the second handwritten image 112 different from the partial handwritten image 112a are replaced with shaped wavy lines obtained by shaping the handwritten wavy lines corresponding to the partial handwritten image 112a, and then displayed. When the user draws a portion of the handwritten image 110 with a handwritten wavy line that resembles a desired shape, the projector 1 can convert the entire handwritten image 110 into a corrected-drawn image 120, which is a shaped wavy line corresponding to the desired shape, and display it.

[0089] The pointed coordinate data group PDs includes a partial pointed coordinate data group pPDs including first pointed coordinate data PD1, second pointed coordinate data PD2, third pointed coordinate data PD3, ..., kth pointed coordinate data PDk, which indicate a partial handwritten image 112a that is a part of the second handwritten image 112. The control flow includes determining a waveform of the second corrected-drawn image 122 based on the partial pointed coordinate data group pPDs. Displaying the second corrected-drawn image 122 preferably includes displaying the second corrected-drawn image 122 having the determined waveform. The projector 1 can display a shaped wavy line using the partial designated coordinate data group pPDs indicating the partial handwritten image 112a.

[0090] It is preferable that displaying the second corrected drawing image 122 means displaying the second handwritten drawing image 112, which is different from the partial handwritten drawing image 112a, as the second corrected drawing image 122 having a waveform determined based on the partial instruction coordinate data group pPDs. When the user draws a part of the second handwritten image 112 with handwritten wavy lines that approximate a desired shape, the projector 1 can display the entire second handwritten image 112 with shaped wavy lines that correspond to the desired shape.

[0091] The user can edit the graphic shape of the corrected drawn image 120 displayed on the display surface 2S. Figures 14, 15, 16, and 17 show the projected image PG displayed on the display surface 2S when the user edits the corrected drawn image 120.

[0092] Fig. 14 shows an example of a projection image PG displayed on the display surface 2S. Fig. 14 shows a projection image PG including a corrected drawn image 120. Fig. 14 shows a projection image PG including a third corrected drawn image 123, which is an example of the corrected drawn image 120. The third corrected drawn image 123 is a shaped wavy line obtained by shaping a third handwritten drawn image (not shown). The third corrected drawn image 123 shown in Fig. 14 is a shaped wavy line with a first corrected image amplitude RA1.

[0093] FIG. 14 shows a state when the user has brought the operation pen 5 into contact with a portion of the third-corrected-drawn image 123. By bringing the operation pen 5 into contact with a portion of the third-corrected-drawn image 123, the user can edit the third-corrected-drawn image 123. The user performs a predetermined operation while the operation pen 5 is in contact with a portion of the third-corrected-drawn image 123. One example of the predetermined operation is moving the operation pen 5 on the display surface 2S. The user transforms the third-corrected-drawn image 123 by performing the predetermined operation using the operation pen 5. One example of the user moving the operation pen 5 in the -Y direction. When the user moves the operation pen 5 in the -Y direction, the third-corrected-drawn image 123 is edited into a fourth-corrected-drawn image 124 shown in FIG. 15.

[0094] Fig. 15 shows an example of a projection image PG displayed on the display surface 2S. Fig. 15 shows a projection image PG including a corrected drawn image 120. Fig. 15 shows a projection image PG including a fourth corrected drawn image 124, which is an example of the corrected drawn image 120.

[0095] The fourth corrected-drawn image 124 is a shaped wavy line obtained by editing the third corrected-drawn image 123. The fourth corrected-drawn image 124 shown in FIG. 15 is a shaped wavy line with the second corrected image amplitude RA2. When the user moves the operation pen 5 in the -Y direction while the operation pen 5 is in contact with the third corrected-drawn image 123, the waveform of the shaped wavy line changes from the waveform with the first corrected image amplitude RA1 to the waveform with the second corrected image amplitude RA2. The second corrected image amplitude RA2 is narrower than the first corrected image amplitude RA1. The user can change the amplitude of the shaped wavy line by performing a predetermined operation. The projector 1 displays the fourth corrected-drawn image 124, which is a shaped wavy line with the second corrected image amplitude RA2, on the display surface 2S.

[0096] Fig. 16 shows an example of a projected image PG displayed on the display surface 2S. Fig. 16 shows a projected image PG including a corrected drawn image 120. Fig. 16 shows a projected image PG including a third corrected drawn image 123, which is an example of the corrected drawn image 120. The third corrected drawn image 123 is a shaped wavy line obtained by shaping a third handwritten drawn image (not shown). The third corrected drawn image 123 shown in Fig. 16 is a shaped wavy line with a first corrected image width RW1 and a first corrected image wavelength RL1.

[0097] FIG. 16 shows a state when the user has brought the operation pen 5 into contact with an edge of the third-corrected-drawn image 123. By bringing the operation pen 5 into contact with the edge of the third-corrected-drawn image 123, the user can edit the third-corrected-drawn image 123. The user performs a predetermined operation while the operation pen 5 is in contact with the edge of the third-corrected-drawn image 123. The user transforms the third-corrected-drawn image 123 by performing the predetermined operation using the operation pen 5. As an example, the user moves the operation pen 5 in the −X direction. When the user moves the operation pen 5 in the −X direction, the third-corrected-drawn image 123 is edited into a fifth-corrected-drawn image 125 shown in FIG. 17.

[0098] Fig. 17 shows an example of a projected image PG displayed on the display surface 2S. Fig. 17 shows a projected image PG including a corrected drawn image 120. Fig. 17 shows a projected image PG including a fifth corrected drawn image 125, which is an example of the corrected drawn image 120.

[0099] The fifth corrected-drawn image 125 is a shaped wavy line obtained by editing the third corrected-drawn image 123. The fifth corrected-drawn image 125 shown in FIG. 17 is a shaped wavy line with a second corrected image width RW2 and a second corrected image wavelength RL2. The second corrected image width RW2 is narrower than the first corrected image width RW1. The second corrected image wavelength RL2 is shorter than the first corrected image wavelength RL1. When the user moves the operation pen 5 in the −X direction while touching the third corrected-drawn image 123, the waveform of the shaped wavy line changes from the waveform shown in FIG. 16 to the waveform shown in FIG. 17. The user can change the width and wavelength of the shaped wavy line by performing a predetermined operation. The user can change the waveform of the shaped wavy line by performing a predetermined operation. The projector 1 displays the fifth corrected-drawn image 125 with the second corrected image width RW2 and the second corrected image wavelength RL2 on the display surface 2S. 17, the fifth corrected drawn image 125 has a changed width and wavelength from the third corrected drawn image 123, but is not limited to this. The drawing control unit 73 may change either the width or the wavelength of the shaped wavy line when the user performs a predetermined operation.

[0100] It is preferable that displaying the fourth corrected drawn image 124 includes changing and displaying the waveform of the third corrected drawn image 123 when a predetermined operation is performed by the user on the third corrected drawn image 123. The user can edit the wavy line to shape it to the desired waveform.

[0101] 4, the drawing control unit 73 determines the waveform of the shaped wavy line by calculating the wavy line dimensions including the amplitude, wavelength, and width of the handwritten wavy line, but is not limited to this. The drawing control unit 73 may also determine the waveform of the shaped wavy line by selecting a set waveform SF stored in the storage unit 60.

[0102] The drawing control unit 73 calculates the wavy line dimensions, including the amplitude, wavelength, and width, of the handwritten wavy line that is the handwritten image 110. The drawing control unit 73 selects one set waveform SF from multiple set waveforms SF stored in the storage unit 60 based on the calculated wavy line dimensions. As an example, the drawing control unit 73 selects a set waveform SF that matches or approximates the wavy line dimensions of the handwritten wavy line as the waveform of the shaped wavy line. By selecting one set waveform SF, the drawing control unit 73 determines the waveform of the shaped wavy line. By determining the waveform of the shaped wavy line using the set waveform SF stored in advance, the drawing control unit 73 can reduce the load when determining the waveform.

[0103] 18, 19, and 20 show examples of the set waveform SF. FIG. 18 shows the waveform of a first set waveform SF1, which is an example of the set waveform SF. FIG. 19 shows the waveform of a second set waveform SF2, which is an example of the set waveform SF. FIG. 20 shows the waveform of a third set waveform SF3, which is an example of the set waveform SF. The set waveforms SF stored in the storage unit 60 are not limited to the first set waveform SF1, the second set waveform SF2, and the third set waveform SF3. The storage unit 60 stores two or more set waveforms SF in advance.

[0104] The first set waveform SF1 shown in FIG. 18 is a wavy line of a first set wavelength SWL1 and a first set amplitude SA1. The first set waveform SF1 corresponds to an example of a first waveform. The second set waveform SF2 shown in FIG. 19 is a wavy line of a second set wavelength SWL2 and a second set amplitude SA2. The second set waveform SF2 corresponds to an example of a second waveform. The second set waveform SF2 is different from the first set waveform SF1. The second set wavelength SWL2 is shorter than the first set wavelength SWL1. The second set amplitude SA2 is narrower than the first set amplitude SA1. The third set waveform SF3 shown in FIG. 20 is a wavy line of a third set wavelength SWL3 and a third set amplitude SA3. The third set waveform SF3 is different from the first set waveform SF1 and the second set waveform SF2. The third set wavelength SWL3 is shorter than the first set wavelength SWL1 and the second set wavelength SWL2. The third set amplitude SA3 is narrower than the first set amplitude SA1 and the second set amplitude SA2.

[0105] When the drawing control unit 73 determines that the wavy line dimensions of the handwritten wavy line in the handwritten image 110 match or are similar to those of the first set waveform SF1, it calculates a first set of corrected coordinate data for displaying a shaped wavy line having the first set waveform SF1 as its waveform. The first set of corrected coordinate data is an example of a set of corrected coordinate data RDs and corresponds to an example of a first coordinate data set. The drawing control unit 73 outputs the first set of corrected coordinate data to the display control unit 75. The display control unit 75 controls the image projection device 30 to project a projection image PG including a corrected drawn image 120, which is a shaped waveform, based on the first set of corrected coordinate data. The image projection device 30 displays the shaped waveform of the first set waveform SF1 on the display surface 2S.

[0106] When the drawing control unit 73 determines that the wavy line dimensions of the handwritten wavy line in the handwritten image 110 match or are similar to those of the second set waveform SF2, it calculates a second set of corrected coordinate data for displaying a shaped wavy line having the second set waveform SF2 as its waveform. The second set of corrected coordinate data is an example of a set of corrected coordinate data RDs and corresponds to an example of a second coordinate data set. The drawing control unit 73 outputs the second set of corrected coordinate data to the display control unit 75. The display control unit 75 controls the image projection device 30 to project a projection image PG including a corrected drawn image 120, which is a shaped waveform, based on the second set of corrected coordinate data. The image projection device 30 displays the shaped waveform of the second set waveform SF2 on the display surface 2S.

[0107] It is preferable that the control flow includes calculating, based on the instruction coordinate data group PDs, a first correction coordinate data group that displays a shaped wave line of the first setting waveform SF1, or a second correction coordinate data group that displays a shaped wave line of the second setting waveform SF2 that is different from the first setting waveform SF1, as the correction coordinate data group RDs. The projector 1 can reduce the load when determining the waveform of the shaped wave line to be displayed on the display surface 2S.

[0108] FIG. 21 shows an example of a control flow executed by the projector 1. FIG. 21 shows the control flow in a flowchart. FIG. 21 shows the control flow when a user uses the operation pen 5 to draw a handwritten wavy line on the display surface 2S on which a character string image 130 is previously displayed. When the handwritten wavy line is drawn in a predetermined setting area SR for the character string image 130, the projector 1 can display a shaped wavy line in a predetermined display area DR set for the character string image 130. In addition, the width of the shaped wavy line is corrected to the corrected image width RW. FIG. 21 shows the control flow when the handwritten wavy line, which is the handwritten image 110, is corrected to a shaped wavy line, which is the corrected image 120, and the shaped wavy line is displayed on the display surface 2S.

[0109] In step S201, the projector 1 generates captured data. The capturing unit 15 captures the operation pen 5 at predetermined time intervals to generate the captured data. The capturing unit 15 captures the display surface 2S before the user draws on the display surface 2S and the display surface 2S while the user is drawing on the display surface 2S to generate the captured data. The capturing unit 15 captures the drawing image included in the projected image PG by capturing the display surface 2S before the user draws on the display surface 2S. The capturing unit 15 captures the operation pen 5 on the display surface 2S by capturing the display surface 2S while the user is drawing on the display surface 2S with the operation pen 5. The capturing unit 15 outputs the captured data to the control processor 70.

[0110] In step S203, the projector 1 acquires character string image information. The drawing control unit 73 acquires, from the imaging unit 15, imaging data of the display surface 2S before the user draws on the display surface 2S. The drawing control unit 73 acquires character string image information based on the imaging data. The character string image information is information related to the character string image 130 included in the projection image PG. The character string image 130 is an image in which multiple characters are arranged side by side. The character string image 130 may be a drawing image handwritten by the user or a drawing image based on image data received from an external device. The character string image 130 corresponds to an example of a character string in which multiple characters are arranged side by side. The character string image information includes the character string image position where the character string image 130 is drawn, the character string image width CW of the character string image 130, the character string height, and the character string inclination. The character string image width CW is the width along the X-axis. The character string image width CW corresponds to an example of the character string width. The character string height is the length along the Y-axis. The character string inclination is the inclination of the arrangement direction of the characters with respect to the X-axis. The drawing control unit 73 acquires the character string image information and stores it in the storage unit 60.

[0111] In step S205, the projector 1 acquires the pointed coordinate data PD. The drawing control unit 73, which operates on the control processor 70, receives the captured image data when the user is drawing on the display surface 2S. The drawing control unit 73 acquires the pointed coordinate data PD based on the captured image data. The drawing control unit 73 acquires the pointed coordinate data PD based on each piece of captured image data captured at a predetermined time interval. The drawing control unit 73 stores the multiple pointed coordinate data PD as a pointed coordinate data group PDs in the memory unit 60.

[0112] After acquiring the pointed coordinate data PD, in step S207, the projector 1 determines the graphic shape of the handwritten image 110. The drawing control unit 73 determines the graphic shape of the handwritten image 110 from the multiple pointed coordinate data PD included in the pointed coordinate data group PDs or from part of the multiple pointed coordinate data PD included in the pointed coordinate data group PDs.

[0113] After determining the graphic shape of the handwritten image 110, the projector 1 determines in step S209 whether the graphic shape is a wavy line. If the drawing control unit 73 determines that the graphic shape is a wavy line, the projector 1 proceeds to step S211 (step S209: YES). If the drawing control unit 73 determines that the graphic shape is not a wavy line, the projector 1 proceeds to step S221 (step S209: NO).

[0114] In step S211, the projector 1 determines whether the handwritten image 110 is drawn within the set region SR. The set region SR is set in advance based on character string image information and stored in the storage unit 60. When the projected image PG includes one character string image 130, the set region SR is, for example, an area below or to the side of the character string image 130, within a predetermined distance from the character string image 130. When the projected image PG includes two character string images 130, the set region SR is an area between the two character string images 130.

[0115] When at least a part of the handwritten wavy line that is the handwritten image 110 is drawn in the set area SR, the drawing control unit 73 determines that the handwritten wavy line is within the set area SR. The projector 1 proceeds to step S213 (step S211: YES). When the handwritten wavy line that is the handwritten image 110 is not drawn in the set area SR, the drawing control unit 73 determines that the handwritten wavy line is not within the set area SR. The projector 1 proceeds to step S215 (step S211: NO).

[0116] In step S213, the projector 1 generates correction data. The drawing control unit 73 acquires character string image information from the storage unit 60. Based on the character string image information, the drawing control unit 73 determines a display area DR in which the corrected drawn image 120 is to be displayed. The display area DR corresponds to an example of a drawing position. When the projected image PG includes one character string image 130, the display area DR is, for example, an area below or to the side of the character string image 130 and within a predetermined distance from the character string image 130. When the projected image PG includes two character string images 130, the display area DR is an area between the two character string images 130. The display area DR may be the same area as the setting area SR or may be a different area. It is preferable that the display area DR be narrower than the setting area SR. The drawing control unit 73 generates display coordinate data corresponding to the display area DR. The drawing control unit 73 determines the display size of the corrected drawn image 120 to be displayed in the display area DR. The size of the corrected drawn image 120 displayed in the display area DR includes the corrected image width RW and the corrected image height of the corrected drawn image 120. The drawing control unit 73 generates display size data based on the character string image information. The drawing control unit 73 generates correction data including display coordinate data and display size data. The drawing control unit 73 stores the correction data in the storage unit 60.

[0117] In step S215, the projector 1 calculates a corrected coordinate data group RDs. The drawing control unit 73 acquires the pointed coordinate data group PDs or a portion of pointed coordinate data PD included in the pointed coordinate data group PDs. The drawing control unit 73 calculates wavy line dimensions, including amplitude, wavelength, and width, of the handwritten wavy line, which is the handwritten image 110, based on the pointed coordinate data group PDs. The amplitude and wavelength are calculated using a predetermined algorithm. The drawing control unit 73 determines the waveform of the shaped wavy line, which is the corrected drawn image 120, by calculating the wavy line dimensions, including amplitude, wavelength, and width, of the handwritten wavy line.

[0118] The drawing control unit 73 calculates corrected coordinate data RD using the designated coordinate data PD and the calculated wavy line dimensions of the handwritten wavy line. The drawing control unit 73 calculates corrected coordinate data RD corresponding to each designated coordinate data PD to calculate a corrected coordinate data group RDs including multiple corrected coordinate data RD. The corrected coordinate data group RDs represents a shaped wavy line.

[0119] When the storage unit 60 stores the correction data, the drawing control unit 73 corrects the corrected coordinate data group RDs calculated using the correction data. The drawing control unit 73 corrects the display position where the shaped wavy line is displayed within the display region DR using the display coordinate data included in the correction data. The drawing control unit 73 corrects the size of the shaped wavy line, such as the amplitude and width, based on the size data included in the correction data. The drawing control unit 73 outputs the corrected coordinate data group RDs or the corrected corrected coordinate data group RDs to the display control unit 75.

[0120] After calculating the corrected coordinate data group RDs, the projector 1 displays the corrected drawn image 120 in step S217. The display control unit 75 causes the image projection device 30 to display the corrected drawn image 120 based on the corrected coordinate data group RDs. The corrected drawn image 120 is a shaped wavy line obtained by shaping a handwritten wavy line in the drawing control unit 73. The shaped wavy line is a wavy line having a constant amplitude and a constant wavelength. The display control unit 75 outputs the drawing data including the corrected coordinate data group RDs to the image generation device 40. The image generation device 40 performs various image processing on the drawing data. After performing the various image processing, the image generation device 40 outputs the drawing data including the corrected coordinate data group RDs to the image projection device 30. The image generation device 40 performs processing to combine the corrected drawn image 120 with an OSD image, and then outputs the drawing data including the corrected coordinate data group RDs to the image projection device 30. The image projection device 30 projects the corrected drawn image 120 based on the corrected coordinate data group RDs onto the display surface 2S. By projecting the corrected drawn image 120 onto the display surface 2S, the image projection device 30 displays the corrected drawn image 120, which is a shaped wavy line, on the display surface 2S. The display surface 2S displays a projection image PG including the shaped wavy line. The shaped wavy line is displayed at a predetermined position and with a predetermined size relative to the character string image 130.

[0121] In step S221, the projector 1 executes a correction process according to the graphic shape. After determining in step S209 that the graphic shape is not a wavy line, the rendering control unit 73 further determines the graphic shape. The determined graphic shape is set in advance. The determined graphic shapes include a circle, a square, a triangle, a straight line, and the like. The determined graphic shape does not have to be set other than a wavy line. If the rendering control unit 73 determines that the graphic shape is a predetermined graphic shape, it executes a correction process according to the graphic shape and determines a correction graphic. The correction process includes a shaping process, a deformation process, a replacement process, and the like. The shaping process is a process of shaping a graphic shape to create a correction graphic. The deformation process is a process of transforming a graphic shape into a predetermined shape to create a correction graphic. The replacement process is a process of replacing a graphic shape with another shape to create a correction graphic. The rendering control unit 73 calculates a correction coordinate data group RDs corresponding to the correction graphic and outputs it to the display control unit 75.

[0122] After performing correction processing according to the shape of the figure, the projector 1 displays a corrected drawn image 120 of the corrected figure in step S223. The display control unit 75 acquires the corrected coordinate data group RDs. The display control unit 75 displays the corrected figure on the display surface 2S based on the corrected coordinate data group RDs. The display control unit 75 outputs drawing data including the corrected coordinate data group RDs to the image generation device 40. The image generation device 40 performs various image processing on the drawing data. After performing various image processing, the image generation device 40 outputs the drawing data including the corrected coordinate data group RDs to the image projection device 30. The image generation device 40 performs processing to combine the correction figure with an OSD image, and then outputs the drawing data including the corrected coordinate data group RDs to the image projection device 30. The image projection device 30 projects the correction figure based on the corrected coordinate data group RDs onto the display surface 2S. The image projection device 30 displays the correction figure on the display surface 2S by projecting the correction figure onto the display surface 2S. The display surface 2S displays the correction figure.

[0123] Fig. 22 shows an example of a projection image PG displayed on the display surface 2S. Fig. 22 shows a projection image PG including a character string image 130. Fig. 22 shows the projection image PG before the user draws a handwritten image 110.

[0124] FIG. 22 shows a projected image PG including a first character string image 131, which is an example of the character string image 130. The first character string image 131 is a drawn image in which a plurality of characters are arranged side by side along a character string direction CD. The character string direction CD shown in FIG. 22 is a direction parallel to the X-axis. The character string direction CD corresponds to an example of an arrangement direction. The first character string image 131 is a drawn image having a first character string image width CW1 along the character string direction CD. The first character string image width CW1 is an example of a character string image width CW.

[0125] The drawing control unit 73 sets a setting region SR for the character string image 130. FIG. 22 shows a first setting region SR1, which is an example of the setting region SR. The first setting region SR1 shown in FIG. 22 is set, as an example, at a position below the first character string image 131. The first setting region SR1 is set at a position overlapping with a portion of the first character string image 131. The position of the first setting region SR1 is not limited to the position shown in FIG. 22. The first setting region SR1 may be set above or to the side of the first character string image 131. The first setting region SR1 may be set at a position not overlapping with a portion of the first character string image 131.

[0126] FIG. 23 shows an example of a projected image PG displayed on the display surface 2S. FIG. 23 shows a projected image PG including a character string image 130 and a handwritten image 110. FIG. 23 shows the projected image PG when a user draws a handwritten image 110. FIG. 23 shows the projected image PG when a fourth handwritten image 114 is drawn on the display surface 2S on which the first character string image 131 is displayed. The fourth handwritten image 114 is an example of the handwritten image 110 and is a handwritten wavy line. As an example, the fourth handwritten image 114 is drawn along the X-axis parallel to the character string direction CD with a fourth handwritten image width MW4. The fourth handwritten image width MW4 is an example of the handwritten image width MW. The handwritten image width MW corresponds to an example of an image width. The fourth handwritten image width MW4 is different from the first character string image width CW1. The fourth handwritten image width MW4 shown in FIG. 23 is longer than the first character string image width CW1.

[0127] A portion of fourth handwritten image 114 shown in Fig. 23 is drawn within first set region SR1. Drawing control unit 73 determines that a portion of fourth handwritten image 114 is within first set region SR1. Drawing control unit 73 determines that the handwritten wavy line is an underline to be drawn below first character string image 131. Drawing control unit 73 displays a shaped wavy line obtained by shaping fourth handwritten image 114, which is a handwritten wavy line, as corrected drawn image 120 below first character string image 131.

[0128] Fig. 24 shows an example of a projection image PG displayed on the display surface 2S. Fig. 24 shows a projection image PG including a character string image 130 and a corrected-drawn image 120. Fig. 24 shows a projection image PG when a user draws a handwritten image 110. The handwritten image 110 is corrected to the corrected-drawn image 120. Fig. 24 shows a projection image PG when a sixth corrected-drawn image 126 is displayed on the display surface 2S on which the first character string image 131 is displayed.

[0129] The sixth corrected drawn image 126 is displayed by the drawing control unit 73 correcting the fourth handwritten drawn image 114. The sixth corrected drawn image 126 is displayed based on the corrected coordinate data group RDs. The corrected coordinate data group RDs is calculated based on the pointed coordinate data group PDs and correction data indicated by the fourth handwritten drawn image 114. The sixth corrected drawn image 126 is an example of the corrected drawn image 120, and is a shaped wavy line.

[0130] The sixth corrected drawn image 126 is displayed in the first display area DR1. The display position of the sixth corrected drawn image 126 is predetermined by the first display area DR1. The first display area DR1 is an example of the display area DR. The shaped wavy line that is the sixth corrected drawn image 126 is displayed as an underline of the first character string image 131. The user can easily draw the shaped wavy line that will be the underline of the character string image 130.

[0131] The width of the first display region DR1 along the X axis is set to be the same as or approximately the same as the first character string image width CW1 of the first character string image 131. The sixth corrected image width RW6 of the sixth corrected drawn image 126 is set to be the width along the character string direction CD of the first display region DR1. The sixth corrected image width RW6 is set within the range of the first character string image width CW1. The sixth corrected drawn image 126 set to the sixth corrected image width RW6 is displayed as an underline of the first character string image 131.

[0132] The control flow includes displaying a first character string image 131 in which a plurality of characters are arranged before drawing a fourth handwritten image 114 which is a handwritten wavy line. It is preferable that displaying a sixth corrected drawn image 126 which is a shaped wavy line is displaying the sixth corrected drawn image 126 in a first display region DR1 which is predetermined for the first character string image 131. The projector 1 can display a shaped wavy line obtained by shaping a handwritten wavy line as a drawing image that decorates the character string image 130.

[0133] It is preferable that the sixth corrected drawing image 126, which is a shaped wavy line, is displayed within the range of the first character string image width CW1 in the character string direction CD when the first character string image width CW1 of the first character string image 131 differs from the fourth handwritten image width MW4 of the fourth handwritten drawing image 114, which is a handwritten wavy line, in the character string direction CD in which multiple characters are lined up. The projector 1 can adjust the sixth corrected image width RW6 of the sixth corrected drawn image 126 in accordance with the first character string image width CW1 of the first character string image 131.

[0134] Fig. 25 shows an example of a projection image PG displayed on the display surface 2S. Fig. 25 shows a projection image PG including a plurality of character string images 130. Fig. 25 shows the projection image PG before the user draws a handwritten image 110.

[0135] FIG. 25 shows a projected image PG including two character string images 130. The two character string images 130 are a second character string image 132 and a third character string image 133. The second character string image 132 and the third character string image 133 are examples of the character string image 130. The second character string image 132 corresponds to an example of a first character string. The third character string image 133 corresponds to an example of a second character string. The second character string image 132 and the third character string image 133 are drawn images in which a plurality of characters are arranged side by side along a first direction D1. The first direction D1 shown in FIG. 25 is parallel to the X-axis. The first direction D1 shown in FIG. 25 is parallel to the X-axis, but is not limited to this. The first direction D1 is set as appropriate. The second character string image 132 is a drawn image having a second character string image width CW2 along the first direction D1. The third character string image 133 is a drawn image having a third character string image width CW3 along the first direction D1. The second character string image width CW2 and the third character string image width CW3 are examples of the character string image width CW.

[0136] The third character string image 133 is disposed at a position separated from the second character string image 132 by a character string distance dc in the second direction D2. The second direction D2 is a direction intersecting the first direction D1. The character string distance dc corresponds to an example of the first distance. The rendering control unit 73 calculates the character string distance dc using the imaging data and stores it in the storage unit 60. The second direction D2 shown in FIG. 25 is a direction parallel to the Y axis, but is not limited to this. The second direction D2 is appropriately set to a direction intersecting the first direction D1. The third character string image 133 is disposed at a position overlapping the second character string image 132 in the first direction D1 when viewed from the second direction D2.

[0137] The drawing control unit 73 sets a setting region SR for the character string image 130. FIG. 25 shows a second setting region SR2, which is an example of the setting region SR. As an example, the second setting region SR2 shown in FIG. 25 is set at a position that includes the region between the second character string image 132 and the third character string image 133. The second setting region SR2 is set at a position that overlaps with a portion of the second character string image 132 and a portion of the third character string image 133. The position of the second setting region SR2 is not limited to the position shown in FIG. 25. The second setting region SR2 may be set at a position that does not overlap with at least one of a portion of the second character string image 132 and a portion of the third character string image 133.

[0138] Fig. 26 shows an example of a projection image PG displayed on the display surface 2S. Fig. 26 shows a projection image PG including a character string image 130 and a handwritten image 110. Fig. 26 shows a projection image PG when a user draws the handwritten image 110. Fig. 26 shows a projection image PG when a fifth handwritten image 115 is drawn on the display surface 2S on which the second character string image 132 and the third character string image 133 are displayed. The fifth handwritten image 115 is an example of a handwritten image 110 and is a handwritten wavy line.

[0139] 26 is drawn within the second set region SR2. The drawing control unit 73 determines that the fifth handwritten image 115 is within the second set region SR2. The drawing control unit 73 determines that the fifth handwritten image 115 is a drawn image drawn between the second character string image 132 and the third character string image 133. The drawing control unit 73 displays the seventh corrected drawn image 127, which is obtained by shaping the fifth handwritten image 115, between the second character string image 132 and the third character string image 133.

[0140] Fig. 27 shows an example of a projection image PG displayed on the display surface 2S. Fig. 27 shows a projection image PG including a character string image 130 and a corrected drawn image 120. Fig. 27 shows a projection image PG when a user draws a handwritten image 110. The handwritten image 110 is corrected to the corrected drawn image 120. Fig. 27 shows a projection image PG when a seventh corrected drawn image 127 is displayed on the display surface 2S on which the second character string image 132 and the third character string image 133 are displayed.

[0141] The seventh corrected drawn image 127 is displayed by the drawing control unit 73 correcting the fifth handwritten drawn image 115. The seventh corrected drawn image 127 is displayed based on a corrected coordinate data group RDs calculated using the correction data. The corrected coordinate data group RDs is calculated based on the pointed coordinate data group PDs and the correction data indicated by the fifth handwritten drawn image 115. The corrected coordinate data group RDs is obtained by calculating a plurality of corrected coordinate data RD using the pointed image data PD and the correction data included in the pointed coordinate data group PDs. The corrected coordinate data group RDs calculated using the correction data corresponds to an example of a corrected calculated coordinate data group. The corrected coordinate data RD calculated using the correction data corresponds to an example of a corrected coordinate. The seventh corrected drawn image 127 is an example of the corrected drawn image 120 and is a shaped wavy line.

[0142] The seventh corrected drawn image 127 is displayed within the second display area DR2. The second display area DR2 is the area between the second character string image 132 and the third character string image 133 in the second direction D2. The second display area DR2 corresponds to an example of the area between the first character string and the second character string. The drawing control unit 73 obtains the character string distance dc in advance and sets the second display area DR2 based on the character string distance dc. The width of the second display area DR2 in the second direction D2 is set to the character string distance dc. The display position of the seventh corrected drawn image 127 is determined in advance by the second display area DR2. The second display area DR2 is an example of a display area DR. The user can easily draw a shaped wavy line between the second character string image 132 and the third character string image 133.

[0143] The control flow includes, before drawing the fifth handwritten image 115, displaying a second character string image 132 in which a plurality of characters are arranged in a first direction D1 and a third character string image 133 arranged at a character string distance dc from the second character string image 132 in a second direction D2 intersecting the first direction D1 and overlapping with the second character string image 132 in the first direction D1, acquiring the character string distance dc, correcting, based on the character string distance dc, the plurality of corrected coordinate data RD included in the corrected coordinate data group RDs to corrected coordinate data RD calculated using the plurality of correction data arranged in the second display region DR2, and generating a corrected coordinate data group RDs calculated using the correction data including the corrected coordinate data RD calculated using the correction data. Displaying the seventh corrected drawn image 127 includes displaying the seventh corrected drawn image 127 in the second display region DR2 based on the corrected coordinate data group RDs calculated using the correction data. The user can easily draw a shaped wavy line between the second character string image 132 and the third character string image 133.

[0144] The following is a summary of this disclosure.

[0145] Appendix 1 A display method including: acquiring a group of coordinate data including a plurality of coordinates indicating a handwritten wavy line drawn by a user; calculating a group of calculated coordinate data including a plurality of calculated coordinates indicating a shaped wavy line based on the group of coordinate data; and displaying the shaped wavy line based on the group of calculated coordinate data. The display device can display a shaped wavy line corresponding to the shape of a handwritten wavy line. The display device can display a plurality of types of wavy lines. This improves usability for users.

[0146] Appendix 2 A display method as described in Appendix 1, wherein the coordinate data group includes a partial coordinate data group including a plurality of partial image coordinates indicating partial images that are part of the handwritten wavy line, and includes determining a waveform of the shaped wavy line based on the partial coordinate data group, and displaying the shaped wavy line includes displaying the shaped wavy line having the determined waveform. The display device can display a shaped wavy line using a group of partial coordinate data indicating a partial image that is a part of a handwritten wavy line.

[0147] Appendix 3 In the display method according to Supplementary Note 1 or Supplementary Note 2, displaying the shaped wavy line includes displaying the shaped wavy line in a display mode different from that of the handwritten wavy line. Users can more easily distinguish between shaped wavy lines and handwritten wavy lines.

[0148] Appendix 4 In the display method described in Supplementary Note 2 or Supplementary Note 3, displaying the shaped wavy line includes displaying the handwritten wavy line that is different from the partial image as the shaped wavy line having the waveform. When a user draws a portion of a handwritten wavy line in a desired shape, the display device can display the entire handwritten wavy line as a shaped wavy line corresponding to the desired shape.

[0149] Appendix 5 In a display method described in any one of Supplementary Notes 1 to 4, displaying the shaped wavy line includes changing the waveform of the shaped wavy line and displaying it when a predetermined operation is performed by the user on the shaped wavy line. The user can edit the wavy line to shape it to the desired waveform.

[0150] Appendix 6 A display method according to any one of Supplementary Notes 1 to 5, comprising calculating, based on the coordinate data group, a first coordinate data group that displays the shaped wavy line of a first waveform, or a second coordinate data group that displays the shaped wavy line of a second waveform different from the first waveform, as the calculated coordinate data group. The display device can reduce the load when determining the waveform of the shaped waveform line.

[0151] Appendix 7 A display method according to any one of Supplementary Notes 1 to 6, comprising displaying a character string consisting of a plurality of characters before the handwritten wavy line is drawn, and displaying the shaped wavy line comprises displaying the shaped wavy line at a predetermined drawing position relative to the character string. The display device can display a shaped wavy line obtained by shaping a handwritten wavy line as an image that decorates a character string.

[0152] Appendix 8 In the display method described in Appendix 7, displaying the shaped wavy line includes, when the character string width of the character string and the image width of the handwritten wavy line differ in the arrangement direction of the multiple characters, displaying the shaped wavy line within the range of the character string width in the arrangement direction. The display device can adjust the width of the shaped wavy line in accordance with the character string width of the character string.

[0153] Appendix 9 A display method according to any one of Supplementary Notes 1 to 6, comprising: before drawing the handwritten wavy line, displaying a first string having a plurality of characters arranged in a first direction, and a second string that is positioned a first distance away from the first string in a second direction intersecting the first direction and overlaps with the first string in the first direction; acquiring the first distance; correcting, based on the first distance, a plurality of calculated coordinates included in the calculated coordinate data group to a plurality of corrected coordinates that are positioned in an area between the first string and the second string in the second direction; and generating a corrected calculated coordinate data group including the corrected coordinates; and displaying the shaped wavy line comprises displaying the shaped wavy line in the area between the first string and the second string in the second direction based on the corrected calculated coordinate data group. The user can easily draw a shaped wavy line between the first string and the second string.

[0154] Appendix 10 A display device comprising a display unit and a control unit, wherein the control unit acquires a group of coordinate data including a plurality of coordinates indicating a handwritten wavy line drawn by a user, calculates a group of calculated coordinate data including a plurality of calculated coordinates indicating a shaped wavy line based on the group of coordinate data, and displays the shaped wavy line on the display unit based on the group of calculated coordinate data. The display device can display a shaped wavy line corresponding to the shape of a handwritten wavy line. The display device can display a plurality of types of wavy lines. [Explanation of symbols]

[0155] 1...projector, 2...display screen, 2S...display surface, 5...operation pen, 11...remote control light receiving unit, 13...signal light transmitting unit, 15...photographing unit, 17...remote control unit, 20...wireless communication unit, 30...image projection device, 31...light source, 33...light modulation device, 35...optical unit, 40...image generating device, 41...image processing unit, 43...OSD synthesis unit, 45...frame memory, 50...control device, 60...storage unit, 70...control processor, 71...communication control unit, 73...drawing control unit, 75...display control unit, 100...display system, 110...handwritten image, 111...first handwritten image, 112...second 2. Handwritten image, 112a...partial handwritten image, 114...fourth handwritten image, 115...fifth handwritten image, 120...corrected image, 121...first corrected image, 122...second corrected image, 123...third corrected image, 124...fourth corrected image, 125...fifth corrected image, 126...sixth corrected image, 127...seventh corrected image, 130...character string image, 131...first character string image, 132...second character string image, 133...third character string image, AD...icon display area, CA...color specification icon group, CD...character string direction, CP...control program, CW...character string image width , CW1...first character string image width, CW2...second character string image width, CW3...third character string image width, D1...first direction, D2...second direction, dc...distance between character strings, DA...drawing area, DR...display area, DR1...first display area, DR2...second display area, dt...detection distance, LA...detection area, LT...detected light, MW...handwritten image width, MW4...fourth handwritten image width, OA...operation area, pPDs...partial designated coordinate data group, PD...designated coordinate data, PD1...first designated coordinate data, PD2...second designated coordinate data, PD3...third designated coordinate data, PDk...kth designated coordinate data, PDn...nth designated coordinate Data, PDs...pointed coordinate data group, PG...projected image, R1...first area, R2...second area, RA1...first corrected image amplitude, RA2...second corrected image amplitude, RD...corrected coordinate data, RD1...first corrected coordinate data, RD2...second corrected coordinate data, RD3...third corrected coordinate data, RDk...kth corrected coordinate data, RDk+1...k+1th corrected coordinate data, RDm...mth corrected coordinate data, RDs...corrected coordinate data group, RL1...first corrected image wavelength, RL2...second corrected image wavelength, RW...corrected image width, RW1...first corrected image width, RW2...second corrected image width, RW6...sixth corrected image width,SA1...first set amplitude, SA2...second set amplitude, SA3...third set amplitude, SF...set waveform, SF1...first set waveform, SF2...second set waveform, SF3...third set waveform, SR...set region, SR1...first set region, SR2...second set region, SWL1...first set wavelength, SWL2...second set wavelength, SWL3...third set wavelength, VS...virtual surface.

Claims

1. acquiring a group of coordinate data including a plurality of coordinates indicating a handwritten wavy line drawn by a user; calculating a calculated coordinate data group including a plurality of calculated coordinates indicating a shaped wavy line based on the coordinate data group; displaying the shaped wavy line based on the calculated coordinate data group; Display methods including.

2. the coordinate data group includes a partial coordinate data group including a plurality of partial image coordinates indicating a partial image that is a part of the handwritten wavy line, determining a waveform of the shaped wave line based on the group of partial coordinate data; displaying the shaped wavy line includes displaying the shaped wavy line having the determined waveform; The display method of claim 1 , comprising:

3. Displaying the shaped wavy line includes displaying the shaped wavy line in a display mode different from that of the handwritten wavy line; The display method of claim 1 , comprising:

4. displaying the shaped wavy line includes displaying the handwritten wavy line different from the partial image as the shaped wavy line having the waveform; The display method according to claim 2 , comprising:

5. displaying the shaped wavy line includes changing and displaying the waveform of the shaped wavy line when a predetermined operation is performed by the user on the shaped wavy line; The display method of claim 1 , comprising:

6. calculating, based on the coordinate data group, a first coordinate data group for displaying the shaped wave line of a first waveform, or a second coordinate data group for displaying the shaped wave line of a second waveform different from the first waveform, as the calculated coordinate data group; The display method of claim 1 , comprising:

7. displaying a string of characters arranged in a row before the handwritten wavy line is drawn; displaying the shaped wavy line includes displaying the shaped wavy line at a predetermined drawing position relative to the character string; The display method of claim 1 , comprising:

8. When a character string width of the character string and an image width of the handwritten wavy line are different in an arrangement direction of the plurality of characters, the display of the shaped wavy line includes displaying the shaped wavy line within a range of the character string width in the arrangement direction; The display method according to claim 7, comprising:

9. displaying, before the handwritten wavy line is drawn, a first character string having a plurality of characters arranged in a first direction, and a second character string that is arranged at a position a first distance from the first character string in a second direction intersecting the first direction and that overlaps with the first character string in the first direction; obtaining the first distance; correcting the calculated coordinates included in the calculated coordinate data group to a plurality of corrected coordinates located within a region between the first character string and the second character string in the second direction, based on the first distance; generating a corrected calculated coordinate data group including the corrected coordinates; Including, displaying the shaped wavy line includes displaying the shaped wavy line in the area between the first character string and the second character string in the second direction based on the corrected calculation coordinate data group; The display method of claim 1 , comprising:

10. A display unit; a control unit, The control unit acquiring a group of coordinate data including a plurality of coordinates indicating a handwritten wavy line drawn by a user; calculating a calculated coordinate data group including a plurality of calculated coordinates indicating a shaped wavy line based on the coordinate data group; and displaying the shaped wavy line on the display unit based on the calculated coordinate data group. Display device.

Images