Information processing device, projection method, and program
The information processing device addresses darkening issues in calligraphy e-learning by correcting color data in overlapping areas, ensuring clear projection of characters and correction lines, thereby improving learning effectiveness.
Patent Information
- Application Number
- JP2022081179
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-05-17
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-05-17
AI Technical Summary
Conventional image projection techniques in calligraphy e-learning result in darkening of instructor's correction lines, reducing visibility and reproducibility of correction results, hindering effective learning.
An information processing device that corrects color data of overlapping areas in projected images to enhance visibility and reproducibility by adjusting RGB values based on non-overlapping areas, ensuring characters and correction lines are clearly projected.
Improves visibility of correction results and reproducibility, enhancing the learning effect of calligraphy e-learning by minimizing darkening and maintaining clarity of instructor's feedback.
Smart Images

Figure 0007593373000001 
Figure 0007593373000002 
Figure 0007593373000003
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an information processing device, a projection method, and a program. [Background technology]
[0002] Conventionally, techniques related to image projection are known. For example, Patent Document 1 discloses a technique for comparing characters written on a piece of writing paper with reference information and projecting a portion to be corrected onto the piece of writing paper. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2005-293290 A Summary of the Invention [Problem to be solved by the invention]
[0004] In e-learning of calligraphy using actual calligraphy tools (writing paper, brush, inkstone, etc.), the brightness or saturation of the color of the instructor's lines (correction lines showing areas for improvement) decreases and the color tone becomes darker, so-called darkening, in areas where the student's characters overlap with the instructor's lines (correction lines showing areas for improvement) twice in total: once when the instructor captures the correction results and once when the correction results are projected by the student. When darkening occurs, the visibility of the areas for improvement and the reproducibility of the correction results on the student's side decrease, making it difficult to improve the learning effect of e-learning of calligraphy. However, these problems are not taken into consideration in conventional image projection. Therefore, there is room for improvement in image projection technology.
[0005] In view of the above circumstances, an object of the present disclosure is to improve techniques relating to image projection. [Means for solving the problem]
[0006] An information processing device according to an embodiment of the present disclosure includes: An information processing device including a control unit, The control unit is Obtaining a first image of a first medium including a first region having text thereon; projecting the first image onto a second medium at a predetermined location; obtaining a second image of the second medium including a second area with lines of a different color than the characters shown in the projected first image; correcting color data of an overlapping area where the first area and the second area on the second image overlap; The corrected second image is projected onto the first medium so that the positions of the characters coincide.
[0007] A projection method according to an embodiment of the present disclosure includes: A projection method executed by an information processing device, comprising: Obtaining a first image of a first medium including a first area having text thereon; projecting the first image onto a second medium at a predetermined location; obtaining a second image of the second medium including a second area with lines of a different color than the text shown in the first projected image; correcting color data of an overlapping area where the first area and the second area on the second image overlap; and projecting the corrected second image onto the first medium so that the positions of the characters coincide with each other; Includes.
[0008] A program according to an embodiment of the present disclosure includes: In the information processing device, Obtaining a first image of a first medium including a first area having text thereon; projecting the first image onto a second medium at a predetermined location; obtaining a second image of the second medium including a second area with lines of a different color than the text shown in the first projected image; correcting color data of an overlapping area where the first area and the second area on the second image overlap; projecting the corrected second image onto the first medium so that the positions of the characters coincide with each other; Execute the command. Effect of the Invention
[0009] According to an embodiment of the present disclosure, techniques relating to image projection are improved. [Brief description of the drawings]
[0010] [Figure 1] 1 is a block diagram showing a schematic configuration of a system according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a block diagram showing a schematic configuration of a first image processing device. [Diagram 3] FIG. 2 is a block diagram showing a schematic configuration of a second image processing device. [Figure 4] 1 is a block diagram showing a schematic configuration of an information processing device; [Diagram 5] 4 is a flowchart showing an operation of the information processing device. [Figure 6] FIG. 11 is a diagram showing a specific example of a first image. [Figure 7] FIG. 13 is a diagram showing a specific example of a second image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] Hereinafter, embodiments of the present disclosure will be described.
[0012] <Overview of the embodiment> An overview of a system 1 according to an embodiment of the present disclosure will be described with reference to Fig. 1. The system 1 includes a first image processing device 10, a second image processing device 20, and an information processing device 30. The first image processing device 10, the second image processing device 20, and the information processing device 30 are communicatively connected to a network 40 including, for example, the Internet and a mobile communication network.
[0013] In this embodiment, the first image processing device 10 is a device equipped with an imaging unit and a projection unit (for example, a projector), but may be any image processing device. The projector may be any type of projector, such as an LCD (liquid crystal display) projector, an LCOS (liquid crystal on silicon) projector, or a DLP (digital light processing) projector. A projector generally projects an image by projecting three primary colors of red (R), green (G), and blue (B) light modulated based on an image signal onto a projection surface (a piece of writing paper, a wall, a screen, etc.). Note that the image may be projected by two-dimensionally scanning the laser light of each color by an optical scanning means. In this embodiment, the first image processing device 10 is installed above the head of a calligraphy student (hereinafter also referred to as a "first user"), but is not limited to this and may be installed at any position. The number of first image processing devices 10 provided in the system 1 may be determined arbitrarily.
[0014] In this embodiment, the second image processing device 20 is a device equipped with an imaging unit and a projection unit (for example, a projector), but may be any image processing device. The projector may be any type of projector, such as an LCD projector, a DLP projector, or an LCOS projector. A projector generally projects an image by projecting three primary colors of red (R), green (G), and blue (B) light modulated based on an image signal onto a projection surface. Note that the image may be projected by two-dimensionally scanning the laser light of each color by an optical scanning means. In this embodiment, the second image processing device 20 is installed above the head of a calligraphy instructor (hereinafter also referred to as a "second user"), but may be installed at any position without being limited thereto. The number of second image processing devices 20 provided in the system 1 may be determined arbitrarily.
[0015] The information processing device 30 is, for example, a computer such as a server device, etc. The information processing device 30 is capable of communicating with the first image processing device 10 and the second image processing device 20 via a network 40.
[0016] First, an overview of this embodiment will be described, and details will be described later. The information processing device 30 acquires a first image of a first medium including a first region in which characters are written. The information processing device 30 projects the first image onto a predetermined position on a second medium. The information processing device 30 acquires a second image of the second medium including a second region in which lines of a different color are written from the characters shown in the projected first image. The information processing device 30 corrects color data of an overlapping region in which the first region and the second region on the second image overlap. The information processing device 30 projects the corrected second image onto the first medium so that the positions of the characters match.
[0017] Thus, according to this embodiment, the second image in which the color data of the overlapping area has been corrected is projected onto the first medium so that the positions of the characters match. Therefore, even if a darkening occurs in the overlapping area on the second image, for example, the second image in which the color data has been corrected so that the darkening is eliminated by a predetermined percentage can be projected onto the first medium. Therefore, the technology related to image projection is improved in that the visibility of the improvements on the first user's side and the reproducibility of the correction results are improved, making it easier to improve the learning effect of e-learning calligraphy.
[0018] Next, each component of the system 1 will be described in detail.
[0019] <Configuration of the first image processing device> As shown in FIG. 2, the first image processing device 10 includes a communication unit 11, an imaging unit 12, a projection unit 13, a detection unit 14, an output unit 15, an input unit 16, a memory unit 17, and a control unit 18.
[0020] The communication unit 11 includes one or more communication interfaces connected to a network 40. The communication interfaces correspond to mobile communication standards such as, but are not limited to, 4G (4th Generation) or 5G (5th Generation). In this embodiment, the first image processing device 10 communicates with the second image processing device 20 and the information processing device 30 via the communication unit 11 and the network 40.
[0021] The imaging unit 12 includes one or more cameras. Each camera included in the imaging unit 12 is a so-called digital camera having an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The imaging unit 12 captures an image of a subject at a predetermined frame rate and outputs image data of the captured image obtained by the imaging. The image data obtained via the imaging unit 12 can be used as frame images to form a moving image. In one embodiment, the imaging unit 12 acquires an image of a first medium.
[0022] The projection unit 13 includes an image display element, a light source, and a projection lens. For example, when the first image processing device 10 is an LCD or LCOS type projector, a liquid crystal is used as the image display element, and when the first image processing device 10 is a DLP type projector, a DMD (Digital Micromirror Device) is used. The light source irradiates the image display element with projection light, and projects an image on the projection surface through the projection unit 13. The light source may also irradiate illumination light in addition to the projection light. The irradiation range of the projection light and the irradiation range of the illumination light may be the same or different. The projection lens displays the image of the image display element on the projection surface by enlarging or reducing it at an arbitrary magnification. In one embodiment, the projection unit 13 projects an image onto the first medium.
[0023] The detection unit 14 includes any sensor module capable of acquiring information indicating the state of a projection surface onto which an image is projected from the projection unit 13. For example, the state of the projection surface includes the distance between the projection surface and the projection unit 13, the displacement of the projection surface, or a combination thereof. For example, the sensor module includes a laser sensor, a photoelectric sensor, an ultrasonic sensor, or a combination thereof.
[0024] The output unit 15 includes at least one output interface. The output interface is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL (Electro Luminescence) display. The output unit 15 outputs data obtained by the operation of the first image processing device 10. The output unit 15 may be connected to the first image processing device 10 as an external output device instead of being provided in the first image processing device 10. As the connection interface, for example, an interface compatible with standards such as USB (Universal Serial Bus) or Bluetooth (registered trademark) can be used.
[0025] The input unit 16 includes at least one input interface. The input interface is, for example, a physical key, a capacitive key, a pointing device, a touch screen integral with a display, or a microphone. The input unit 16 accepts an operation for inputting data used in the operation of the first image processing device 10. The input unit 16 may be connected to the first image processing device 10 as an external input device instead of being provided in the first image processing device 10. As the connection interface, for example, an interface compatible with a standard such as USB or Bluetooth (registered trademark) can be used.
[0026] The storage unit 17 includes one or more memories. The memories are, for example, semiconductor memories, magnetic memories, optical memories, etc., but are not limited thereto. Each memory included in the storage unit 17 may function, for example, as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 17 stores any information used in the operation of the first image processing device 10. For example, the storage unit 17 may store a system program, an application program, and embedded software, etc. The information stored in the storage unit 17 may be updateable with information acquired from the network 40 via the communication unit 11, for example. In this embodiment, the storage unit 17 can store an image or a moving image captured via the imaging unit 12 and information acquired via the detection unit 14.
[0027] The control unit 18 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination of these. The processor is, for example, but is not limited to, a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process. The programmable circuit is, for example, but is not limited to, an FPGA (Field-Programmable Gate Array). The dedicated circuit is, for example, but is not limited to, an ASIC (Application Specific Integrated Circuit). The control unit 18 controls the operation of the entire first image processing device 10.
[0028] <Configuration of the second image processing device> As shown in FIG. 3, the second image processing device 20 includes a communication unit 21, an imaging unit 22, a projection unit 23, a detection unit 24, an output unit 25, an input unit 26, a storage unit 27, and a control unit .
[0029] The communication unit 21 includes one or more communication interfaces connected to the network 40. The communication interfaces correspond to mobile communication standards such as, but are not limited to, 4G or 5G. In this embodiment, the second image processing device 20 communicates with the first image processing device 10 and the information processing device 30 via the communication unit 21 and the network 40.
[0030] The imaging unit 22 includes one or more cameras. Each camera included in the imaging unit 22 is a so-called digital camera having an imaging element such as a CCD or CMOS image sensor. The imaging unit 22 captures an image of a subject at a predetermined frame rate and outputs image data of the captured image obtained by the imaging. The image data obtained via the imaging unit 22 can also be used to configure a moving image as frame images. In one embodiment, the imaging unit 22 acquires an image of the second medium.
[0031] The projection unit 23 includes an image display element, a light source, and a projection lens. For example, when the second image processing device 20 is an LCD or LCOS type projector, a liquid crystal is used as the image display element, and when the second image processing device 20 is a DLP type projector, a DMD is used. The light source irradiates the image display element with projection light, and projects an image on the projection surface through the projection unit 23. The light source may irradiate illumination light in addition to the projection light. The irradiation range of the projection light and the irradiation range of the illumination light may be the same or different. The projection lens displays the image of the image display element on the projection surface by enlarging or reducing it at an arbitrary magnification. In one embodiment, the projection unit 23 projects an image onto the second medium.
[0032] The detection unit 24 includes any sensor module capable of acquiring information indicating the state of the projection surface onto which the image is projected from the projection unit 23. For example, the state of the projection surface includes the distance between the projection surface and the projection unit 23, the displacement of the projection surface, or a combination thereof. For example, the sensor module includes a laser sensor, a photoelectric sensor, an ultrasonic sensor, or a combination thereof.
[0033] The output unit 25 includes at least one output interface. The output interface is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL display. The output unit 25 outputs data obtained by the operation of the second image processing device 20. The output unit 25 may be connected to the second image processing device 20 as an external output device instead of being provided in the second image processing device 20. As the connection interface, for example, an interface compatible with standards such as USB or Bluetooth (registered trademark) can be used.
[0034] The input unit 26 includes at least one input interface. The input interface is, for example, a physical key, a capacitive key, a pointing device, a touch screen integral with a display, or a microphone. The input unit 26 accepts an operation for inputting data used in the operation of the second image processing device 20. The input unit 26 may be connected to the second image processing device 20 as an external input device instead of being provided in the second image processing device 20. As the connection interface, for example, an interface compatible with a standard such as USB or Bluetooth (registered trademark) can be used.
[0035] The storage unit 27 includes one or more memories. The memories are, for example, but not limited to, semiconductor memories, magnetic memories, or optical memories. Each memory included in the storage unit 27 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 27 stores any information used in the operation of the second image processing device 20. For example, the storage unit 27 may store a system program, an application program, and embedded software. The information stored in the storage unit 27 may be updated with information acquired from the network 40 via the communication unit 21, for example. In this embodiment, the storage unit 27 can store an image or a moving image captured via the imaging unit 22 and information acquired via the detection unit 24.
[0036] The control unit 28 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination of these. The processor is, for example, but is not limited to, a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. The programmable circuit is, for example, but is not limited to, an FPGA. The dedicated circuit is, for example, but is not limited to, an ASIC. The control unit 28 controls the operation of the entire second image processing device 20.
[0037] <Configuration of information processing device> As shown in FIG. 4, the information processing device 30 includes a communication unit 31, a storage unit 32, and a control unit 33.
[0038] The communication unit 31 includes one or more communication interfaces connected to the network 40. The communication interface corresponds to, for example, a mobile communication standard, a wired LAN (Local Area Network) standard, or a wireless LAN standard, but is not limited to these and may correspond to any communication standard. In this embodiment, the information processing device 30 communicates with the first image processing device 10 and the second image processing device 20 via the communication unit 31 and the network 40.
[0039] The storage unit 32 includes one or more memories. Each memory included in the storage unit 32 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 32 stores any information used in the operation of the information processing device 30. For example, the storage unit 32 may store a system program, an application program, a database, map information, and the like. The information stored in the storage unit 32 may be updatable with information acquired from the network 40 via the communication unit 31, for example.
[0040] The control unit 33 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination of these. The control unit 33 controls the operation of the information processing device 30 as a whole.
[0041] <Operation flow of information processing device> The operation of the information processing device 30 according to this embodiment will be described with reference to Fig. 5. The operation of Fig. 5 corresponds to the projection method according to this embodiment. The operation of Fig. 5 is repeatedly executed at a predetermined cycle. In this embodiment, the operation of Fig. 5 is executed every time an expression of intention to submit a work, which will be described later, is detected. However, the predetermined cycle is not limited to this and can be determined arbitrarily.
[0042] In the following description, the term "assignment work" refers to a piece of writing paper on which characters are written by a first user using a brush with ink of a specific color on a white piece of writing paper (hereinafter also referred to as "first user's writing paper"). In this embodiment, the specific color is black, and therefore the color of the characters is black. However, the color of the characters is not limited to this, and any color can be adopted. The term "correction result" refers to a piece of writing paper on which a second user adds lines indicating improvements using a brush with ink of a color different from the color of the characters, over at least a part of the first region on which characters are written, within the region of the image showing the first user's writing paper projected onto the white piece of writing paper on the second user's side (hereinafter also referred to as "second user's writing paper"). In this embodiment, the color different from the color of the characters is vermilion, and therefore the color of the lines is vermilion. However, the color of the lines is not limited to this, and any color different from the color of the characters can be adopted.
[0043] In the following description, the first user and the second user each sit on a chair, not sitting upright on the floor, and write characters or lines with a brush dipped in ink of a different color on rectangular pieces of calligraphy paper (e.g., pulp calligraphy paper) of the same size (e.g., 334 mm long x 243 mm wide) placed on a stand (e.g., a desk). In addition, to prevent the stand from getting dirty, a rectangular felt writing pad of the same size (e.g., 360 mm long x 260 mm long) is placed between the stand and the calligraphy paper. However, the materials and sizes of the writing paper and the writing pad are not limited to these examples and can be selected arbitrarily.
[0044] Step S100: The control unit 33 of the information processing device 30 obtains a first image of a first medium including a first area on which text is written.
[0045] Specifically, the communication unit 31 of the information processing device 30 receives an image of the writing paper on the first user's side, including an area where characters are written by the first user, captured by the imaging unit 12, from the communication unit 11 of the first image processing device 10 via the network 40. The area where characters are written (the characters themselves written with ink) corresponds to the first area in this embodiment, the writing paper on the first user's side corresponds to the first medium in this embodiment, and the image of the writing paper corresponds to the first image in this embodiment. Note that the captured image of the writing paper captured by the imaging unit 12 may also include an image of the surroundings of the writing paper (for example, a writing pad or a placing table). In this case, it is desirable to extract an image of only the writing paper portion by performing image analysis on the received captured image, and store this as the first image (an image of the writing paper on the first user's side). The control unit 33 of the information processing device 30 stores the image of the writing paper on the first user's side received via the communication unit 31 in the storage unit 32. Then, the control unit 33 acquires the first image by reading out the image of the writing paper on the first user side from the storage unit 32 every time the operation of Fig. 5 is performed. For example, the control unit 33 may acquire an image S1 of the writing paper M1 on the first user side including an area R1 on which characters are written as shown in Fig. 6. The area R1 corresponds to the first area in this embodiment, the writing paper M1 on the first user side corresponds to the first medium in this embodiment, and the image S1 corresponds to the first image in this embodiment.
[0046] Moreover, the control unit 33 calculates the position of the first region in which the character is written on the first image. Specifically, the control unit 33 calculates a color feature value for each pixel of the first image, thereby extracting a group of pixels (hereinafter also referred to as a "first pixel group") corresponding to the first region from among the pixels included in the first image. In this embodiment, the color feature value is an R value, a G value, and a B value (hereinafter also referred to as an "RGB value") in an RGB color space, but is not limited to this and any value can be adopted. For example, the color feature value may be a hue H, a saturation S, and a brightness V in an HSV color space instead of or in addition to the RGB value. Any method can be adopted for extracting the first pixel group. For example, the first pixel group has an RGB value specific to the first pixel group, which is different from pixels corresponding to other regions. Therefore, the control unit 33 can extract the first pixel group from among the pixels included in the first image by comparing the RGB values of each pixel of the first image. The control unit 33 calculates two-dimensional coordinate values on the first image corresponding to each pixel included in the extracted first pixel group, associates the calculated two-dimensional coordinate values with each pixel, and stores them in the storage unit 32. Note that, in this embodiment, the information indicating the position of the first pixel group is calculated as two-dimensional coordinate values of the first image, but is not limited to this and any information can be used.
[0047] In this embodiment, the image of the writing paper on the first user's side can be captured at any timing. For example, when the control unit 33 of the information processing device 30 detects the first user's intention to submit the assignment work to the second user (hereinafter, also referred to as "intention to submit work"), the control unit 33 may transmit an instruction to capture an image of the writing paper on the first user's side to the communication unit 11 of the first image processing device 10. The control unit 18 of the first image processing device 10 may cause the imaging unit 22 to capture an image of the writing paper on the first user's side in response to the instruction received via the communication unit 11.
[0048] Any method may be used to detect the intention to submit a work. For example, the control unit 33 may determine that the intention to submit a work has been detected when a predetermined condition is satisfied. The predetermined condition may be determined arbitrarily, and the first and second examples are shown below as specific examples.
[0049] The first example is a case where the first user explicitly expresses his / her intention to submit a work. In this case, the predetermined condition may be that a predetermined operation by the first user has been detected. For example, the predetermined operation may be that the first user operates a touch screen included in the input unit 16 of the first image processing device 10 to touch an icon such as a button for submitting a work, or that the first user presses a physical key included in the input unit 16. When the predetermined operation is detected, the control unit 18 of the first image processing device 10 may transmit information indicating that the predetermined operation by the first user has been detected to the communication unit 31 of the information processing device 30 via the communication unit 11 and the network 40. When the information is received by the communication unit 31, the control unit 33 may determine that the predetermined condition has been satisfied and that an expression of intention to submit a work has been detected.
[0050] The second example is a case where the first user implicitly expresses his / her intention to submit a work. In this case, the predetermined condition may be that a predetermined behavior by the first user is detected. For example, the predetermined behavior may be that the first user places the brush in a predetermined position (for example, on an inkstone placed on a stand) and removes his / her hand from the brush. Any method may be adopted for detecting the predetermined behavior. For example, the imaging unit 12 of the first image processing device 10 may capture an image of the first user and the stand as subjects at a predetermined frame rate and output image data of the captured image obtained by the imaging. The control unit 18 of the first image processing device 10 may transmit the image data to the communication unit 31 of the information processing device 30 via the communication unit 11 and the network 40. The control unit 33 of the information processing device 30 may determine whether or not the behavior of the first user corresponds to the predetermined behavior by performing image analysis of the image data received by the communication unit 31. If the image analysis determines that the behavior of the first user corresponds to a predetermined behavior, the control unit 33 may determine that a predetermined condition has been satisfied, and that an expression of an intention to submit a work has been detected.
[0051] In this embodiment, the submission of the assigned work to the second user is performed by transmitting information indicating the assigned work from the first image processing device 10 to the information processing device 30 via the network 40, and then transferring the information from the information processing device 30 to the second image processing device 20. However, the submission of the assigned work to the second user is not limited to this, and any method can be adopted. For example, the information indicating the assigned work may be directly transmitted from the first image processing device 10 to both the information processing device 30 and the second image processing device 20.
[0052] Step S101: The control unit 33 projects the first image acquired in step S100 onto a predetermined position on the second medium. In this embodiment, the predetermined position on the second medium is a position on the second medium where the first image fits within the contour line of the second medium when projected, and is indicated as a two-dimensional coordinate value of the second medium. However, the predetermined position on the second medium is not limited to these examples, and any position and value can be adopted.
[0053] Specifically, the control unit 33 sets a reference point for alignment on the second medium, aligns the first image with the second medium based on the reference point, and projects the first image on the second medium, thereby projecting the first image at a predetermined position on the second medium. Any point on the second medium can be used as the reference point for alignment on the second medium. For example, the control unit 33 sets the four corners of the second medium (rectangular half-sized paper) as the reference points. In this case, the control unit 33 receives a captured image of the second medium from the second image processing device 20 via the communication unit 31 and the network 40, performs image analysis on the received captured image, and identifies the four corners of the second medium that will be the projection surface, and sets the identified four corners as the reference points. Then, the control unit 33 transmits the set reference point to the communication unit 21 of the second image processing device 20 via the communication unit 31 and the network 40. The control unit 28 of the second image processing device 20 causes the projection unit 23 to project the first image onto the second medium, while aligning the first image with the second medium so that the four corners of the projected first image fit within the frame of the reference points (i.e., the four corners of the second medium) received via the communication unit 21. For example, it is desirable to use the same size of writing paper for the first medium and perform processing such as enlarging, reducing, and tilting the first image as necessary so that the four corners of the first image match the four corners of the second medium, and project the first image onto the second medium. In this way, the image S1 shown in FIG. 6 is aligned to a position on the writing paper M2 so that it fits within the contour line of the writing paper M2 on the second user side as shown in FIG. 7 when projected. As a result, the first image fits within the contour line of the second medium when projected, and the first image is projected at a predetermined position on the second medium.
[0054] In this embodiment, the characters shown in the first image are black and do not emit light, so that it is difficult to project them clearly onto the second medium. Therefore, when projecting the first image onto the second medium, the control unit 33 preferably irradiates the entirety of the table on which the second medium is placed (i.e., the top plate of the table) with illumination light, avoiding the first region. Any method can be adopted for irradiating the illumination light avoiding the first region. For example, the control unit 33 reads out from the storage unit 32 two-dimensional coordinate values on the first image corresponding to each pixel included in the first pixel group. The control unit 33 controls so that the illumination light from the light source of the projection unit 23 is not irradiated onto the position indicated by the two-dimensional coordinate values read out from the storage unit 32 by restricting the illumination range, for example, by masking. Then, the control unit 33 irradiates the entirety of the table on which the second medium is placed with illumination light, while projecting the first image onto a predetermined position on the second medium as described above. This allows the illumination light to be irradiated onto the entirety of the table on which the second medium is placed, avoiding the first region. As a result, when the first image is projected onto the second medium, the region where the characters corresponding to the first region are written becomes darker than the other regions. Therefore, it becomes easier to project the characters shown in the first image onto the second medium clearly. In this embodiment, the illumination light is irradiated from the light source of the projection unit 23, but this is not limited to this. For example, the illumination light may be irradiated from an external light source separate from the projection unit 23. Here, if light is irradiated only onto the second medium placed on the stand instead of the entire stand on the second user side, the contrast ratio between the part corresponding to the first region on the second medium and the other parts will be insufficient, and the first region on the second medium will appear to be a lighter color (for example, light gray) than black, which may reduce the reproducibility of the first medium. For this reason, it is preferable to irradiate the illumination light onto the entire stand on which the second medium is placed.
[0055] Moreover, it is more preferable that the stand is shielded from light other than the projection light and illumination light for projecting the first image. Any method can be adopted for shielding the stand. For example, if the stand is a rectangular desk, the stand may be shielded from light by surrounding it in a U-shape with a light-shielding panel (for example, an opaque acrylic board), that is, by surrounding three sides except the side where the second user is seated. By shielding the stand from light other than the projection light and illumination light for projecting the first image, it becomes easier to increase the contrast ratio between the part corresponding to the first region of the stand and the remaining part. As a result, it becomes easier to project the characters shown in the first image more clearly onto the second medium.
[0056] Step S102: The control unit 33 obtains a second image of a second medium including a second area in which lines are drawn in a different color from the characters shown in the projected first image.
[0057] Specifically, the communication unit 31 of the information processing device 30 receives a set of images of the second user's writing paper captured by the imaging unit 22 from the communication unit 21 of the second image processing device 20 via the network 40. The set of images includes an image of the second user's writing paper onto which the first image is projected (hereinafter also referred to as an "image of characters"), an image of the second user's writing paper onto which a line is drawn by the second user in a state in which the first image is projected onto the second user's writing paper (hereinafter also referred to as an "image of the correction result"), and an image of the second user's writing paper onto which a line is drawn by the second user in a state in which the first image is not projected onto the second user's writing paper by interrupting the projection of the first image, etc. (hereinafter also referred to as an "image of the line"). The area onto which the line is drawn corresponds to the second area in this embodiment, the second user's writing paper corresponds to the second medium in this embodiment, and the set of images corresponds to the second image in this embodiment. Note that the captured image of the writing paper captured by the imaging unit 22 may also include an image of the surroundings of the writing paper (for example, the writing pad or the stand on which it is placed). In this case, it is desirable to perform image analysis of the received captured image to extract an image of only the writing paper portion and store this as a second image (a set of images including an image of the characters, an image of the correction result, and an image of the lines). The control unit 33 of the information processing device 30 links the set of images received via the communication unit 31 and stores them in the storage unit 32. Then, the control unit 33 acquires the second image by reading out the set of images from the storage unit 32 every time the operation of FIG. 5 is performed.
[0058] For example, the control unit 33 may acquire three types of images as a set of images: an image S2 (corresponding to an image of the correction result) of the writing paper M2 on the second user side including an area R1 with characters written, an area R2 with lines written, and an overlapping area R3 where the areas R1 and R2 overlap, an image of the writing paper M2 including only the area R1 (corresponding to an image of characters), and an image of the writing paper M2 including only the area R2 (corresponding to an image of lines). The area R2 (the red line itself) corresponds to the second area in this embodiment, the writing paper M2 on the second user side corresponds to the second medium in this embodiment, and the three types of images correspond to the second image consisting of a set of images in this embodiment. In FIG. 7, in order to make the drawing easier to understand, the area R1 is shown by a solid line and the area R2 is shown by a dashed line, and the shaded area corresponds to the overlapping area R3.
[0059] In this embodiment, the imaging of a set of images of the second user's writing paper can be performed at any timing. For example, for an "image of characters", the control unit 33 of the information processing device 30 may transmit an instruction to the communication unit 21 of the second image processing device 20 to image the writing paper with the first image projected onto it. In response to the instruction received via the communication unit 21, the control unit 28 of the second image processing device 20 may cause the imaging unit 22 to capture an image of the writing paper while continuing the projection of the first image onto the writing paper by the projection unit 23. This allows the "image of characters" to be captured.
[0060] Furthermore, regarding the "image of the correction result" and the "image of the line", when the control unit 33 detects an intention of the second user to notify the first user of the correction result (hereinafter also referred to as "intention to notify result"), it may transmit an instruction to capture an image of the writing paper on the second user's side to the communication unit 21 of the second image processing device 20. The control unit 28 of the second image processing device 20 may cause the imaging unit 22 to capture an image of the writing paper while continuing or interrupting the projection of the first image by the projection unit 23 onto the writing paper on the second user's side, in response to the instruction received via the communication unit 21. This allows the "image of the correction result" and the "image of the line" to be captured.
[0061] Any method can be used to detect the intention to notify the result. For example, the control unit 33 may determine that the intention to notify the result has been detected when a predetermined condition is satisfied. The predetermined condition can be determined arbitrarily, and the third and fourth examples are shown below as specific examples.
[0062] The third example is a case where the second user explicitly expresses his / her intention to notify the result. In this case, the predetermined condition may be that a predetermined operation by the second user is detected. For example, the predetermined operation may be that the second user operates a touch screen included in the input unit 26 of the second image processing device 20 to touch an icon such as a button for notifying the correction result, or that the second user presses a physical key included in the input unit 26. When the predetermined operation is detected, the control unit 28 of the second image processing device 20 may transmit information indicating that the predetermined operation by the second user has been detected to the communication unit 31 of the information processing device 30 via the communication unit 21 and the network 40. When the information is received by the communication unit 31, the control unit 33 may determine that the predetermined condition is satisfied and that an intention to notify the result has been detected.
[0063] The fourth example is a case where the second user implicitly expresses his / her intention to notify the result. In this case, the predetermined condition may be that a predetermined behavior by the second user is detected. For example, the predetermined behavior may be that the second user places the brush in a predetermined position (for example, on an inkstone placed on a stand) and removes his / her hand from the brush. Any method may be adopted for detecting the predetermined behavior. For example, the imaging unit 22 of the second image processing device 20 may capture an image of the second user and the stand as subjects at a predetermined frame rate and output image data of the captured image obtained by the imaging. The control unit 28 of the second image processing device 20 may transmit the image data to the communication unit 31 of the information processing device 30 via the communication unit 21 and the network 40. The control unit 33 of the information processing device 30 may determine whether or not the behavior of the second user corresponds to the predetermined behavior by performing image analysis of the image data received by the communication unit 31. When it is determined as a result of the image analysis that the behavior of the second user corresponds to a predetermined behavior, the control unit 33 may determine that a predetermined condition has been satisfied, and determine that an indication of an intention to notify the result has been detected.
[0064] In this embodiment, the first user is notified of the correction result by transmitting information indicating the correction result from the second image processing device 20 to the information processing device 30 via the network 40, and then the information processing device 30 transfers the information to the first image processing device 10. However, the method of notifying the first user of the correction result is not limited to this, and any method can be adopted. For example, the information indicating the correction result may be directly transmitted from the second image processing device 20 to both the information processing device 30 and the first image processing device 10.
[0065] Step S103: The control unit 33 determines whether or not there is an overlapping area on the second image acquired in step S102. The "overlapping area" refers to an area where the first area and the second area overlap. If it is determined that there is an overlapping area (step S103-Yes), the process proceeds to step S104. If it is determined that there is no overlapping area (step S103-No), the process proceeds to step S106.
[0066] Specifically, the control unit 33 calculates the positions of the first and second areas on the second image by performing the following image analysis on the second image consisting of a set of images (including an "image of characters," an "image of correction results," and an "image of lines") acquired in step S102.
[0067] The control unit 33 extracts a group of pixels (hereinafter, also referred to as a "second pixel group") corresponding to the first region from among the pixels included in the "image of the character". Any method can be adopted for extracting the second pixel group. For example, the second pixel group has an RGB value specific to the second pixel group, which is different from the pixels corresponding to other regions, similar to the first pixel group described above for the first image. Therefore, the control unit 33 can extract the second pixel group from among the pixels included in the image of the character by comparing the RGB values of each pixel of the image of the character. The control unit 33 calculates two-dimensional coordinate values on the image of the character corresponding to each pixel included in the extracted second pixel group, and stores the calculated two-dimensional coordinate values in the storage unit 32 in association with each pixel. Note that, in this embodiment, the information indicating the position of the second pixel group is calculated as a two-dimensional coordinate value of the image of the character, but is not limited thereto and any information can be adopted.
[0068] Moreover, the control unit 33 extracts a group of pixels (hereinafter also referred to as a "third pixel group") corresponding to the second region from among the pixels included in the "line image". Any method can be adopted for extracting the third pixel group. For example, the third pixel group has an RGB value specific to the third pixel group, which is different from pixels corresponding to other regions, similar to the above-mentioned first pixel group. Therefore, the control unit 33 can extract the third pixel group from among the pixels included in the line image by comparing the RGB values of each pixel in the line image. The control unit 33 calculates two-dimensional coordinate values on the line image corresponding to each pixel included in the extracted third pixel group, and stores the calculated two-dimensional coordinate values in the storage unit 32 in association with each pixel. Note that, in this embodiment, the information indicating the position of the third pixel group is calculated as a two-dimensional coordinate value of the line image, but is not limited thereto and any information can be adopted.
[0069] Then, the control unit 33 compares the two-dimensional coordinate values of the second pixel group and the third pixel group calculated in this way to determine whether or not there is an overlapping area on the "corrected image". In this embodiment, the positions indicated by the two-dimensional coordinate values on the character image, the two-dimensional coordinate values on the line image, and the two-dimensional coordinate values on the corrected image are equal to each other. Therefore, if the comparison results in the identification of at least one pixel having the same two-dimensional coordinate values, the control unit 33 can determine that there is an overlapping area on the second image. The control unit 33 calculates two-dimensional coordinate values on the corrected image corresponding to each pixel included in a pixel group (hereinafter also referred to as a "fourth pixel group") consisting of the identified at least one pixel, and stores the calculated two-dimensional coordinate values in the storage unit 32 in association with each pixel. The fourth pixel group corresponds to the overlapping area in this embodiment. In the example shown in FIG. 7, the control unit 33 can identify an overlapping area R3 (shaded portion) where the area R1 and the area R2 overlap as an overlapping area, and determine that there is an overlapping area on the second image. In this embodiment, the information indicating the position of the fourth pixel group is calculated as two-dimensional coordinate values of the corrected image, but the information is not limited to this and any arbitrary information can be used.
[0070] Step S104: If it is determined that there is an overlapping area (step S103-Yes), the control unit 33 corrects the color data of the overlapping area.
[0071] Specifically, the control unit 33 corrects the RGB values corresponding to the color of the overlapping region identified in step S103 based on the RGB values corresponding to the color of the lines in the region on the second image where the first region and the second region do not overlap. In the example shown in Fig. 7, the control unit 33 may correct the RGB values corresponding to the color of the overlapping region R3 based on the RGB values corresponding to the color of the lines in the region on the image S2 where the regions R1 and R2 do not overlap. The image in which the RGB values corresponding to the color of the overlapping region R3 in the second image (here, the "image resulting from the correction") have been corrected is referred to as the "corrected second image."
[0072] Any method can be used to correct the RGB values. For example, the control unit 33 selects one pixel from the pixels of the third pixel group that is not included in the fourth pixel group. The one pixel is a pixel of the third pixel group that is not affected by the darkening, and holds an RGB value corresponding to the original color of the third pixel group. The control unit 33 reads out the RGB value corresponding to the one selected pixel from the storage unit 32. The control unit 33 calculates the RGB value corresponding to the color of each pixel of the fourth pixel group. The control unit 33 calculates the difference between the RGB value corresponding to the one selected pixel and the calculated RGB value corresponding to the color of each pixel of the fourth pixel group. Based on the calculated difference, the control unit 33 can correct the RGB value corresponding to the color of each pixel of the fourth pixel group so that it approaches the RGB value corresponding to the one selected pixel by a predetermined ratio. The predetermined ratio can be arbitrarily determined for each pixel. For example, the smaller the predetermined ratio is (i.e., the closer to 0%), the more the difference is maintained, and the easier it is to maintain the appearance of the overlapping area. On the other hand, the larger the predetermined percentage is (i.e., the closer to 100%), the smaller the difference will be, and the closer the third pixel group in the overlapping area will be to the original color, making it easier to improve the visibility of the improvements. Therefore, the predetermined percentage can be set by taking into consideration the balance between the degree to which the appearance of the correction result is maintained and the visibility of the improvements.
[0073] Furthermore, the control unit 33 may correct the RGB values corresponding to the color of the overlapping region based on the RGB values corresponding to the color of the characters in the first region, in addition to the RGB values corresponding to the color of the lines in the region where the first region and the second region do not overlap on the second image. Specifically, the control unit 33 may set, as the predetermined value, an increase rate that increases a part or all of the R value, the G value, and the B value by a predetermined ratio according to the RGB value corresponding to the color of the characters in the first region. For example, when the color of the characters in the first region is black as in this embodiment, the R value, the G value, and the B value corresponding to the color of each pixel corresponding to the fourth pixel group all decrease. In this case, the control unit 33 may set, as the predetermined value, an increase rate (e.g., +10%) that increases all of the R value, the G value, and the B value by a predetermined ratio. The increase rate can be set individually for each of the R value, the G value, and the B value according to the RGB value corresponding to the color of the characters in the first region.
[0074] In addition, in this embodiment, the predetermined value is set in consideration of the effect of darkening that occurs when the correction result is projected on the first image processing device 10 side. This makes it possible to mitigate the effect of darkening that occurs twice in total, once when the correction result is captured on the second image processing device 20 side and once when the correction result is projected on the first image processing device 10 side, to a desired extent. Therefore, the visibility of the improvement points and the reproducibility of the correction result on the first user side are improved, making it easier to further improve the learning effect of e-learning calligraphy. However, the correction of the color data of the overlapping area is not limited to these examples and any method can be adopted.
[0075] Step S105: The control unit 33 projects the corrected second image onto the first medium so that the positions of the characters match.
[0076] Specifically, the control unit 33 identifies a position on the first medium where the contour of the second pixel group (corresponding to the first region) included in the corrected second image matches the contour of the character written on the first medium (the character itself written in ink) at the time of projection. In this embodiment, the position on the first medium is indicated as a two-dimensional coordinate value of the first medium, but is not limited to this and any value can be adopted. Any method can be adopted to identify the position on the first medium. For example, the control unit 33 reads the first image from the storage unit 32 and calculates the area of the character written on the first medium that occupies the first medium. The control unit 33 reads the second image from the storage unit 32 and calculates the area of the second pixel group that occupies the second image. The control unit 33 enlarges or reduces the second image while maintaining the aspect ratio until the area of the second pixel group matches the area of the character written on the first medium, which is the projection surface. The control unit 33 calculates the enlargement or reduction magnification at this time as the projection magnification. At this time, the control unit 33 also calculates values (hereinafter, also referred to as "other adjustment values") to be used in other processes such as tilt correction as necessary. The control unit 33 transmits the calculated projection magnification and / or other adjustment values to the communication unit 11 of the first image processing device 10 via the communication unit 31 and the network 40. The control unit 18 of the first image processing device 10 causes the projection unit 13 to project the second image onto the first medium according to the projection magnification and / or other adjustment values received via the communication unit 11. Then, the control unit 18 aligns the contour of the second pixel group included in the second image so as to overlap with the contour of the character written on the first medium without deviation. For example, the contour of the image S2 shown in FIG. 7, in which the color data has been corrected, is aligned so as to overlap with the contour of the region R1 corresponding to the character written on the writing paper M1 on the first user side shown in FIG. 6 without deviation. As a result, the corrected second image is projected onto the first medium so that the position of the character matches. However, the projection of the second image on the first medium is not limited to these examples, and any method can be adopted.
[0077] Step S106: If it is determined that there is no overlapping area (step S103-No), the control unit 33 projects the second image onto the first medium so that the position of the characters coincides, without correcting the color data of the second image. The projection method of the second image is the same as that described above in step S105, except that the color data is not corrected, and therefore the description is omitted.
[0078] As described above, the information processing device 30 according to this embodiment acquires a first image of a first medium including a first region in which characters are written. The information processing device 30 projects the first image onto a predetermined position on a second medium. The information processing device 30 acquires a second image of the second medium including a second region in which lines of a different color are written than the characters shown in the projected first image. The information processing device 30 corrects color data of an overlapping region in which the first region and the second region on the second image overlap. The information processing device 30 projects the corrected second image onto the first medium so that the positions of the characters match.
[0079] According to this configuration, the second image with the color data of the overlapping area corrected is projected onto the first medium so that the positions of the characters match. Therefore, even if a darkening occurs in the overlapping area on the second image, the second image with the color data corrected so that the darkening is eliminated by a predetermined percentage can be projected onto the first medium. This improves the visibility of the improvements on the first user's side and the reproducibility of the correction results, and improves the technology related to image projection in that it is easier to improve the learning effect of e-learning calligraphy.
[0080] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art may make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each component or step can be rearranged so as not to be logically inconsistent, and multiple components or steps can be combined or divided into one.
[0081] For example, in the above-described embodiment, the configuration and operation of the information processing device 30 may be distributed among a plurality of computers that can communicate with each other. Also, for example, an embodiment in which some or all of the components of the information processing device 30 are provided in the first image processing device 10 and the second image processing device 20 may be provided.
[0082] Also, for example, in the above embodiment, after the first user completes the assignment, a first image (still image) is captured, the first image is projected on the second user side, a second image (still image) is captured after the first image is projected on the second user side and after the correction result is completed, and the corrected second image is projected on the first user side. In contrast, in a first modification of this embodiment, moving image data showing the state (e.g., appearance) of each other's writing paper that changes over time may be transmitted and received in real time between the first and second users, and projected on each other's writing paper in real time. In the first modification, moving image data of each other's writing paper is transmitted and received between the first and second users, and the moving image is projected on each other's writing paper in real time.
[0083] Specifically, in the first modified example, the first image is a first image of a predetermined number of frames, and the second image is a second image of a predetermined number of frames. In the first modified example, the information processing device 30 acquires a first image of a first medium including a first region for a predetermined number of frames. The information processing device 30 projects the first image of the predetermined number of frames onto a predetermined position on the second medium. The information processing device 30 acquires a second image of a second medium including a second region for a predetermined number of frames. The information processing device 30 corrects color data of an overlapping region where the first region and the second region on the second image overlap. The information processing device 30 projects the corrected second image onto the first medium so that the positions of the characters match.
[0084] The first image of the predetermined number of frames may be a first image (first video) captured by the imaging unit 12 of the first image processing device 10 at a predetermined frame rate for a first period. For example, the first period may be a period from when a start event (for example, the first user pressing an icon such as a button or a physical key included in the input unit 16, or picking up a brush on an inkstone) on the first image processing device 10 side is detected to when an end event (for example, an expression of intention to submit a work) is detected. The detection of the start event and the end event can be performed in the same manner as in the first and second examples described above. Alternatively, the first period may be a period from the start time to the end time of a lesson. The control unit 18 of the first image processing device 10 may transmit the first image of the predetermined number of frames in real time to the second image processing device 20 via the information processing device 30, or directly to both the information processing device 30 and the second image processing device 20. On the second image processing device 20 side, the state of the writing paper on the first user side that changes over time can be displayed in real time by projecting the received first image of the predetermined number of frames onto a second medium as a moving image.
[0085] The second image of the predetermined number of frames may be a second image (second video) captured by the imaging unit 22 of the second image processing device 20 at a predetermined frame rate for a second period. For example, the second period may be a period from the time when the "image of characters" is captured on the second image processing device 20 side to the time when a start event (for example, the second user pressing an icon such as a button or a physical key included in the input unit 26, or picking up a brush on an inkstone) on the second image processing device 20 side is detected and an end event (for example, an indication of intention to notify the result) is detected. The detection of the start event and the end event can be performed in the same manner as in the third and fourth examples described above. Alternatively, the second period may be a period from the start time to the end time of a lesson. The control unit 28 of the second image processing device 20 may transmit the second image of the predetermined number of frames in real time to the first image processing device 10 via the information processing device 30, or directly to both the information processing device 30 and the first image processing device 10. On the first image processing device 10 side, the received second image of a predetermined number of frames is projected onto the first medium as a moving image, so that the state of the writing paper on the second user side that changes over time can be displayed in real time.
[0086] According to the first modification, the first user and the second user can check not only each other's work results but also the work process (for example, the stroke order or the brush strokes, etc.). As a result, it becomes easier to create a sense of realism in e-learning of calligraphy that is equivalent to a face-to-face class.
[0087] For example, in a second modified example of the present embodiment, in addition to the first modified example, a predetermined number of frames of image data in which the first user and the second user are each imaged from the front may be acquired and transmitted in real time between the first image processing device 10 and the second image processing device 20 indirectly or directly via the information processing device 30. The first image processing device 10 and the second image processing device 20 may display each other's appearance in real time on panels such as liquid crystal displays located in front of the first user and the second user, respectively.
[0088] Specifically, in the second modified example, the control unit 33 of the information processing device 30 further acquires a third image of a predetermined number of frames and a fourth image of a predetermined number of frames, which are captured from the front of the first user writing characters on the first medium and the second user writing lines on the second medium, and displays the appearances of the first user and the second user in real time on a panel (e.g., a display device such as a liquid crystal display) arranged in front of each of them based on the third image of the predetermined number of frames and the fourth image of the predetermined number of frames. For example, the first user and the second user may be captured via an external imaging device such as a camera installed on a panel arranged in front of each of them.
[0089] The third image of the predetermined number of frames may be a third image (third video) captured by the imaging unit 12 of the first image processing device 10 at a predetermined frame rate for a third period. For example, the third period may be the same as the first period, or may be different. For example, if the first period is a period from the time when a start event is detected on the first image processing device 10 side to the time when an end event is detected, the third period may be a period from the start time to the end time of a lesson. The control unit 18 of the first image processing device 10 may transmit the third image of the predetermined number of frames in real time to the second image processing device 20 via the information processing device 30, or directly to both the information processing device 30 and the second image processing device 20. On the second image processing device 20 side, the received third image of the predetermined number of frames is displayed as a moving image on a panel arranged in front of the second user, so that the state of the first user that changes over time can be displayed in real time.
[0090] The fourth image of the predetermined number of frames may be a fourth image (fourth video) captured by the imaging unit 22 of the second image processing device 20 at a predetermined frame rate for a fourth period. For example, the fourth period may be the same as the second period, or may be different. For example, if the second period is a period from the time when the "image of characters" is captured on the second image processing device 20 side to the time when a start event and an end event are detected, the fourth period may be a period from the start time to the end time of a lesson. The control unit 28 of the second image processing device 20 may transmit the fourth image of the predetermined number of frames in real time to the first image processing device 10 via the information processing device 30, or directly to both the information processing device 30 and the first image processing device 10. On the first image processing device 10 side, the received fourth image of the predetermined number of frames is displayed as a moving image on a panel arranged in front of the first user, so that the state of the second user that changes over time can be displayed in real time.
[0091] According to the second modification, the first user and the second user can confirm additional information such as each other's facial expressions or postures. As a result, it becomes easier to create a sense of realism in e-learning of calligraphy that is equivalent to that of a face-to-face class.
[0092] Also, for example, an embodiment is possible in which a general-purpose computer functions as the information processing device 30 according to the above-described embodiment. Specifically, a program describing the processing contents for realizing each function of the information processing device 30 according to the above-described embodiment is stored in the memory of the general-purpose computer, and the program is read and executed by a processor. Therefore, the present disclosure can also be realized as a program executable by a processor, or a non-transitory computer-readable medium that stores the program.
[0093] Some of the embodiments of the present disclosure will be described below as examples. However, it should be noted that the embodiments of the present disclosure are not limited to these examples. [Appendix 1] An information processing device including a control unit, The control unit is Obtaining a first image of a first medium including a first region having text thereon; projecting the first image onto a second medium at a predetermined location; obtaining a second image of the second medium including a second area with lines of a different color than the characters shown in the projected first image; correcting color data of an overlapping area where the first area and the second area on the second image overlap; An information processing device that projects the corrected second image onto the first medium so that the positions of the characters coincide with each other. [Appendix 2] 2. The information processing device according to claim 1, The color of the characters is black, The information processing device, wherein when projecting the first image onto the second medium, the control unit irradiates an entirety of a platform on which the second medium is placed with illumination light, avoiding the first area. [Appendix 3] The information processing device according to claim 2, The information processing device, wherein the placement table is shielded from light other than the projection light for projecting the first image and the illumination light. [Appendix 4] An information processing device according to any one of claims 1 to 3, An information processing device, wherein the color data is an RGB value corresponding to a color of the overlapping region. [Appendix 5] The information processing device according to claim 4, The control unit corrects RGB values corresponding to the color of the overlapping area based on RGB values corresponding to the color of the line in an area on the second image where the first area and the second area do not overlap. [Appendix 6] 6. The information processing device according to claim 1, The first image is a first image having a predetermined number of frames, and the second image is a second image having a predetermined number of frames. [Appendix 7] 7. The information processing device according to claim 6, The control unit further acquires a third image of a predetermined number of frames and a fourth image of a predetermined number of frames, each captured from the front of a first user writing the characters on the first medium and a second user writing the lines on the second medium, and displays the appearance of each other in real time on a panel positioned in front of each of the first user and the second user based on the third image of the predetermined number of frames and the fourth image of the predetermined number of frames. [Appendix 8] A projection method executed by an information processing device, comprising: Obtaining a first image of a first medium including a first area having text thereon; projecting the first image onto a second medium at a predetermined location; obtaining a second image of the second medium including a second area with lines of a different color than the text shown in the first projected image; correcting color data of an overlapping area where the first area and the second area on the second image overlap; and projecting the corrected second image onto the first medium so that the positions of the characters coincide with each other; A projection method comprising: [Appendix 9] 9. The projection method according to claim 8, further comprising: The color of the characters is black, When the first image is projected onto the second medium, an illumination light is irradiated onto an entirety of a mounting table on which the second medium is placed, while avoiding the first region. The projection method further comprises: [Appendix 10] 10. The projection method according to claim 9, further comprising: The projection method, wherein the mounting table is shielded from light other than the projection light for projecting the first image and the illumination light. [Appendix 11] 11. The projection method according to any one of claims 8 to 10, further comprising: A projection method, wherein the color data is an RGB value corresponding to a color of the overlap region. [Appendix 12] 12. The projection method according to claim 11, further comprising: correcting an RGB value corresponding to a color of the overlapping region based on an RGB value corresponding to a color of the line in a region on the second image where the first region and the second region do not overlap; The projection method further comprises: [Appendix 13] 13. The projection method according to any one of claims 8 to 12, further comprising: A projection method, wherein the first image is a first image of a predetermined frame number, and the second image is a second image of a predetermined frame number. [Appendix 14] 14. The projection method according to claim 13, further comprising: further acquiring a third image of a predetermined number of frames and a fourth image of a predetermined number of frames obtained by capturing an image of a first user writing the character on the first medium and a second user writing the line on the second medium from the front, and displaying the appearance of each of the first user and the second user in real time on a panel disposed in front of each of the first user and the second user based on the third image of the predetermined number of frames and the fourth image of the predetermined number of frames. The projection method further comprises: [Appendix 15] In the information processing device, Obtaining a first image of a first medium including a first area having text thereon; projecting the first image onto a second medium at a predetermined location; obtaining a second image of the second medium including a second area with lines of a different color than the text shown in the first projected image; correcting color data of an overlapping area where the first area and the second area on the second image overlap; and projecting the corrected second image onto the first medium so that the positions of the characters coincide with each other; A program to execute. [Appendix 16] 16. The program of claim 15, The color of the characters is black, When the first image is projected onto the second medium, an illumination light is irradiated onto an entirety of a mounting table on which the second medium is placed, while avoiding the first region. The program further causes the information processing device to execute the above. [Appendix 17] 17. The program of claim 16, The mounting base is shielded from light other than the projection light for projecting the first image and the illumination light. [Appendix 18] 18. The program according to any one of appendices 15 to 17, The color data is an RGB value corresponding to a color of the overlapping region. [Appendix 19] 19. The program of claim 18, correcting an RGB value corresponding to a color of the overlapping region based on an RGB value corresponding to a color of the line in a region on the second image where the first region and the second region do not overlap; The program further causes the information processing device to execute the above. [Appendix 20] 20. The program according to any one of claims 15 to 19, The first image is a first image of a predetermined frame number, and the second image is a second image of a predetermined frame number. [Explanation of symbols]
[0094] 1 System 10 First image processing device 11 Communications Department 12 Imaging unit 13 Projection section 14 Detection unit 15 Output section 16 Input section 17 Memory section 18 Control Unit 20 Second image processing device 21 Communications Department 22 Imaging unit 23 Projection section 24 Detection unit 25 Output section 26 Input section 27 Memory section 28 Control Unit 30 Information processing device 31 Communications Department 32 Storage section 33 Control Unit 40 Network M1 First user's paper M2 Second user's paper R1 Area with written text R2 Area with lines R3 overlap area S1 Image of the first user's paper S2: Second user's paper image
Claims
1. An information processing device including a control unit, The control unit is Obtaining a first image of a first medium including a first area having text thereon; projecting the first image onto a predetermined location on a second medium; obtaining a second image of the second medium including a second area having lines of a different color than the characters shown in the projected first image; correcting color data of an overlapping area where the first area and the second area on the second image overlap so that a decrease in lightness or saturation due to the overlap is eliminated by a predetermined ratio; An information processing device that projects the corrected second image onto the first medium so that the positions of the characters coincide with each other.
2. 2. The information processing device according to claim 1, The color of the characters is black, The control unit, when projecting the first image onto the second medium, irradiates the entirety of a platform on which the second medium is placed with illumination light, avoiding the first area.
3. 3. The information processing device according to claim 2, The information processing apparatus, wherein the placement table is shielded from light other than the projection light for projecting the first image and the illumination light.
4. 4. The information processing device according to claim 1, The information processing device, wherein the color data is RGB values corresponding to the color of the overlapping area.
5. 5. The information processing device according to claim 4, The control unit corrects RGB values corresponding to the color of the overlapping area based on RGB values corresponding to the color of the line in an area on the second image where the first area and the second area do not overlap.
6. 4. The information processing device according to claim 1, An information processing device, wherein the first image is a first image having a predetermined number of frames, and the second image is a second image having a predetermined number of frames.
7. 7. The information processing device according to claim 6, The control unit further acquires a third image of a predetermined number of frames and a fourth image of a predetermined number of frames, each captured from the front of a first user writing the characters on the first medium and a second user writing the lines on the second medium, and displays the appearance of each other in real time on a panel positioned in front of each of the first user and the second user based on the third image of the predetermined number of frames and the fourth image of the predetermined number of frames.
8. A projection method executed by an information processing device, comprising: Obtaining a first image of a first medium including a first area having text thereon; projecting the first image onto a second medium at a predetermined location; acquiring a second image of the second medium including a second area of lines having a different color than the text shown in the first projected image; correcting color data of an overlapping area where the first area and the second area on the second image overlap so that a decrease in lightness or saturation due to the overlap is eliminated by a predetermined ratio; and projecting the corrected second image onto the first medium so that the positions of the characters coincide with each other; A projection method comprising:
9. 9. The projection method according to claim 8, The color of the characters is black, When the first image is projected onto the second medium, an illumination light is irradiated onto an entirety of a mounting table on which the second medium is placed, while avoiding the first region. The projection method further comprises:
10. 10. The projection method according to claim 9, The projection method, wherein the mounting base is shielded from light other than the projection light for projecting the first image and the illumination light.
11. A projection method according to any one of claims 8 to 10, comprising: A projection method, wherein the color data is RGB values corresponding to the color of the overlap region.
12. 12. The projection method according to claim 11, correcting an RGB value corresponding to a color of the overlapping region based on an RGB value corresponding to a color of the line in a region on the second image where the first region and the second region do not overlap; The projection method further comprises:
13. A projection method according to any one of claims 8 to 10, comprising the steps of: A projection method, wherein the first image is a first image of a predetermined frame number, and the second image is a second image of a predetermined frame number.
14. 14. The projection method according to claim 13, further acquiring a third image of a predetermined number of frames and a fourth image of a predetermined number of frames, each of which is captured from the front of a first user who writes the characters on the first medium and a second user who writes the lines on the second medium, and displaying the appearances of the first user and the second user on a panel disposed in front of each of the first user and the second user in real time based on the third image of the predetermined number of frames and the fourth image of the predetermined number of frames. The projection method further comprises:
15. In the information processing device, Obtaining a first image of a first medium including a first area having text thereon; projecting the first image onto a second medium at a predetermined location; acquiring a second image of the second medium including a second area of lines having a different color than the text shown in the first projected image; correcting color data of an overlapping area where the first area and the second area on the second image overlap so that a decrease in lightness or saturation due to the overlap is eliminated by a predetermined ratio; and projecting the corrected second image onto the first medium so that the positions of the characters coincide with each other; A program to execute.
16. The program according to claim 15, The color of the characters is black, When the first image is projected onto the second medium, an illumination light is irradiated onto an entirety of a mounting table on which the second medium is placed, while avoiding the first region. The program further causes the information processing device to execute the above.
17. The program according to claim 16, The placement stand is shielded from light other than the projection light for projecting the first image and the illumination light.
18. 18. The program according to claim 15, The color data is RGB values corresponding to the color of the overlapping area.
19. 20. The program of claim 18, correcting an RGB value corresponding to a color of the overlapping region based on an RGB value corresponding to a color of the line in a region on the second image where the first region and the second region do not overlap; The program further causes the information processing device to execute the above.
20. 18. The program according to any one of claims 15 to 17, The first image is a first image of a predetermined number of frames, and the second image is a second image of a predetermined number of frames.
Citation Information
Patent Citations
Correction system
JP2002023603A
Calligraphy studying material, underlay for calligraphy, and japanese writing paper for calligraphy
JP2004170736A
Image input / output device
JP2005293290A
Calligraphy animation teaching material and calligraphy educational system
JP2006154691A
Display device, display system, and display method
JP2011197381A
