Terminal device

Abstract

To provide a technique capable of making correction results and the same easier to see.SOLUTION: A terminal 10 includes a control unit 17. The control unit 17 generates a third image by processing and combining at least one of a first image and a second image based on the result of comparing the first image and the second image. The control unit 17 causes a display unit 14 to display the third image. The first image includes at least letters or illustrations. The second image is different from the first image and includes at least letters or illustration.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a terminal device.

Background Art

[0002] Conventionally, a technique for projecting proofreading results and the like is known. For example, Patent Document 1 describes that when imaging an image of a piece of paper with writing as a subject, the imaging image data is compared with reference information as a result of being judged by a work result judgment program, and the part to be corrected is projected as a correction information image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is a need for a technique to make proofreading results and the like easier to view.

[0005] In view of this point, an object of the present disclosure is to provide a technique for making proofreading results and the like easier to view.

Means for Solving the Problems

[0006] A terminal device according to an embodiment of the present disclosure includes a control unit that generates a third image by processing and synthesizing at least one of the first image and the second image based on a result of comparing a first image including at least characters or illustrations and a second image different from the first image and including at least characters or illustrations, and causes the third image to be displayed on a display device and includes.

Effects of the Invention

[0007] According to an embodiment of the present disclosure, a technique for making correction results and the like easier to view can be provided.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments according to the present disclosure will be described with reference to the drawings.

[0010] As shown in FIG. 1, the system 1 includes a terminal device 10A and a terminal device 10B. Hereinafter, when the terminal device 10A and the terminal device 10B are not particularly distinguished, they are also described as "terminal device 10". The system 1 includes two terminal devices 10. However, the system 1 may include two or more terminal devices 10.

[0011] The terminal device 10A and the terminal device 10B can communicate with each other via the network 2. The network 2 may be any network including a mobile communication network and the Internet.

[0012] The system 1 is applied to the correction of calligraphy practice. However, as will be described later, what the system 1 is applied to is not limited to the correction of calligraphy practice.

[0013] The terminal device 10A is used by the user 3A. The user 3A is a calligraphy teacher. The user 3A uses the terminal device 10A to correct the characters written by the user 3B.

[0014] The terminal device 10B is used by the user 3B. The user 3B is a calligraphy student. The user 3B uses the terminal device 10B to learn calligraphy from the user 3A.

[0015] The terminal device 10 is a terminal device such as, for example, a desktop PC (Personal Computer), a tablet PC, a notebook PC, or a smartphone.

[0016] (Configuration of the terminal device) As shown in FIG. 1, the terminal device 10 includes a communication unit 11, an input unit 12, an output unit 13, a display unit 14, a camera 15, a storage unit 16, and a control unit 17. The terminal device 10 includes a display device. The display device includes the display unit 14 and the projector of the output unit 13.

[0017] The communication unit 11 is configured to include at least one communication module connectable to the network 2. The communication module is, for example, a communication module compliant with a standard such as a wired LAN (Local Area Network) or a wireless LAN, or a communication module compliant with a mobile communication standard such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation).

[0018] The input unit 12 is capable of receiving an input from a user. The input unit 12 is configured to include at least one input interface capable of receiving an input from a user. The input interface is, for example, a physical key, a capacitive key, a pointing device, a touch screen provided integrally with the display of the display unit 14, or a microphone.

[0019] The output unit 13 is capable of outputting data. The output unit 13 is configured to include at least one output interface capable of outputting data. The output interface includes a speaker or a projector. In the present embodiment, the output unit 13 is configured to include a projector.

[0020] The display unit 14 can display data. The display unit 14 is configured to include, for example, a display or the like. The display is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display or the like.

[0021] The camera 15 can capture a subject and generate a captured image. The camera 15 is, for example, a visible light camera. The camera 15 continuously captures the subject at an arbitrary frame rate, for example. The captured image generated by the camera 15 is, for example, a color (RGB) image. However, the captured image generated by the camera 15 may be a monochrome image.

[0022] The storage unit 16 is configured to include at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The storage unit 16 may function as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 16 stores data used for the operation of the terminal device 10 and data obtained by the operation of the terminal device 10.

[0023] The control unit 17 is configured to include at least one processor, at least one dedicated circuit, or a combination of these. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for specific processing. The control unit 17 executes processes related to the operation of the terminal device 10 while controlling each part of the terminal device 10.

[0024] (Operation of the system) FIG. 2 is a sequence diagram showing the operation procedure of the system 1 shown in FIG. 1. The operation procedure shown in FIG. 2 is an example of the display method according to the present embodiment. When the user 3B writes characters on the paper placed in front, the system 1 starts the process of step S1. Alternatively, when the control unit 17 of the terminal device 10B receives an input of a proofreading instruction from the user 3B through the input unit 12, the system 1 may start the process of step S1.

[0025] In the process of step S1, in the terminal device 10B, the control unit 17 acquires the data of the first image including at least characters or illustrations. In the present embodiment, the control unit 17 acquires the data of the image of the characters written by the user 3B on the paper as the data of the first image. However, depending on the application to which the system 1 is applied, the first image may include only characters, only illustrations, or both characters and illustrations. Hereinafter, when the characters and illustrations included in the first image are not particularly distinguished, these are also referred to as "objects to be proofread". As an example of the process, the control unit 17 causes the camera 15 to image the paper placed in front of the user 3B as a subject, thereby acquiring the data of the image of the paper. The control unit 17 executes image processing on the data of the image of the paper to acquire the data of the first image including the characters written by the user 3B. In the example of FIG. 3 described later, the first image includes the character 21 of "heart".

[0026] In the process of step S2, in the terminal device 10B, the control unit 17 transmits the data of the first image to the terminal device 10A through the network 2 by the communication unit 11.

[0027] In the process of step S3, in the terminal device 10A, the control unit 17 receives the data of the first image from the terminal device 10B through the network 2 by the communication unit 11.

[0028] In the process of step S4, in the terminal device 10A, the control unit 17 projects the first image onto the piece of paper placed in front of the user 3A by the projector of the output unit 13. Through such a process, the characters written by the user 3B are projected onto the piece of paper placed in front of the user 3A. The user 3A corrects the characters written by the user 3B projected on the piece of paper. In the example of FIG. 3 described later, the user 3A corrects the character 21 of "heart" written by the user 3B by writing the dot illustration 22 and the double-circle illustration 23 on the piece of paper.

[0029] In the process of step S5, in the terminal device 10A, the control unit 17 acquires the data of the second image. The second image is an image different from the first image and includes at least characters or illustrations. In the present embodiment, the control unit 17 acquires the data of the image of the characters or the like written by the user 3A on the piece of paper by correction as the data of the second image. However, depending on the application to which the system 1 is applied, the second image may include only characters, only illustrations, or both characters and illustrations. Hereinafter, when the characters and illustrations included in the second image are not particularly distinguished, these are also described as "correction results". As a processing example, the control unit 17 causes the camera 15 to image the piece of paper placed in front of the user 3A as a subject, thereby acquiring the data of the image of the piece of paper. The control unit 17 executes image processing on the data of the image of the piece of paper to acquire the data of the second image including the correction result. In the example of FIG. 3 described later, the second image includes the dot illustration 22 and the double-circle illustration 23. Since the dot illustration 22 is a part of the character, it may be interpreted as a character. The double circle means that it has a high evaluation in calligraphy.

[0030] In the process of step S6, in the terminal device 10A, the control unit 17 transmits the data of the second image to the terminal device 10B through the network 2 by the communication unit 11. The control unit 17 may transmit the data of the image of the piece of paper together with the data of the second image to the terminal device 10B.

[0031] In the process of step S7, in the terminal device 10B, the control unit 17 receives the data of the second image from the terminal device 10A via the network 2 by the communication unit 11. In the terminal device 10B, the control unit 17 may receive the data of the image of the half sheet together with the data of the second image.

[0032] In the process of step S8, in the terminal device 10B, the control unit 17 compares the first image and the second image. In the comparison between the first image and the second image, the control unit 17 may compare any element related to the display mode.

[0033] For example, the control unit 17 may compare the luminance value of the correction target of the first image with the luminance value of the correction result of the second image. In this case, the result of comparing the first image and the second image is the luminance difference between the correction target of the first image and the correction result of the second image.

[0034] For example, the control unit 17 may compare the lightness of the correction target of the first image with the lightness of the correction result of the second image. In this case, the result of comparing the first image and the second image is the lightness difference between the correction target of the first image and the correction result of the second image.

[0035] For example, the control unit 17 may compare the position of the correction target in the first image with the position of the correction result in the second image. In this case, the result of comparing the first image and the second image is the overlapping part between the correction target of the first image and the correction result of the second image.

[0036] In the process of step S9, in the terminal device 10B, based on the result of comparing the first image and the second image, the control unit 17 generates a third image by processing and synthesizing at least one of the first image and the second image. Here, the control unit 17 may process at least one of the first image and the second image so that the correction result of the second image is easier to view based on the result of comparing the first image and the second image. The control unit 17 may process the overlapping portion between the correction target of the first image and the correction result of the second image. When the control unit 17 receives the data of the half-sheet image in the process of step S7, the control unit 17 may synthesize the first image and the second image with reference to the received data of the half-sheet image. As an example, the control unit 17 may synthesize the first image and the second image so that the positional relationship between the correction target of the first image and the correction result of the second image in the third image is the same as the positional relationship between the correction target and the correction result in the received half-sheet image.

[0037] For example, based on the result of comparing the first image and the second image, the control unit 17 may change at least one color of the first image and the second image. When the brightness difference between the object to be corrected in the first image and the correction result of the second image is equal to or less than the first threshold value, the control unit 17 may change at least one color of the first image and the second image. The first threshold value may be set in consideration of the visibility of the color of characters or illustrations in the display by the display device. Alternatively, when the luminance difference between the object to be corrected in the first image and the correction result of the second image is equal to or less than the second threshold value, the control unit 17 may change at least one color of the first image and the second image. The second threshold value may be set in consideration of the visibility of the color of characters or illustrations in the display by the display device. The control unit 17 may change at least one color of the first image and the second image so that the color of the object to be corrected in the first image and the color of the correction result of the second image are positioned exactly opposite each other on the color wheel. The control unit 17 may change only the color of the correction result of the second image received from the terminal device 10A, which is the communication partner of the terminal device 10B, to an easy-to-see color. The easy-to-see color may be set based on the color that an ordinary person feels is easy to see. Regarding the part where the color is changed, the control unit 17 may change all the colors of the object to be corrected in the first image or all the colors of the correction result of the second image, or may change some of the colors of the object to be corrected in the first image or some of the colors of the correction result of the second image. When changing some of the colors, the control unit 17 may change only the colors of the overlapping part between the object to be corrected in the first image and the correction result of the second image.

[0038] For example, based on the result of comparing the first image and the second image, the control unit 17 may increase the transparency of at least one of the first image and the second image. The control unit 17 may increase the transparency of only the portion where the proofreading target of the first image overlaps with the proofreading result of the second image in at least one of the first image and the second image. The control unit 17 may increase the transparency of the portion where the transparency is to be increased in at least one of the first image and the second image so that the transparency of the portion is higher than the original transparency of the portion. The control unit 17 may determine the degree of increasing the transparency based on the visibility of the proofreading target or the proofreading result in the display by the display device. When the brightness difference between the proofreading target of the first image and the proofreading result of the second image is equal to or less than the first threshold value or the luminance difference between the proofreading target of the first image and the proofreading result of the second image is equal to or less than the second threshold value, the control unit 17 may increase the transparency of only the proofreading target of the first image. By increasing the transparency of only the proofreading target of the first image, the proofreading result of the second image can be made more visible.

[0039] In the process of step S10, in the terminal device 10B, the control unit 17 causes the display device to display the third image. In the present embodiment, the control unit 17 causes the display unit 14 as the display device to display the third image. However, the display device is not limited to the display unit 14. As another example, the display device may be a projector of the output unit 13. In this case, the control unit 17 causes the projector of the output unit 13 to project the third image onto a desk or the like, thereby causing the projector to display the third image.

[0040] By executing the process of step S10, a third image 20 as shown in FIG. 3 is displayed on the display unit 14 of the terminal device 10B. The third image 20 includes a character 21 of "heart", a dot illustration 22, and a double-circle illustration 23. The character 21 is a character included in the first image. That is, the character 21 is a character written by the user 3B on the Japanese paper. The illustrations 22 and 23 are the correction results included in the second image. That is, the illustrations 22 and 23 are what the user 3A wrote on the Japanese paper by correction. In the third image 20, the character 21 and the illustration 22 overlap at a portion 24. Also, in the third image 20, the character 21 and the illustration 23 overlap at portions 25, 26, and 27. The transparency of portions 24 to 27 of the character 21 is increased by the process of the control unit 17 compared to the original transparency of portions 24 to 27. With such a configuration, the dot illustration 22 and the double-circle illustration 23 can be made easier to see.

[0041] Here, in the process of step S2, the terminal device 10B may transmit the data of the captured image of the previous Japanese paper of the user 3B to the terminal device 10A instead of the first image. In this case, in the process of step S3, the terminal device 10A may receive the data of the Japanese paper image from the terminal device 10B. Also, in the process of step S4, the control unit 17 of the terminal device 10A may obtain the data of the first image including the character written by the user 3B by performing image processing on the received data of the Japanese paper image. In the process of step S4, the control unit 17 of the terminal device 10A may project the obtained first image onto the Japanese paper placed in front of the user 3A by the projector of the output unit 13.

[0042] Also, in the process of step S6, the terminal device 10A may transmit the data of the captured image of the front half-sheet of paper of the user 3A to the terminal device 10B instead of the second image. In this case, the terminal device 10A may not execute the process of step S5. Also, in the process of step S7, the terminal device 10B may receive the data of the image of the half-sheet of paper from the terminal device 10A. In the process of step S7, the control unit 17 of the terminal device 10B may obtain the data of the second image including the correction result by the user 3A by performing image processing on the received data of the image of the half-sheet of paper.

[0043] As described above, in the terminal device 10B according to this embodiment, the control unit 17 generates the third image by processing and synthesizing at least one of the first image and the second image based on the result of comparing the first image and the second image. With such a configuration, the correction result of the second image can be made more visible in the third image. Here, a method of presenting the correction result of the second image to the user 3B by projecting the second image onto the front half-sheet of paper of the user 3B using a projector or the like is also conceivable. However, with this method, the correction result of the second image may be difficult to see due to reasons such as the color of the characters of the user 3B actually written on the half-sheet of paper being darker. In contrast, in this embodiment, the control unit 17 does not project the second image onto the front half-sheet of paper of the user 3B using a projector or the like, but generates the third image and causes the generated third image to be displayed on the display device. With such a configuration, the correction result of the second image can be made more visible than when the second image is projected onto the front half-sheet of paper of the user 3B.

[0044] Therefore, according to this embodiment, it is possible to provide a technique for making correction results and the like more visible.

[0045] Furthermore, in the terminal device 10B according to this embodiment, the control unit 17 may change the color of at least one of the first image and the second image based on the result of comparing the first image and the second image. The process of changing the color can be implemented, for example, by simply replacing the color. Therefore, with such a configuration, the process of processing the first image and the second image can be simplified.

[0046] Also, in the terminal device 10B according to the present embodiment, the control unit 17 may change at least one color of the first image and the second image so that the color of the object to be corrected in the first image and the color of the correction result in the second image are located exactly opposite to each other on the color wheel. With such a configuration, the correction result of the second image can be made more visible in the third image.

[0047] Also, in the terminal device 10B according to the present embodiment, the control unit 17 may increase the transparency of at least one of the first image and the second image based on the result of comparing the first image and the second image. With such a configuration, the correction result of the second image can be made visible in the third image without changing the colors of the first image and the second image. By not changing the colors of the first image and the second image, the originality of the first image and the second image can be maintained. Also, the process of increasing the transparency is a relatively easy process in image processing. Therefore, the process of processing the first image and the second image can be simplified.

[0048] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art may make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions etc. included in each component or each step etc. can be rearranged so as not to be logically contradictory, and it is possible to combine or divide a plurality of components or steps etc. into one.

[0049] For example, in the above-described embodiment, the student who receives the correction of calligraphy practice has been described as being one of the users 3B. However, the students who receive the correction of calligraphy practice may be plural.

[0050] For example, in the above-described embodiment, the system 1 has been described as being applied to the correction of calligraphy practice. However, what the system 1 is applied to is not limited to the correction of calligraphy practice. The system 1 may be applied to any thing for which correction is performed. As another example, the system 1 may be applied to the correction of paintings or arithmetic drills etc.

[0051] For example, in the above-described embodiment, the terminal device 10A and the terminal device 10B have been described as directly communicating via the network 2. However, the terminal device 10A and the terminal device 10B may communicate via a server device.

[0052] For example, an embodiment in which a general-purpose computer functions as the terminal device 10 according to the above-described embodiment is also possible. Specifically, a program describing the processing content for realizing each function of the terminal device 10 according to the above-described embodiment is stored in the memory of a general-purpose computer, and the program is read and executed by a processor. Therefore, the present disclosure can also be realized as a program executable by a processor or a non-transitory computer-readable medium storing the program.

[0053] Hereinafter, some embodiments of the present disclosure will be exemplified. However, it should be noted that the embodiments of the present disclosure are not limited to these. [Appendix 1] A control unit that generates a third image by processing and synthesizing at least one of the first image and the second image based on a result of comparing a first image including at least characters or illustrations and a second image different from the first image and including at least characters or illustrations, and causes the third image to be displayed on a display device A terminal device comprising: [Appendix 2] The terminal device according to Appendix 1, wherein the control unit changes the color of at least one of the first image and the second image based on a result of comparing the first image and the second image. [Appendix 3] The terminal device according to Appendix 1 or 2, wherein the control unit changes the color of at least one of the first image and the second image such that the color of the characters or illustrations included in the first image and the color of the characters or illustrations included in the second image are positioned exactly opposite each other on the color wheel. [Appendix 4] The control unit is the terminal device according to any one of Appendices 1 to 3, which increases the transparency of at least one of the first image and the second image based on the result of comparing the first image and the second image. [Appendix 5] The result of comparing the first image and the second image includes a portion where the characters or illustrations included in the first image overlap with the characters or illustrations included in the second image. The control unit processes the overlapping portion. The terminal device according to any one of Appendices 1 to 4. [Appendix 6] The first image includes the object to be corrected. The second image includes the correction result. The terminal device according to any one of Appendices 1 to 5. [Appendix 7] Based on the result of comparing a first image including at least characters or illustrations and a second image different from the first image and including at least characters or illustrations, at least one of the first image and the second image is processed and synthesized to generate a third image, and the third image is displayed on a display device. Including, a display method. [Appendix 8] Causing a computer to Based on the result of comparing a first image including at least characters or illustrations and a second image different from the first image and including at least characters or illustrations, at least one of the first image and the second image is processed and synthesized to generate a third image, and the third image is displayed on a display device. To execute, a program.

Explanation of Signs

[0054] 1: System, 2: Network, 3A, 3B: Users, 10, 10A, 10B Terminal devices, 11: Communication unit, 12: Input unit, 13: Output unit, 14: Display unit, 15: Camera, 16: Storage unit, 17: Control unit, 20: Third image, 21: Characters, 22, 23: Illustrations, 24, 25, 26, 27: Portions

Claims

1. A terminal device comprising a communication unit and a control unit, wherein the control unit transmits data of a first image including at least characters or illustrations to a terminal device of a communication partner by the communication unit, receives data of a second image from the terminal device of the communication partner by the communication unit, the second image being a second image different from the first image and including at least characters or illustrations, generates a third image by processing and synthesizing at least one of the first image and the second image based on a result of comparing the first image and the second image, and causes the third image to be displayed on a display device.

2. The terminal device according to claim 1, wherein the control unit changes a color of at least one of the first image and the second image based on a result of comparing the first image and the second image.

3. The terminal device according to claim 2, wherein the control unit changes a color of at least one of the first image and the second image such that a color of a character or illustration included in the first image and a color of a character or illustration included in the second image are located exactly opposite each other on a color wheel.

4. The terminal device according to claim 1, wherein the control unit increases an opacity of at least one of the first image and the second image based on a result of comparing the first image and the second image.

5. The result of comparing the first image and the second image includes a portion where a character or illustration included in the first image overlaps with a character or illustration included in the second image, The terminal device according to any one of claims 1 to 4, wherein the control unit changes only the color or opacity of the overlapping portion.

Images