Methods, programs, and electronic devices for displaying learning data.

Abstract

To provide further learning assistance to users.SOLUTION: A learning data generation method is provided, comprising causing a control unit to acquire target data representing learning content and mask management data for managing masks for hiding parts of the target data, recognize specific symbols included in the acquired target data, and generate mask-processed data by adding mask data based on the recognized specific symbols and the acquired mask management data to the target data.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] The present invention relates to a method for learning data display and a program and an electronic device. ,

Background Art

[0002] In learning content (for example, learning materials such as dictionaries, word books, textbooks, reference books, notes, etc.), it has been common practice to create a problem that allows a user (learner) to easily check whether they have memorized an item by masking the item to be memorized with a marker or the like. Also, techniques for automatically creating such problems have been developed. For example, Patent Document 1 discloses an information extraction device that can generate problems related to drawings using image data of learning materials.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The information extraction device disclosed in Patent Document 1 analyzes image data of learning materials, identifies a drawing area, a drawing peripheral area, and a character area respectively, and then performs character recognition. Then, text data in the drawing area and the drawing peripheral area is extracted as keywords, and the created keywords are searched for and erased from the text data in the drawing area and the character area, thereby creating a fill-in-the-blank problem. However, in the information extraction device disclosed in Patent Document 1, since problems are created based on keywords extracted from the drawing area and the drawing peripheral area, it was not always possible to create problems suitable for the user.

[0005] This invention was made in view of the above circumstances, and provides learning data that can further support user learning. display method , The objective is to provide programs and electronic devices. [Means for solving the problem]

[0006] To achieve the above objective, the training data according to the present invention display One aspect of the method is, The control unit, We obtain target data, which is learning content, and mask management data, which is data that manages a mask that hides a part of the target data. Recognizing a specific symbol included in the acquired target data, Masked data is created by adding the recognized specific symbol and the mask data based on the acquired mask management data to the acquired target data. death, The mask processing data created above is acquired, The mask data is obtained from the mask processing data acquired above, The aforementioned target data is partially hidden and displayed based on the acquired mask data. . [Effects of the Invention]

[0007] According to the present invention, user learning can be further supported. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the functional configuration of the electronic device according to Embodiment 1. [Figure 2] This figure shows an example of displaying image data of pages from an English-Japanese dictionary. [Figure 3] This figure shows an example of specific symbol information according to Embodiment 1. [Figure 4] This figure shows an example of mask management data according to Embodiment 1. [Figure 5] This figure shows an example of displaying mask processing data according to Embodiment 1. [Figure 6]This is a diagram showing an example of mask data according to Embodiment 1. [Figure 7] This is a diagram showing a display example of image data obtained by photographing the pages of other English-Japanese dictionaries. [Figure 8] This is a diagram showing another example of specific symbol information according to Embodiment 1. [Figure 9] This is a flowchart of the mask creation process according to Embodiment 1. [Figure 10] This is a flowchart of the mask display process according to Embodiment 1. [Figure 11] This is a flowchart of the audio content playback process according to Embodiment 2.

Embodiments for Carrying Out the Invention

[0009] The electronic device and the like according to the embodiment will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

[0010] (Embodiment 1) The electronic device 100 according to Embodiment 1 is a smartphone with a built-in camera, a tablet, a PC (Personal Computer), or the like. The electronic device 100 acquires image data (target data) serving as learning content by photographing the pages of an English-Japanese dictionary or the screen of an electronic device (e.g., an electronic dictionary) with the built-in camera. Then, based on the symbols (symbols indicating importance rank, word class, usage examples, etc.) on the page, mask data indicating the position of the mask for hiding a part of the page is generated, and learning data (masked data) is created by adding the mask data to the target data.

[0011] As shown in FIG. 1, the electronic device 100 according to Embodiment 1 includes a control unit 110, a storage unit 120, a data acquisition unit 130, a display unit 140, an operation input unit 150, a communication unit 160, and an audio output unit 170.

[0012] The control unit 110 is composed of a processor such as a CPU (Central Processing Unit). The control unit 110 executes mask creation processing and the like, which will be described later, according to the program stored in the storage unit 120.

[0013] The storage unit 120 stores the programs executed by the control unit 110 and necessary data. The storage unit 120 may include a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, etc., but is not limited thereto. Note that the storage unit 120 may be provided inside the control unit 110.

[0014] In addition, the storage unit 120 stores specific symbol information 121 and mask management data 122. The specific symbol information 121 is information for image recognition of specific symbols (symbols indicating importance rank, part of speech, examples, etc.) from the image data of the target data. The mask management data 122 is data for managing masks, such as the importance rank of the word to be masked, the position to be masked, the type of information to be masked (the word itself, part of speech, examples, etc.). Note that the specific symbol information 121 and the mask management data 122 may be acquired from an external device or a cloud service on the Internet via the communication unit 160, which will be described later.

[0015] The data acquisition unit 130 includes a camera and acquires image data serving as learning content by photographing a paper surface of an English-Japanese dictionary or the like (or a screen of an electronic device (e.g., an electronic dictionary)) with the camera.

[0016] The display unit 140 includes a display device such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 140 displays, for example, the image data acquired by the data acquisition unit 130.

[0017] The operation input unit 150 is a user interface that receives user operation inputs (such as touch, drag, click, key input, etc.) such as a touch panel, a keyboard, a mouse, etc.

[0018] The communication unit 160 is a network interface compatible with wireless LAN (Local Area Network), LTE (Long Term Evolution), etc. The electronic device 100 can access external devices and cloud services on the internet via the communication unit 160.

[0019] The audio output unit 170 is equipped with a speaker and an earphone jack, and can play audio content stored in the memory unit 120 or output sound effects.

[0020] The specific symbols (symbols indicating importance rank, part of speech, example usage, etc.) stored in the memory unit 120 as specific symbol information 121 differ depending on the learning content. Therefore, this explanation will use the example of when image data 200, as shown in Figure 2, is obtained when the pages of an English-Japanese dictionary used as learning content are photographed with a camera. However, the learning content is not limited to English-Japanese dictionaries. If the learning content contains specific symbols, it is also possible to target dictionaries of other languages ​​such as Chinese, German, and Spanish, and it is also possible to target textbooks, reference books, etc., in addition to dictionaries.

[0021] As shown in Figure 2, this English-Japanese dictionary includes specific symbols to indicate the importance rank of words: symbol 211 for rank A (e.g., words at the junior high school level), symbol 212 for rank B (e.g., words at the high school level), symbol 213 for rank C (e.g., words at the university / working adult level), and symbol 214 for rank D (other general words).

[0022] Furthermore, specific symbols indicating the part of speech of a word are provided: symbol 221 for verbs, symbol 222 for transitive verbs, symbol 223 for intransitive verbs, symbol 224 for nouns, symbol 225 for uncountable nouns, symbol 226 for countable nouns, and symbol 227 for adjectives. In addition, if examples of usage are provided for each word, a symbol 230 indicating that examples have been provided is provided.

[0023] The control unit 110 is pre-trained to recognize these specific symbols by image recognition, and the trained data is stored in the storage unit 120 as specific symbol information 121. As shown in Figure 3, the specific symbol information 121 includes, for each specific symbol, the meaning of the symbol (symbol type), information for image recognition of that symbol (symbol recognition information), and information on the positional relationship of that symbol in the image data 200 (position information). Note that although the shape of the specific symbol is displayed in Figure 3, the symbol recognition information is actually information from the trained data that has been learned to recognize that symbol by image recognition.

[0024] The positional information included in the specific symbol information 121, as shown in Figure 3, indicates the positional relationship between the specific symbol and words, definitions, examples, etc., and is used by the control unit 110 to determine the mask position when masking the image data 200 based on the specific symbol. For example, when masking a word based on the importance rank specific symbol, the positional information is "word (English) to the right," so the English characters are masked from the right (immediately after) the specific symbol. Similarly, when masking a definition based on the part of speech specific symbol, the positional information is "definition (Japanese) to the right," so the Japanese characters are masked from the right (immediately after) the specific symbol. Furthermore, when masking the English in an example based on the example symbol, the positional information is "example (English), newline, example (Japanese) to the right," so the English characters from the right (immediately after) the specific symbol up to the newline are masked. The language in parentheses in the positional information is not limited to English or Japanese. For example, if a Chinese-Japanese dictionary is being used, the language in parentheses in the positional information may be Chinese.

[0025] When performing these masking processes, the extent to which the masking extends can simply be set to "up to the line break." Alternatively, image recognition can be used to identify spaces exceeding a certain size, and the masking can extend to the point before those spaces. Furthermore, image recognition can be used to recognize periods and mask up to those periods. Finally, the string obtained from image recognition can be morphologically analyzed, and certain groups of words or phrases (e.g., English clauses, Japanese phrases, etc.) can be masked.

[0026] Furthermore, the control unit 110 masks the target data (image data 200) acquired by photographing the pages of the learning content through a mask creation process described later. The mask management data 122 is data related to information such as which parts of the target data to mask. The mask management data 122 includes, for example, mask conditions, vocabulary list information, and proficiency level information, as shown in Figure 4.

[0027] Vocabulary list information refers to information about words registered by the user. Some commercially available electronic dictionaries have a vocabulary list function, and the electronic device 100 may have a similar vocabulary list function, or it may be able to obtain vocabulary list information registered by the user in a commercially available electronic dictionary via the communication unit 160. Furthermore, when obtaining vocabulary list information via the communication unit 160, the electronic device 100 may obtain the information by communicating directly with the electronic dictionary, or it may obtain it from the internet (such as a cloud service where vocabulary list information can be registered).

[0028] Proficiency information refers to information about the user's proficiency level. Some commercially available electronic dictionaries have a function to acquire the user's proficiency level by conducting proficiency tests, but the electronic device 100 may acquire the user's proficiency level in a similar manner, or it may be able to acquire the user proficiency information acquired by a commercially available electronic dictionary via the communication unit 160. Furthermore, when acquiring proficiency information via the communication unit 160, the electronic device 100 may acquire the proficiency information by communicating directly with the electronic dictionary, or it may acquire it from the internet (such as a cloud service where proficiency information can be registered).

[0029] A masking condition is a condition that masks the target data. In the example in Figure 4, the masking conditions include a symbol condition, a vocabulary condition, and a proficiency condition.

[0030] A symbolic condition is a condition that sets the mask position based on a specific symbol. In the example in Figure 4, the symbolic conditions are set as "mask words with importance rank B or higher" and "mask example sentences of words with importance rank B or higher." Therefore, words with importance rank A and their example sentences, as well as words with importance rank B and their example sentences, will be masked. In this example, masking the words can help in memorizing their spelling. Also, masking the example sentences can help in practicing Japanese-to-English translation from the Japanese example sentences. Note that symbolic conditions are not limited to this example. For example, if the symbolic condition is set as "mask the part of speech and meaning of words with importance rank B," the words themselves will not be masked, but their part of speech and meaning will be masked, which can help in memorizing the part of speech and meaning of words with importance rank B.

[0031] The vocabulary list condition is a masking condition based on vocabulary list information. In the example in Figure 4, the condition is "mask if registered in the vocabulary list." Also, in the example in Figure 4, words such as "tea,teach,tear,technic,technical" are registered in the vocabulary list, so these words registered in the vocabulary list will be masked. If the vocabulary list condition is set to "mask if not registered in the vocabulary list," words registered in the vocabulary list will not be masked. The use of the vocabulary list is flexible, so for example, if a user registers words they have already learned in the vocabulary list, they might consider setting it in this way.

[0032] The proficiency condition is a masking condition based on proficiency information. In the example in Figure 4, the condition is "mask if proficiency is 50% or less." Also, in the example in Figure 4, the proficiency information is "words with importance rank A: 50%, words with importance rank B: 30%", so both words with importance rank A and words with importance rank B will be masked. If the user's proficiency improves, for example to "words with importance rank A: 70%, words with importance rank B: 40%", then under the same proficiency condition, words with importance rank A will not be masked, while words with importance rank B will be masked.

[0033] Furthermore, in the example in Figure 4, the mask condition is set to "(symbol condition * vocabulary condition * proficiency condition)", meaning that the mask is applied only when all of these conditions are met. The mask condition is not limited to this; for example, if you want to ignore the vocabulary condition or proficiency condition and apply the mask, you can set the mask condition to "(symbol condition)". Alternatively, if you want to apply the mask when any of these conditions are met, you can set it to "(symbol condition + vocabulary condition + proficiency condition)". For more complex conditions, for example, if you want to apply the mask when the symbol condition is met AND at least one of the vocabulary condition or proficiency condition is met, you can set it to "(symbol condition * (vocabulary condition + proficiency condition))".

[0034] When the image data 200 (target data) shown in Figure 2 is masked based on the mask conditions of the mask management data 122 shown in Figure 4, the masked data 300 displayed as shown in Figure 5 is obtained. This masked data 300 is data to which the mask data 320 created based on the target data and the mask management data 122 has been added to the target data (image data 200).

[0035] As shown in Figure 6, the mask data 320 includes the specific symbol on which the mask is based (symbol type), the word being masked (masked word), the object being masked (masked target), and the position being masked (masked position). Although not shown in Figure 6, the mask data 320 may also include the translation of the mask word in other languages. This translation can be obtained, for example, by image recognition of the semantic portion of the image data 200. Conversely, the mask data 320 may also contain only a part of this information (for example, only the mask position). If the mask position information is available, the control unit 110 can apply a mask to the target data and display it using the mask display processing described later, and can also obtain information such as the symbol type and mask word by image recognition of the target data.

[0036] Figure 6 shows an example of mask data 320 added to the masked data 300 shown in Figure 5, and the masks 311, 312, 313, 314, and 315 in Figure 5 correspond to the mask data 321, 322, 323, 324, and 325 in each row of Figure 6, respectively.

[0037] For example, mask 311 is a mask targeting the word "teach," which has an importance rank of A. Mask data 321 indicates that it is a rectangle with a diagonal line connecting coordinates (100,25) and (1100,100) in image data 200. This mask data 321 is created based on the symbol condition "mask words with an importance rank of B or higher" in mask management data 122. The symbol 211, which has an importance rank of A, is recognized from image data 200, and based on the position information of specific symbol information 121, "word to the right (English)," the mask data is created by masking from the right of symbol 211 to the end of the line. The word to the right of symbol 211 is then recognized as text, and the mask word "teach" is obtained.

[0038] Furthermore, for example, mask 312 is a mask that targets the English portion of the example usage of the word "teach," which has an importance rank of A. Mask data 322 indicates that it is a rectangle with a diagonal line connecting coordinates (300,350) and (1100,425) in image data 200. This mask data 322 is based on the symbol condition of mask management data 122, "mask the English sentences of example usage of words with an importance rank of B or higher." Image recognition is performed on the symbol 211 of importance rank A and the example usage symbol 230 from image data 200. Based on the position information of specific symbol information 121, "example usage (English) to the right, newline, example usage (Japanese)," the mask data is created by masking from the right of symbol 230 to the end of the line. The word to the right of symbol 211 is then recognized as text, and the mask word "teach" is obtained.

[0039] Note that the format of the pages varies depending on the learning content (English-Japanese dictionary, etc.), and the images of specific symbols and their placement also differ. Therefore, it may be necessary to change the specific symbol information 121 to match the learning content that the user photographs. For example, the specific symbol information 121 corresponding to an English-Japanese dictionary that yields image data 201 as shown in Figure 7 when the page is photographed with a camera would be as shown in Figure 8.

[0040] In this English-Japanese dictionary, the English word is listed to the left of the symbol indicating the importance rank, so the position information for the importance rank in specific symbol information 121 is "Word (English) to the left." Also, since there is no symbol indicating importance rank D in this English-Japanese dictionary, in Figure 8, the column for the symbol with importance rank D is "None." However, as shown in Figure 7, the English word is listed to the left of the phonetic transcription, so the position information is "Word (English) to the left of the phonetic transcription." In the case of an English-Japanese dictionary like this example, the " / " indicating the phonetic transcription can be treated as a specific symbol indicating importance rank D, and the position information can be set to "Word (English) to the left."

[0041] Furthermore, in this English-Japanese dictionary, the example sentences are displayed consecutively without line breaks, as indicated by the positional information "Example sentence (English), Example sentence (Japanese) to the right." Therefore, when performing masking, instead of using methods such as "from the right of the symbol to the end of the line" or "from the beginning of the line to the end of the line," it is advisable to determine whether the text is English or Japanese using character recognition and then use methods such as "from the right of the symbol to where the language switches" or "from where the language switches to the end of the line."

[0042] Thus, the specific symbol information 121 may be switched according to the learning content captured by the user. Furthermore, to accommodate a large number of anticipated learning contents, the specific symbol information 121 may be prepared with multiple symbol recognition information registered for a single symbol type.

[0043] Furthermore, to accommodate any of the numerous anticipated learning contents, multiple specific symbol information 121 may be prepared, and the control unit 110 may recognize which learning content it is and switch the specific symbol information 121 when it acquires image data of the paper. This switching of specific symbol information 121 may be performed automatically by the control unit 110 based on the image data of the paper, or it may be possible to switch it manually by the user.

[0044] Next, the mask creation process will be explained with reference to Figure 9. This mask creation process starts when the user instructs the electronic device 100 to create mask processing data.

[0045] First, the control unit 110 acquires target data using the data acquisition unit 130 (step S101). Specifically, it acquires image data (for example, a JPEG (Joint Photographic Experts Group) image file) as target data by taking a picture of the learning content (such as the pages of an English-Japanese dictionary or the screen of an electronic dictionary) with the camera of the data acquisition unit 130.

[0046] Next, the control unit 110 acquires the mask management data 122 (step S102). The mask management data 122 is set in advance by the user and stored in the storage unit 120 or a cloud service on the internet. If the mask management data 122 is stored in the cloud service, the control unit 110 acquires the mask management data 122 via the communication unit 160 in step S102. Note that step S102 may be executed before step S101.

[0047] Next, the control unit 110 performs image recognition of a specific symbol from the target data (image data acquired in step S101) based on the specific symbol information 121 (step S103).

[0048] Then, based on the image recognition results in step S103, the control unit 110 determines whether or not a specific symbol that satisfies the mask conditions of the mask management data 122 exists in the target data (step S104). If no specific symbol that satisfies the mask conditions exists (step S104; No), the mask creation process is terminated.

[0049] If a specific symbol that satisfies the masking conditions exists (step S104; Yes), the control unit 110 creates mask data 320 from the target data based on the specific symbol information 121 and the image recognition result in step S103 (step S105). The mask data 320 created here includes "symbol type," "mask word," "mask target," "mask position," etc., as shown in Figure 6.

[0050] "Symbol Type" refers to the type of the specific symbol recognized from the image. "Mask Word" is the result of character recognition performed on the image of a word, after the control unit 110 identifies the position of the word surrounding the specific symbol based on the position information of the specific symbol information 121. "Mask Target" is information indicating whether the target of the mask is a word, a definition, an example, etc., and is acquired by the control unit 110 based on the position information of the specific symbol information 121. "Mask Position" is information indicating the coordinates to be masked, and is acquired by the control unit 110 based on the position information of the specific symbol information 121.

[0051] If multiple specific symbols that satisfy the masking conditions are recognized from the target data, mask data 320 is created and added for each of the specific symbols that satisfy the masking conditions. For example, Figure 6 shows the mask data 320 when five specific symbols are recognized.

[0052] Then, the control unit 110 adds mask data 320 to the target data to create masked data (step S106), saves it to the storage unit 120, and ends the mask creation process. Specifically, in step S106, if the target data is JPEG data, the control unit 110 creates masked data by embedding the mask data 320 as JPEG metadata into the target data, and saves it to the storage unit 120.

[0053] Furthermore, even if the determination in step S104 is No (i.e., no specific symbol that satisfies the mask condition exists), the control unit 110 may add empty mask data 320 to the target data to create masked data and save it in the storage unit 120.

[0054] If the target data is JPEG data, the masked data created in this way will be displayed as a normal JPEG image in a standard JPEG image viewer because the mask data is embedded as JPEG metadata. In other words, the masked data can be image data (such as JPEG) that can be displayed in a standard JPEG image viewer.

[0055] Next, the mask display process, which recognizes the mask data attached to the masked data and displays the mask, will be explained with reference to Figure 10. This mask display process is started when the user instructs the electronic device 100 to display the masked data. However, this mask display process is not limited to masked data; it can also display image data that does not have mask data attached.

[0056] First, the control unit 110 acquires image data (step S201). Specifically, in step S201, the control unit 110 may acquire image data such as mask processing data stored in the storage unit 120, or it may acquire image data from external devices or cloud services via the communication unit 160.

[0057] The control unit 110 then determines whether or not the acquired image data contains mask data (step S202). For example, if the image data is JPEG data, it determines whether or not mask data is embedded as JPEG metadata. If no mask data is included (step S202; No), the control unit 110 displays the image data as is on the display unit 140 (step S203) and terminates the mask display process.

[0058] If mask data is included (step S202; Yes), the control unit 110 masks the image data based on the mask data and displays it on the display unit 140 (step S204), and then terminates the mask display process.

[0059] Through the mask display process, image data that does not contain mask data is displayed on the display unit 140 without a mask, as shown in Figure 2, for example, while image data that contains mask data is displayed on the display unit 140 with a mask, as shown in Figure 5, for example.

[0060] The electronic device 100 can automatically create test questions with items to be memorized masked simply by acquiring images of learning content through the mask creation process and mask display process described above, thereby supporting the user's learning.

[0061] Since the items to be masked can be freely set by the user using the mask management data 122, it can be used not only for memorizing word spellings, but also for memorizing parts of speech, word meanings, and practicing Japanese-to-English and English-to-Japanese translation based on examples.

[0062] (Modification 1 of Embodiment 1) Furthermore, in the above-described embodiment 1, the electronic device 100 acquires target data by photographing learning content with the camera of the data acquisition unit 130, but the method of acquiring target data is not limited to this. For example, as a modification 1 of embodiment 1, an embodiment in which the electronic device 100 is an electronic dictionary can also be considered.

[0063] In this modified example 1, the memory unit 120 also stores various learning content (electronic data) provided by the electronic dictionary. Furthermore, the data acquisition unit 130 does not need to have a camera and acquires the electronic data of the electronic dictionary stored in the memory unit 120 as target data. Electronic dictionaries usually store various data as learning content (data such as text, video (MPEG (Moving Picture Expert Group) etc.), still images (JPEG etc.), music (MP3 (MPEG audio layer 3) etc.)), but the mask creation process and mask display process executed by the control unit 110 target character data containing specific symbols. Depending on the electronic dictionary, this character data may be stored as image data or as character code data.

[0064] If the character data is stored as image data, the control unit 110 only needs to acquire this image data from the storage unit 120 in step S101 of the mask creation process (no camera capture is required), except that the mask creation process and mask display process can be performed in basically the same manner as in Embodiment 1 described above.

[0065] If the character data is stored as character code data, in step S101 of the mask creation process, the control unit 110 only needs to acquire this character code data from the storage unit 120 (no camera capture is required). Also, in step S103, the control unit 110 does not need to perform image recognition on the target data; it only needs to recognize specific symbols, words, definitions, examples, etc., from the character code data based on that character code.

[0066] Therefore, in this case, the symbol recognition information of the specific symbol information 121 does not contain information for image recognition of the specific symbol, but rather information of the character code representing the specific symbol. The control unit 110 then recognizes the specific symbol by directly searching for the character code representing the specific symbol within the target data. Furthermore, the control unit 110 stores information about which character in the character code data to mask as the mask position data of the mask data 320, rather than coordinate data on the image data. Then, in step S204 of the mask display processing, the control unit 110 masks and displays the corresponding character based on the mask position information of the mask data 320.

[0067] Aside from the points mentioned above, the electronic device 100 according to Modification 1 of Embodiment 1 can perform mask creation processing and mask display processing with the same configuration as the electronic device 100 according to Embodiment 1. Since the electronic device 100 according to Modification 1 can create mask processing data without capturing learning content with a camera, the effort required from the user can be further reduced. In addition, since image recognition is not required, there is no possibility of misrecognizing specific symbols or characters, and mask data can be created more accurately.

[0068] (Modification 2 of Embodiment 1) Furthermore, as a modification 2 of Embodiment 1, an embodiment in which the electronic device 100 is a server that provides an online dictionary service on the internet is also conceivable. In this modification 2, as with the above-described modification 1, the data acquisition unit 130 does not need to be equipped with a camera, and acquires the electronic data of the online dictionary stored in the storage unit 120 as target data. In other respects as well, only the electronic data of the electronic dictionary is replaced with the electronic data of the online dictionary, and the control unit 110 of Modification 2 can perform mask creation processing and mask display processing in the same manner as the above-described modification 1.

[0069] The online dictionary server, which is an electronic device 100 in Modification 2, can also create masked data without taking photos of the learning content with a camera, thus reducing the effort required from the user. Furthermore, since image recognition is not required, there is no possibility of misrecognizing specific symbols or characters, and masked data can be created more accurately.

[0070] (Embodiment 2) In Embodiment 1 and its modifications described above, learning could be supported by masking a portion of the learning content. However, Embodiment 2 will be described in which the masked data is used for audio playback instead of display.

[0071] The electronic device 100 according to Embodiment 2 is, for example, an audio playback device such as a smartphone or a digital audio player. This electronic device 100 stores audio content in a storage unit 120 and plays the audio content from an audio output unit 170.

[0072] Furthermore, if the audio content stored in the memory unit 120 has mask processing data created by the mask creation process of Embodiment 1 added to it, the electronic device 100 will either mute the portion of the audio content containing the masked words or convert the language and play it back while the audio content is being played.

[0073] While various types of audio content are conceivable, in this embodiment, the audio content is assumed to be audio data recorded in MP3 format containing learning audio in a first language (e.g., English). Furthermore, while it is possible to embed image data, such as a jacket image, in an MP3 file, in this embodiment, the image data embedded in the MP3 is assumed to be masked data created in the mask creation process of Embodiment 1. This masked data includes not only words from the first language as masked words, but also their translations in a second language (e.g., Japanese). These translations can be obtained, for example, by character recognition of the definition portion of an English-Japanese dictionary when the control unit 110 creates the masked data.

[0074] The functional configuration of the electronic device 100 according to Embodiment 2 is the same as that of Embodiment 1. Therefore, the control unit 110 can perform the same processing as the mask creation process and mask display process of Embodiment 1. However, in the electronic device 100 according to Embodiment 2, the mask creation process and mask display process are not essential processes. If the mask creation process is not required, the electronic device according to Embodiment 2 does not need to be equipped with a data acquisition unit 130. Also, if the mask display process is not required, the electronic device 100 according to Embodiment 2 does not need to be equipped with a display unit 140. The essential process of the electronic device 100 according to Embodiment 2 is the audio content playback process, which will be described next.

[0075] The audio content playback process by the electronic device 100 according to Embodiment 2 will be described with reference to Figure 11. The audio content playback process starts when the user instructs the electronic device 100 to play audio content.

[0076] First, the control unit 110 acquires audio data, which is audio content (step S301). For example, the control unit 110 acquires an MP3 file stored in the storage unit 120. Alternatively, in step S301, the control unit 110 may acquire an MP3 file as audio data from an external device or the internet via the communication unit 160.

[0077] Next, the control unit 110 determines whether or not the audio data acquired in step S301 contains image data (step S302). For example, the control unit 110 determines whether or not the MP3 file acquired in step S301 has an APIC (Attached Picture) frame with ID3 tag v2 and whether or not JPEG image data is included within the APIC frame.

[0078] If the audio data does not contain image data (step S302; No), the control unit 110 plays the audio data as is (step S303) and terminates the audio content playback process.

[0079] If the audio data contains image data (step S302; Yes), the control unit 110 determines whether or not the image data contains mask data (step S304). For example, the control unit 110 determines whether or not mask data is embedded as JPEG metadata in the JPEG image data within the APIC frame of the MP3 file.

[0080] If the image data does not contain mask data (step S304; No), the control unit 110 plays the audio data as is (step S303) and terminates the audio content playback process.

[0081] If the image data contains mask data (step S304; Yes), the control unit 110 performs speech recognition on the audio data and identifies the playback positions of the mask words included in the mask data (step S305). The audio data performed speech recognition in step S305 may be the entire audio data included in the audio content, or it may be audio data extracted in a certain unit (for example, one sentence, or a certain time unit (for example, a few seconds)). If, as a result of speech recognition, the mask word exists at multiple playback positions, the control unit 110 identifies all of those playback positions.

[0082] Then, the control unit 110 replaces the audio data at the playback position of the identified mask word with synthesized audio data of the second language (e.g., Japanese) translation of the mask word (step S306). For example, if the audio data is "I teach English", the mask word in the mask data is "teach", and the translation is "to teach", the control unit 110 generates the audio data "I teach English". The control unit 110 may also replace the audio data at the playback position of the identified mask word with silence. In that case, it replaces it with silence data for the length of time the mask word is spoken (e.g., the pronunciation time of "teach"). As a result, the audio data in the above example is replaced with the audio data "I (silence) English". Alternatively, the audio data at the playback position of the identified mask word may be replaced with a sound effect such as "beep".

[0083] Then, the control unit 110 plays the replaced audio data from the audio output unit 170 (step S307). This plays the audio data in which masked words are silenced or replaced with other languages ​​(for example, Japanese). In addition, a sound effect such as "beep" may be output along with the replaced audio data to inform the user that the audio data has been replaced.

[0084] The control unit 110 then determines whether or not the audio data has been played to the end (step S308). If the audio data has not been played to the end (step S308; No), the control unit 110 returns to step S305 and performs speech recognition on the rest of the audio data. If the audio data has been played to the end (step S308; Yes), the control unit 110 terminates the audio content playback process.

[0085] Through the above audio content playback processing, the audio data of masked words is processed and replaced with silence, sound effects, or audio in another language before playback. Therefore, the electronic device 100 can effectively utilize masked data even for audio content. As a result, users can efficiently advance their learning, for example, to improve their listening skills.

[0086] (Other embodiments) In the embodiments and modifications described above, the target data was assumed to be existing paper-based learning content and electronic dictionaries, but the target data is not limited to these. For example, it may also be user-created data (image data of handwritten notes taken with a camera, data created with a word processor, etc.). In this case, the specific symbol may be an existing specific symbol, or a user-specific specific symbol may be defined and registered in the specific symbol information 121.

[0087] In this way, the electronic device 100 can automatically create problems in which predetermined parts are masked based on specific symbols, not only from existing learning content but also from data created by the user.

[0088] (Effects, etc.) Since users can freely set the mask management data 122, the electronic device 100 can automatically create mask processing data that can mask various parts of the learning content, and can flexibly create problems (learning data) according to the user's learning progress. Furthermore, the electronic device 100 can also acquire the mask management data 122 via data communication, so for example, if a teacher at a school or cram school wants to distribute common practice problems to students, the mask management data 122 set by the teacher can be distributed to each student's electronic device 100 via data communication.

[0089] Furthermore, since the specific symbol information 121 can be freely switched or expanded, the electronic device 100 can create mask data based on specific symbols not only for existing learning content such as dictionaries, reference books, and electronic dictionaries, but also for future learning content.

[0090] Furthermore, since mask conditions can be set in the mask management data 122 based on the registered contents of the vocabulary list data and the user's proficiency level, the electronic device 100 can create mask data suitable for each user according to the contents of the vocabulary list data and the user's proficiency level, even if the symbol conditions are the same.

[0091] Furthermore, since masked data is data in which the mask data is embedded as metadata within the image data, software that does not recognize metadata will not recognize the mask data and will treat it as ordinary image data. Therefore, if masked data is displayed with regular software that does not support mask data, the original image before masking will be displayed, and if it is displayed with software that supports mask data, the masked image will be displayed.

[0092] The electronic device 100 can also be implemented using a computer such as a tablet, smartphone, or PC. Specifically, in the above embodiment, it was described that the programs for the mask creation process and other processes executed by the electronic device 100 are pre-stored in the storage unit 120. However, these programs may be stored and distributed on non-temporary computer-readable recording media such as flexible disks, CD-ROMs (Compact Disc Read Only Memory), DVDs (Digital Versatile Discs), MOs (Magneto-Optical discs), memory cards, or USB memory, and a computer capable of executing the above-mentioned processes may be configured by loading and installing these programs into a computer.

[0093] Furthermore, the program can be superimposed on a carrier wave and applied via a communication medium such as the Internet. For example, the program could be posted and distributed on a bulletin board system (BBS) on a communication network. This program could then be launched and executed under the control of the operating system (OS), just like any other application program, to perform the aforementioned processes.

[0094] Furthermore, the control unit 110 may consist of any single processor, such as a single processor, multi-processor, or multi-core processor, or it may be configured by combining any of these processors with processing circuits such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array).

[0095] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these specific embodiments, and the present invention includes the invention described in the claims and its equivalents. The invention described in the original claims of this application is listed below.

[0096] (Note 1) The control unit, We obtain target data, which is learning content, and mask management data, which is data that manages a mask that hides a part of the target data. Recognizing a specific symbol included in the acquired target data, Masked data is created by adding the recognized specific symbol and the mask data based on the acquired mask management data to the acquired target data. Method for creating training data.

[0097] (Note 2) The aforementioned mask management data includes mask conditions, which are the criteria for selecting the data to be masked. The method for creating training data as described in Appendix 1.

[0098] (Note 3) The mask condition includes a symbol condition indicating which of the specified symbols is subject to masking. The method for creating training data is described in Appendix 2.

[0099] (Note 4) The aforementioned mask management data includes information from the vocabulary list registered by the user. The masking conditions include a word list condition that determines whether a word is subject to masking based on the information in the word list. The method for creating training data is described in Appendix 3.

[0100] (Note 5) The aforementioned mask management data includes information on the user's proficiency level. The masking conditions include proficiency conditions that determine whether or not to be subject to masking based on the proficiency information. The method for creating training data is described in Appendix 3.

[0101] (Note 6) The control unit, The mask management data is acquired via data communication. The method for creating training data as described in Appendix 1.

[0102] (Note 7) The aforementioned target data is image data acquired through photography. The method for creating training data as described in Appendix 1.

[0103] (Note 8) The control unit, The masked data created by the training data creation method described in any one of the appendices 1 to 7 is obtained, The mask data is obtained from the mask processing data acquired above, The aforementioned target data is displayed with a portion of it hidden based on the acquired mask data. Method for displaying training data.

[0104] (Note 9) The control unit, Acquire audio data to which the masked data created by the training data creation method described in any one of Appendix 1 to 7 has been added. The mask processing data is obtained from the acquired audio data. From the masked data obtained above, the masked words, which are the words to be masked, are extracted. The acquired audio data is subjected to speech recognition. If the speech recognition result contains the extracted masked word, when playing the audio data, the audio corresponding to the masked word in the audio data is processed and played back. Method for playing back training data.

[0105] (Note 10) The aforementioned modification involves replacing the sound with silence, sound effects, or audio in another language. The method for playing back the training data described in Appendix 9.

[0106] (Note 11) Computers We obtain target data, which is learning content, and mask management data, which is data that manages a mask that hides a part of the target data. Recognizing a specific symbol included in the acquired target data, Masked data is created by adding the recognized specific symbol and the mask data based on the acquired mask management data to the acquired target data. program.

[0107] (Note 12) Equipped with a control unit, The control unit, We obtain target data, which is learning content, and mask management data, which is data that manages a mask that hides a part of the target data. Recognizing a specific symbol included in the acquired target data, Masked data is created by adding the recognized specific symbol and the mask data based on the acquired mask management data to the acquired target data. electronic equipment. [Explanation of Symbols]

[0108] 100...Electronic device, 110...Control unit, 120...Storage unit, 121...Specific symbol information, 122...Mask management data, 130...Data acquisition unit, 140...Display unit, 150...Operation input unit, 160...Communication unit, 170...Audio output unit, 200,201...Image data, 211,212,213,214,221,222,223,224,225,226,227,230...Symbols, 300...Mask processing data, 311,312,313,314,315...Masks, 320,321,322,323,324,325...Mask data

Claims

1. The control unit, We obtain target data, which is learning content, and mask management data, which is data that manages a mask that hides a part of the target data. Recognizing a specific symbol included in the acquired target data, Mask processing data is created by adding the recognized specific symbol and the mask data based on the acquired mask management data to the acquired target data. The mask processing data created above is acquired, The mask data is obtained from the mask processing data acquired above, The aforementioned target data is displayed with a portion of it hidden based on the acquired mask data. Method for displaying training data.

2. The aforementioned mask management data includes mask conditions, which are the criteria for selecting the data to be masked. The method for displaying learning data according to claim 1.

3. The mask condition includes a symbol condition indicating which of the specified symbols is subject to masking. The method for displaying learning data according to claim 2.

4. The aforementioned mask management data includes information from the vocabulary list registered by the user. The masking conditions include a word list condition that determines whether a word is subject to masking based on the information in the word list. The method for displaying learning data according to claim 3.

5. The aforementioned mask management data includes information on the user's proficiency level. The masking conditions include proficiency conditions that determine whether or not to be subject to masking based on the proficiency information. The method for displaying learning data according to claim 3.

6. The control unit, The mask management data is acquired via data communication. The method for displaying learning data according to claim 1.

7. The aforementioned target data is image data acquired through photography. The method for displaying learning data according to claim 1.

8. Computers We obtain target data, which is learning content, and mask management data, which is data that manages a mask that hides a part of the target data. Recognizing a specific symbol included in the acquired target data, Mask processing data is created by adding the recognized specific symbol and the mask data based on the acquired mask management data to the acquired target data. The mask processing data created above is acquired, The mask data is obtained from the mask processing data acquired above, The aforementioned target data is displayed with a portion of it hidden based on the acquired mask data. program.

9. Equipped with a control unit, The control unit, We obtain target data, which is learning content, and mask management data, which is data that manages a mask that hides a part of the target data. Recognizing a specific symbol included in the acquired target data, Mask processing data is created by adding the recognized specific symbol and the mask data based on the acquired mask management data to the acquired target data. The mask processing data created above is acquired, The mask data is obtained from the mask processing data acquired above, The aforementioned target data is displayed with a portion of it hidden based on the acquired mask data. electronic equipment.

Images