Memory support processing device and memory support program

Abstract

To provide a memory support processing device that enables more effective learning by offering a clue for recalling the beginning of a word through vocal sound association, a game property and an edutainment property.SOLUTION: A memory support processing device includes: a first storage unit that stores characters of a word phrase, characters representing the designation of the word phrase, characters representing the meaning of the word phrase, a first image conceptually associated with the word phrase and a word phrase through vocal sound association represented by word play for facilitating memory of the word phrase; a character selection unit that selects, from a second storage unit, a second character that is relevant in the field that the meaning of the word phrase belongs and shares at least one word with the initial sound of the characters representing the meaning of the word phrase; an image selection unit that selects, from a third storage unit, a second image that matches in the meaning with the second character selected by the character selection unit; and an image processing unit that generates a third image by compositing the second image to the first image at a display position determined by an image position determination unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a memory assistance processing device and a memory assistance program. [Background technology]

[0002] Mnemonics that use mnemonics have been known for memorizing English words, historical dates, etc. Also, methods that use pictures to memorize English words and historical dates have been known for memorizing English words and historical dates.

[0003] Patent document 1 states, "A memory assistance method for assisting memory using a computer, comprising an image display step of displaying an image showing content to be memorized on the left side of the display screen of the computer for a predetermined period of time, and a character display step of displaying character data of language information associated with the image displayed in the image display step on the right side of the display screen of the computer for a predetermined period of time, the method repeating the above two steps to successively display a large number of images and character data, thereby assisting the memory of a person using the computer." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2003-316245 A Summary of the Invention [Problem to be solved by the invention]

[0005] The memory support method described in Patent Document 1 displays an image and characters corresponding to the image. The memory support method described in Patent Document 1 is monotonous and is not expected to be very effective for learners who have learning problems due to illnesses such as ADHD (attention deficit hyperactivity disorder) and dyslexia (reading disorder).

[0006] The objective of the present disclosure is to provide video teaching materials that are based on imagery created by applying the principles of mnemonics and that have been specially processed to provide a more intuitive memory support processing device and memory support program for learners having learning problems, by adding clues for recalling word beginnings through phonological association, as well as a game-like and edutainment element. [Means for solving the problem]

[0007] Among the inventions disclosed in this application, a brief summary of representative inventions is as follows.

[0008] According to one embodiment of the present invention, there is provided a memory support processing device for a user to memorize words and phrases, the memory support processing device having a first memory unit in which are stored characters of the word or phrase, characters indicating the pronunciation of the word or phrase, characters indicating the meaning of the word or phrase, a first image conceptually linked to the word or phrase, and words and phrases based on phonetic associations, typically mnemonics, to make it easier to memorize the word or phrase. The memory support processing device also includes a character selection unit that selects from the second memory unit a second character that is related in the field to which the characters indicating the meaning of the phrase belong and has at least one character that matches the initial sound of the characters indicating the meaning of the phrase; an image selection unit that selects from the third memory unit a second image that matches in meaning with the second characters selected by the character selection unit; an image position determination unit that determines a display position of the second image selected by the image selection unit relative to the first image; an image processing unit that generates a third image by combining the second image with the first image at the display position determined by the image position determination unit; and a display unit that is capable of displaying the characters of the phrase, the characters indicating the pronunciation of the phrase, the characters indicating the meaning of the phrase, and the third image generated by the image processing unit. Effect of the Invention

[0009] According to the present disclosure, the teaching materials are image data and video data created by applying the principles of mnemonics, and a series of special processes are added to them, making them intuitive even for learners who have problems with learning, and by adding clues for recalling the beginnings of words through phonological association, as well as game-like and edutainment properties, they have the effect of making learning more beneficial. [Brief description of the drawings]

[0010] [Figure 1] 1 is a configuration diagram of an information processing system according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram of an information processing device according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a block diagram of a storage unit according to an embodiment of the present invention. [Figure 4] 1 is a screen example (1) of a storage unit according to an embodiment of the present invention. [Diagram 5] 1 is an example (1) of a memory assistance processing device according to an embodiment of the present invention. [Figure 6] 1 is an example (2) of a memory assistance processing device according to an embodiment of the present invention. [Figure 7] 1 is an example (3) of a memory assistance processing device according to an embodiment of the present invention. [Figure 8] 1 is an example (4) of a memory assistance processing device according to an embodiment of the present invention. [Figure 9] 13 is an example (5) of a memory assistance processing device according to an embodiment of the present invention. [Figure 10] 1 is a flowchart (1) of image processing according to an embodiment of the present invention. [Figure 11] 13 is a flowchart (2) of image processing according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. Note that this is merely an example, and the technical scope of the present disclosure is not limited to this example. In addition, in all drawings for explaining the embodiment, the same parts are generally designated by the same reference numerals, and repeated explanations thereof will be omitted. According to one embodiment of the present invention, the memory support processing device 100 is for a user to memorize words and phrases, and includes a first memory unit 1021 in which are stored letters of a word or phrase, letters indicating the pronunciation of the word or phrase, letters indicating the meaning of the word or phrase, a first image conceptually linked to the word or phrase, and words and phrases based on phonetic associations, typically mnemonics, for facilitating memorization of the word or phrase. The memory support processing device 100 also includes a character selection unit 105 that selects from the second memory unit 1022 a second character that is related in the field to which the meaning of the phrase belongs and has at least one character that matches the initial sound of the characters indicating the meaning of the phrase, an image selection unit 106 that selects from the third memory unit 1023 a second image that matches the meaning of the second character selected by the character selection unit 105, an image position determination unit 107 that determines a display position of the second image selected by the image selection unit 106 relative to the first image, an image processing unit 108 that generates a third image by combining the second image with the first image at the display position determined by the image position determination unit 107, and a display unit 104 that enables the characters of the phrase, the characters indicating the pronunciation of the phrase, the characters indicating the meaning of the phrase, and the third image generated by the image processing unit 108 to be displayed. The memory assistance processing device 100 includes an image processing unit 108 that performs processing to cause the second image in the third image to blink. The memory assistance processing device 100 includes an object image recognition unit 109 that recognizes a fourth image of an object displayed in the first video, a character conversion unit 110 that converts the name of the fourth image recognized by the object recognition unit into a third character, an image identification unit 111 that identifies a fourth image corresponding to the third character selected by the character selection unit 105, and a marking processing unit 112 that applies a circular marking to the fourth image identified by the image identification unit 111. The memory assistance processing device 100 includes an image dividing section 113 that divides the first image into a plurality of images so that the fourth image selected by the image selection section 106 fits within one divided screen. <System configuration> FIG. 1 is a diagram illustrating an overview of an information processing system according to an embodiment of the present disclosure.

[0012] 1 is a configuration diagram of an information processing system 1. The information processing system 1 is composed of a memory assistance processing device 100, an information processing terminal device 120, an EC server 130, and a communication network 140. The memory assistance processing device 100, the information processing terminal device 120, and the EC server 130 are connected to each other via the communication network 140.

[0013] The memory assistance processing device 100 is a PC device for a general server, and plays a central role in the present disclosure. When accessing the memory assistance-processing device 100, a predetermined authentication, for example, ID information and password information, is required, and access is possible by checking the ID information against the password information. The memory support processing device 100 performs processes such as image generation processing and character search processing for words and phrases in which one or more initial characters match. A phrase is a term that includes a word, a phrase, or a short sentence.

[0014] The information processing terminal device 120 is a typical home PC device, and is a terminal device that plays a central role in this disclosure. The information processing terminal device 120 issues commands to the memory support processing device 100 and causes the memory support processing device 100 to perform various processes.

[0015] The communication network 140 includes the Internet, an intranet, a wireless LAN, a 4G or 5G mobile phone system network, and a short-range wireless network.

[0016] 2 is a block diagram of the memory support processing device 100. The memory support processing device 100 includes a control unit 101, a storage unit 102, a communication unit 103, a display unit 104, a character selection unit 105, an image selection unit 106, an image position determination unit 107, an image processing unit 108, an object image recognition unit 109, a character conversion unit 110, an image specification unit 111, a marking processing unit 112, and a screen division unit 113.

[0017] The control unit 101 is mainly a CPU. The control unit 101 executes applications (programs) installed in the memory support processing device 100, and controls the memory support processing device 100. The control unit 101 issues commands for image generation processing, character search processing for words and phrases in which one or more initial characters match, and various other processing.

[0018] In addition to installed applications (programs), predetermined data is stored in the storage unit 102. The storage unit 102 includes a memory as a temporary storage, and an HDD or SSD as a permanent storage. The memory functions as a work area when the control unit 101 performs processing. The memory is, for example, a dynamic random access memory (DRAM) or a static random access memory (SRAM). In the present disclosure, in addition to conventionally used character data for memory assistance, image data, and word data based on phonetic association such as mnemonics, image data processed in the present disclosure and the like are also stored.

[0019] The communication unit 103 allows the memory support processing device 100 to exchange information with the outside of the memory support processing device 100. Specifically, the communication unit 103 is connected to the outside of the memory support processing device 100 via a cable such as optical fiber or Ethernet.

[0020] The display unit 104 outputs character data, image data, etc. stored in the storage unit 102. Specifically, the display unit 104 is a liquid crystal display or the like.

[0021] The character selection unit 105 selects from the second storage unit 1022 second characters that are related to the field to which the meaning of the phrase the user wishes to remember belongs and that have at least one initial character that matches the initial sound of the character indicating the meaning of the phrase. For example, if there is the word "Composer," in the field of "music" to which the word "Composer" belongs, the second character "sakkusu" may be selected, which matches the initial sound "sa" of "sakkyokuka," which is the meaning of the word "Composer," and the next sound "tsu."

[0022] Image selection unit 106 searches third storage unit 1023 for a graphic that matches the second character selected by character selection unit 105. If image selection unit 106 finds a second image that matches the second character selected by character selection unit 105, image selection unit 106 selects the second image. If image selection unit 106 does not find a second image that matches the second character selected by character selection unit 105, image selection unit 106 reselects the second character in character selection unit 105.

[0023] An image position determination unit 107 determines the relative position of the selected second image with respect to the first image. The second image may be positioned at the center of the first image. The relative position may be determined by using a bounding box based on object recognition to position the second image in the first image.

[0024] The image processing unit 108 performs processing to superimpose and synthesize the second image onto the first image at the relative position of the second image with respect to the first image determined by the image position determination unit 107, thereby generating a new third image. The image processing unit 108 performs processing to superimpose the second image on the first image and blink the second image. The image processing unit 108 may process the first image and the second image as stereoscopic images. The image processor 108 performs processing such as pausing for a certain period of time, or changing the playback speed to 0.5 or 0.75 times the normal speed. This is because it can be used for recall training by pausing before a scene that is desired to be impressed, or by making it possible to adjust the display time of the image.

[0025] The object image recognition unit 109 recognizes objects in a video. The object image recognition unit 109 utilizes object detection and object recognition by deep learning. Specifically, the object image recognition unit 109 can detect objects in a video in real time by surrounding the object with a boundary box using YOLO (You ONLY Look Once) or SSD (Single Shot MultiBox Detector). The object image recognition unit 109 recognizes the object detected in the boundary box.

[0026] The character conversion unit 110 searches for the name of an object based on the image of the object recognized by the object image recognition unit 109. The character conversion unit 110 may provide several candidates for the name of the object. For example, when an image of a hand is recognized, the words might be "te (hand)," "ude (arm)," "joushi (upper limb)," etc.

[0027] The image specification unit 111 specifies a fourth image corresponding to the third character selected by the character selection unit 105. If the image specification unit 111 cannot specify the fourth image from the fourth storage unit 1024, the image specification unit 111 causes the character conversion unit 110 to reselect the fourth character.

[0028] The marking processing unit 112 performs a circular marking process on the image. More specifically, the marking processing unit 112 performs a circular marking process on the image so as to surround the boundary box of the image specified by the image specifying unit 111. The marking processing unit 112 is also capable of performing masking processing and mosaic processing in addition to marking.

[0029] The image division unit 113 divides the first image into a plurality of images. The number of divisions by the image division unit 113 is 2, 3, 4, 6, 8, 9, 12, or the like. The image division unit 113 not only divides the image, but also uses a boundary box to divide the image so that the object image recognized by the object image recognition unit 109 fits within a partition of the divided image.

[0030] 3 is a conceptual diagram of the storage unit 102. Data stored in the storage unit 102 can be viewed, copied, edited, or deleted upon receiving a command from the control unit 101. The storage unit 102 is composed of a first storage unit 1021, a second storage unit 1022, a third storage unit 1023, and a fourth storage unit 1024. The first storage unit 1021, the second storage unit 1022, the third storage unit 1023, and the fourth storage unit 1024 are connected to each other via a bus.

[0031] The first storage unit 1021 stores information related to existing mnemonics. Specifically, this information includes image data, video data, and the like, for facilitating memorization for the user. Furthermore, the first storage unit 1021 also stores separately character data written in the image data and image data of pictures and photographs drawn in the image data.

[0032] Second character data is stored in second storage unit 1022. Second storage unit 1022 stores general names, as well as abbreviations, puns, buzzwords, and the like.

[0033] The second image data is stored in the third storage unit 1023. The third storage unit 1023 stores the second image data corresponding to the second character data.

[0034] The fourth storage unit 1024 stores the third character data and the fourth image data.

[0035] The fifth storage unit 1025 stores the synthesized third image data, the marking-processed fourth image data, and video data.

[0036] 4 is a conceptual diagram of the character data stored in the second storage unit 1022. The character data stored in the second storage unit 1022 is classified by category. For example, there are "architectures", "automobiles", "stationery", "music", "fruit", "sea creatures", etc. In the example of "sakkyokuka," the composer is closest in meaning to the "music" category. Within the "music" category, search for characters starting with "sa" or "sa." This will return results such as "saxophone," "sakushi," and "sakushika." In addition, the system may be designed to exclude "sakkyoku", which can also be used as an abbreviation for "sakkyokuka".

[0037] FIG. 5 is a diagram of conventional image data 200 for ease of storage according to one embodiment of the present disclosure. Word characters 201 are the textual representation of the word the user wishes to remember. The pronunciation 202 of a word is a character representation showing the pronunciation or pronunciation of the characters 201 of the word. The meaning of the word 203 is a textual representation that indicates the meaning of the characters 201 of the word.

[0038] The image 204 is central image data for facilitating storage in the image data 200. The image 204 can be recognized by object recognition using deep learning. Image 205 is image data that is determined to be a person or a musician through object recognition using deep learning. Image 206 is image data that is determined to be musical notation or sheet music through object recognition using deep learning.

[0039] Image 207 is image data that is determined to be packaging material through object recognition using deep learning. Image 208 is image data that is determined to be cardboard or packaging material through object recognition using deep learning.

[0040] FIG. 6 is an image for ease of memorization for a user of one embodiment of the present disclosure. The image on the left is a conventional image, and the image on the right is an image according to the present disclosure. The difference between the two images is the image of a saxophone located near the center of the right image (dotted box).

[0041] In conventional image data, it is difficult for a user to recall the word "composer," which is one of the meanings of "Composer," simply by looking at the text notation of "Composer" and the image 204.

[0042] In the present disclosure, an image 209 of "saxophone" is synthesized in the center of the image 204. This image 209 of the saxophone naturally evokes the initial "sa" or "sa" in the user's mind. When the initial part of the word is evoked in the user's mind, the word "composer" that starts with "sa" or "sa" comes to mind.

[0043] FIG. 7 is an image for ease of memorization for a user of one embodiment of the present disclosure. The image on the left is conventional image data, and the image on the right is image data according to the present disclosure. The difference between the two images is the image data of a diamond located near the center of the image on the right (circled by a dotted line).

[0044] With conventional image data, it was difficult for users to recall one of the meanings of Cherish, "to cherish," just by looking at the text and image of Cherish.

[0045] In the present disclosure, image data of a "diamond" is synthesized near the center of the image data (encircled by a dotted line). This diamond image data naturally evokes the initial "da" or "dai" in the user's mind. When the initial of a word is evoked in the user's mind, words beginning with "da" or "dai" such as "daiji" come to mind.

[0046] FIG. 8 is an image for ease of memorization for a user of one embodiment of the present disclosure. The image on the left is conventional image data, and the image on the right is image data according to the present disclosure. The difference between the two images is that the image data on the right is divided. By dividing the image data, it becomes easier to identify objects in the image data, and it has the effect of evoking the meaning of words.

[0047] With conventional image data, it was difficult for users to recall the word "diligence," one of the meanings of "Industry," simply by looking at the written word and image of "Industry."

[0048] In this disclosure, the image data is divided into nine parts. Kinbin (golden bin), a pun or paraphrase of diligence, is located in the lower center of the divided image data. This divided image data naturally brings to the user's mind the initial part of the word "kinbin." When "kinbin" is brought to the user's mind, the word "diligence" comes to mind.

[0049] FIG. 9 is an animation for facilitating memorization by a user of one embodiment of the present disclosure. A video is a series of consecutive images.

[0050] With conventional video data, it was difficult for users to recall the meaning of "suitable" simply by seeing the text and video of "Suitable."

[0051] In the present disclosure, the marking process is applied to the final image data of the video. In the present disclosure, the marking process is applied to image data of "hand" which corresponds to the initial sound "te" of "suitable," one of the meanings of "suitable."

[0052] The object image recognition unit 109 performs object recognition processing of deep learning to recognize objects in the video. Then, the character conversion unit 110 replaces the objects recognized by the object image recognition unit 109 with characters. Furthermore, the character selection unit 105 searches for characters that match "te" and "teki," which are the initial words of "suitable," one of the meanings of "suitable." If there are matching characters, the image specification unit 111 selects an image corresponding to the characters from the fourth storage unit 1024. The marking processing unit 112 performs circular marking processing that surrounds a boundary box for the image.

[0053] FIG. 10 is a flow chart according to one embodiment of the present disclosure. The control unit 101 receives a message from the information processing terminal device 120 and acquires information used in conventional mnemonics from the storage unit 102 (S2). The information used in conventional mnemonics is a collection of letters of a word, letters indicating the pronunciation of the word, letters indicating the meaning of the word, a first image conceptually linked to the word, and a mnemonic for making it easier to remember the word.

[0054] The character selection unit 105 classifies the meaning of the word that the user wants to memorize into a field (category) to which the meaning belongs (S4, FIG. 4). The category may be a broad classification or a medium classification divided into some small parts.

[0055] The character selection unit 105 selects second characters, at least one of which matches the beginning of the characters indicating the meaning of the word, from the category to which the meaning of the word belongs in the second storage unit 1022 (S6). The character selection unit 105 selects second characters that match only one character at the beginning of a word, second characters that match only two characters at the beginning of a word, etc. The character selection unit 105 may list the second characters selected by the character selection unit 105 in order of the number of matching characters.

[0056] The image selection unit 106 selects from the third storage unit 1023 a second image whose meaning matches the second character selected by the character selection unit 105 (S8). If the second character is not present in the third image stored in the third memory, the third image may be generated from the second character using AI technology such as Craiyon.

[0057] If there is no second image whose meaning matches the second character selected by the character selection unit 105, the image selection unit 106 returns to the character selection unit 105 and selects the second character.

[0058] The image position determination unit 107 determines the relative position of the second image to the first image (S10). The size of the second image is smaller than the size of the first image, and the size ratio of the first image to the second image is 1 / 20 to 1 / 4. The second image can be positioned so that the center of the second image is located at the center of the first image.

[0059] The image processing unit 108 creates a third image by superimposing and synthesizing the second image on the first image. The image processing unit 108 may change the color, shape, aspect ratio, etc. during the synthesis process (S12).

[0060] The image processing unit 108 can add a process of blinking only the second image in the third image. The image processing unit 108 may use a method of alternately switching the display of the first image and the third image (S12). The image processing unit 108 can perform processing to make the third image into a stereoscopic image.

[0061] The control unit 101 displays the third image processed by the image processing unit 108 on the display unit of the information processing terminal device 120 (S16). The control unit 101 displays the third image on the display unit of the information processing terminal device 120 even when the image dividing unit 113 divides the screen.

[0062] FIG. 11 is a flow chart according to one embodiment of the present disclosure. The control unit 101 receives a message from the information processing terminal device 120 and acquires information used in conventional mnemonics from the storage unit 102 (S20). The information used in conventional mnemonics is a collection of letters of a word, letters indicating the pronunciation of the word, letters indicating the meaning of the word, a first video conceptually linked to the word, and a mnemonic for making it easier to remember the word.

[0063] The object image recognition unit 109 recognizes objects in the first moving image. Since a moving image is a collection of images, objects in the images are recognized (S22). The object image recognition unit 109 uses deep learning, specifically, YOLO (You ONLY Look Once) or SSD (Single Shot MultiBox Detector), to surround an object with a boundary box and detect and recognize an object in a video in real time. The image recognized by the object image recognition unit 109 is stored in the fourth memory unit 1024 .

[0064] The character conversion unit 110 converts the name of the image recognized by the object image recognition unit 109 into characters (S24). For example, in the case of an image of a car, the name is a noun such as "car," "automobile," or "passenger car." Multiple names may be given. The characters converted by the character conversion unit 110 are stored in the fourth memory unit 1024 .

[0065] The character selection unit 105 selects from the fourth storage unit 1024 a third character having at least one character that matches the initial sound of the character indicating the meaning of the word (S26). For example, the fourth storage unit 1024 is searched for and selected as a character that matches the initial sound "te" of "tekishita", which is one of the meanings of "Suitable". The fourth storage unit 1024 is searched for and selected as a character that matches the initial sound "teki" of "tekishita", which is one of the meanings of "suitable".

[0066] The image selection unit 106 selects an image corresponding to the character selected by the character selection unit 105 from the fourth storage unit 1024 (S28). If the image selection unit 106 can select an image, the image specification unit 111 specifies the image (S30). The image specified by the image specification unit 111 is an image in the video recognized by the object image recognition unit 109.

[0067] The marking processing unit 112 performs a marking process on the image specified by the image specification unit 111 (S32). Specifically, the marking processing unit 112 performs a circular marking process that surrounds the boundary box used when the object image recognition unit 109 recognizes the object.

[0068] The marking processing unit 112 performs a masking process instead of a marking process. The masking process means a blurring process, a mosaic process, or a filter process. The masking process has an effect different from the marking process in that it enhances the user's imagination and stimulates the curiosity by making the object invisible visually. For example, applying masking to important blocks of a segmented image will have the effect of encouraging recall of those parts, making it possible to train the efficient recall of important parts.

[0069] The image processing unit 108 can add a process of blinking the image marked by the marking processing unit 112 (S34). The image processing unit 108 can perform processing to create a stereoscopic image. The image processing unit 108 can perform a process of adjusting the playback speed of the video. For example, when the second image appears on the screen, the image can be paused for a certain period of time, or the playback speed can be set to 0.5 or 0.75 times the normal speed. This is because the image can be used for recall training by pausing the image before a scene that is desired to be impressed, or by adjusting the display time of the image.

[0070] The control unit 101 displays the moving image processed by the image processing unit 108 on the display unit of the information processing terminal device 120 (S36).

[0071] <Other embodiments> The words may be Japanese words or phrases or foreign words or phrases, numbers such as historical dates, or words or phrases related to geography, chemistry, biology, law, etc. Audio and text information, music and sound effects as background music may be added to video data created using the memory assistance processing device according to the present disclosure and the memory assistance processing program according to the present disclosure. The video data created using the memory assistance processing device and the memory assistance processing program according to the present disclosure may be a stereoscopic image (3D image) or a stereoscopic video (3D video) that the user can change the viewing angle from. Video data and video data created using the memory assistance processing device according to the present disclosure and the memory assistance processing program according to the present disclosure may be sold as educational materials on an EC site. Video data created using the memory support processing device and the memory support processing program according to the present disclosure may be printed and bound into paper media and sold as teaching materials. In this case, a QR code (registered trademark) may be printed in the required location to allow the user to obtain video information and peripheral information via an information terminal such as a smartphone.

[0072] <Effects of this embodiment>

[0073] According to the present disclosure, the teaching materials are image data and video data created by applying the principles of mnemonics, and a series of special processes are added to them, making them intuitive even for learners who have problems with learning, and by adding clues for recalling the beginnings of words through phonological association, as well as game-like and edutainment properties, they have the effect of making learning more beneficial. In particular, compared to the previous method of using image pictures, the addition of images that evoke the beginning of the word to be recalled increases the probability of arriving at the correct answer through the word-initial priming effect in psychology. It also has the effect of encouraging attention and maintaining concentration for learners who have learning problems due to illnesses such as ADHD (attention deficit hyperactivity disorder) and dyslexia (reading disorder). [Explanation of symbols]

[0074] 1. Information Processing Systems 100 Information processing device 101 Control section 102 Storage section 103 Communications Department 104 Display section 105 Character Selection Section 106 Image Selection Section 107 Image position determination unit 108 Image Processing Unit 109 Object Image Recognition Unit 110 Character conversion section 111 Image Identification Unit 112 Marking Processing Section 113 Image division unit 1021 First memory unit 1022 Second memory section 1023 Third Memory Section 1024 4th memory section 1025 5th Memory Section

Claims

1. A memory assistance processing device for a user to memorize words and phrases, comprising: a first storage unit in which characters of the phrase, characters indicating the pronunciation of the phrase, characters indicating the meaning of the phrase, a first image conceptually linked to the phrase, and a phrase based on phonetic association, typically a mnemonic, for facilitating memorization of the phrase; a character selection unit that selects from a second storage unit second characters that are related to the field to which the characters indicating the meaning of the phrase belong and have at least one initial character that matches the initial sound of the characters indicating the meaning of the phrase; an image selection unit that selects from a third storage unit a second image whose meaning matches the second character selected by the character selection unit; an image position determination unit that determines a display position of the second image selected by the image selection unit relative to the first image; an image processing unit that generates a third image by combining the second image with the first image at the display position determined by the image position determination unit; a display unit that displays the characters of the phrase, the characters indicating the pronunciation of the phrase, the characters indicating the meaning of the phrase, and a third image generated by the image processing unit; A memory support processing device comprising:

2. The memory assistance processing device according to claim 1 , further comprising the image processing unit performing processing for blinking the second image in the third image.

3. A memory assistance processing device for a user to memorize words and phrases, comprising: a first storage unit in which characters of the phrase, characters indicating the pronunciation of the phrase, characters indicating the meaning of the phrase, a first video conceptually linked to the phrase, and a mnemonic for facilitating memorization of the phrase are stored; an object image recognition unit that recognizes a fourth image of an object displayed in the first moving image; a character conversion unit that converts the name of the fourth image recognized by the object image recognition unit into a third character; a fourth storage unit in which a fourth image recognized by the object image recognition unit and the third character converted by the character conversion unit are stored; the character selection unit selecting from the fourth storage unit at least one of the third characters that matches an initial sound of a character that indicates the meaning of the phrase; an image specification unit that specifies the fourth image corresponding to the third character selected by the character selection unit; a marking processing unit that applies a circular marking to the fourth image identified by the image identification unit; A memory support processing device comprising:

4. A memory assistance processing device for a user to memorize words and phrases, comprising: a first storage unit in which are stored characters of the phrase, characters indicating the pronunciation of the phrase, characters indicating the meaning of the phrase, a first image conceptually linked to the phrase, and a phrase based on phonetic association, typically a mnemonic device, for facilitating memorization of the phrase; an object image recognition unit that recognizes an image of an object displayed in the first image; a character conversion unit that converts the name of the object recognized by the object image recognition unit into characters; the image selection unit selecting a fifth image in which the initial sound of the characters converted by the character conversion unit matches at least one initial sound of the keyword in the mnemonic; an image division unit that divides the first image into a plurality of images so that the fourth image selected by the image selection unit fits within one divided screen; a display unit that displays a boundary line between the images divided by the image dividing unit; A memory support processing device comprising:

5. A memory assistance program for a user to memorize words and phrases, comprising: Computer, a first storage unit in which are stored characters of the phrase, characters indicating the pronunciation of the phrase, characters indicating the meaning of the phrase, a first image conceptually linked to the phrase, and a phrase based on phonetic association, typically a mnemonic device, for facilitating memorization of the phrase; a character selection unit that selects from a second storage unit second characters that are related to the field to which the characters indicating the meaning of the phrase belong and have at least one initial character that matches the initial sound of the characters indicating the meaning of the phrase; an image selection unit that selects from a third storage unit a second image whose meaning matches the second character selected by the character selection unit; an image position determination unit that determines a display position of the second image selected by the image selection unit relative to the first image; an image processing unit that generates a third image by combining the second image with the first image at the display position determined by the image position determination unit; a display unit that displays the characters of the phrase, the characters indicating the pronunciation of the phrase, the characters indicating the meaning of the phrase, and a third image generated by the image processing unit; A memory support program that functions as a

6. In claim 5, A memory assistance program that functions as the image processing unit to perform a process of blinking the second image in the third image.

7. A memory assistance program for a user to memorize words and phrases, comprising: Computer, a first storage unit in which are stored characters of the phrase, characters indicating the pronunciation of the phrase, characters indicating the meaning of the phrase, a first video conceptually linked to the phrase, and a phrase based on phonetic association, typically a mnemonic device, for facilitating memorization of the phrase; an object image recognition unit that recognizes a fourth image of an object displayed in the first moving image; a character conversion unit that converts the name of the fourth image recognized by the object image recognition unit into a third character; a fourth storage unit in which a fourth image recognized by the object image recognition unit and the third character converted by the character conversion unit are stored; the character selection unit selecting from the fourth storage unit at least one of the third characters that matches an initial sound of a character that indicates the meaning of the phrase; an image specification unit that specifies the fourth image corresponding to the third character selected by the character selection unit; a marking processing unit that applies a circular marking to the fourth image identified by the image identification unit; a display unit that displays the characters of the phrase, the characters indicating the pronunciation of the phrase, the characters indicating the meaning of the phrase, and a fourth image specified by the image specification unit; A memory support program that functions as a

8. A memory assistance program for a user to memorize words and phrases, comprising: Computer, a first storage unit in which are stored characters of the phrase, characters indicating the pronunciation of the phrase, characters indicating the meaning of the phrase, a first image conceptually linked to the phrase, and a phrase based on phonetic association, typically a mnemonic device, for facilitating memorization of the phrase; an object image recognition unit that recognizes an image of an object displayed in the first image; a character conversion unit that converts the name of the object recognized by the object image recognition unit into characters; the image selection unit selecting a fifth image in which the initial sound of the characters converted by the character conversion unit matches at least one initial sound of the keyword in the mnemonic; an image division unit that divides the first image into a plurality of images so that the fourth image selected by the image selection unit fits within one divided screen; a display unit that displays a boundary line between the images divided by the image dividing unit; A memory support program that functions as a

Images