Information retrieval system and program

Abstract

To provide an information retrieval system capable of implementing efficient information retrieval in accordance with experience or preference of each user.SOLUTION: An information retrieval system 10 is configured, in retrieving information related to a word on a screen of display means 60, to display hierarchical images while screen-transitioning from higher to lower layers according to hierarchical information. An arbitrary image is reached by the above retrieval with screen transition. The system receives, from the image, a registration for allowing a user to perform transition to a designated transition destination image. Accordingly, the system enables transition to an image of a layer immediately below the above reached image through selection operation of an image selection unit, the image being included in the reached image, and transition not limited to transition to return to the previous image of the reached image on a search path leading to the reached image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information retrieval system and a program configured by a computer that retrieves information related to a word on the screen of a display means. For example, when retrieving the spelling of a word indicated by an image, an explanation of the semantic content, the pronunciation of a word, a manufacturing method of an object corresponding to a word, or when solving a problem including an image schema learning problem in an image of a space or an object and learning a language other than the mother tongue such as Chinese, French, or English, etc., it can be used.

Background Art

[0002] Conventionally, an electronic dictionary using a computer has been used. However, for the search performed to obtain target information in such an electronic dictionary, there are searches by the number of strokes, pronunciation, radicals, triangular codes, square codes, etc. Note that the search by triangular codes and square codes is a search method peculiar to Chinese (Chinese characters).

[0003] Also, usually, one dictionary performs information conversion (translation) between two languages, such as between Japanese and Chinese, and furthermore, the direction of conversion is also determined, such as from Japanese to Chinese or from Chinese to Japanese. Therefore, when purchasing a Japanese-Chinese dictionary, it can only be used for the purpose of conversion from Japanese to Chinese, and it cannot be used as a Chinese-Japanese dictionary. Furthermore, it cannot be used for the purpose of conversion between completely different languages, such as an English-French dictionary or a French-English dictionary, etc.

[0004] In the above-mentioned conventional search method using electronic dictionaries, the number of strokes and pronunciation must be specified as search keys, which can be time-consuming and difficult to use. For example, if you know the spelling (how to write) of a Chinese word but not its pronunciation (how to read it), you can search for the word by the number of strokes to find out its pronunciation, but since you need knowledge to count the number of strokes in kanji in the first place, the search can be time-consuming. Therefore, in the case of children with little knowledge, there is a risk that they will not be able to perform a search. This problem is not limited to searches by number of strokes, but also applies to searches by pronunciation, radicals, triangular code, square code, etc.

[0005] Also, for example, when an object exists and the spelling (how to write) and pronunciation (how to read) of the Chinese word for the object are unknown, in order to search for the Chinese word, it is necessary to know the word for the object in another language (usually the native language of the searcher). For example, if there is an object that erases characters written on paper with a pencil, unless one knows that the object is called "eraser" in Japanese or "eraser" in English, one cannot search for and find the Chinese character "橡皮" and its pronunciation, which are the Chinese words with the same meaning. In other words, if one knows "eraser" in Japanese or "eraser" in English, one can use it as a key to perform a search, but if one simply recognizes the existence of the object itself or its use or attributes, one cannot obtain a key for searching and therefore cannot arrive at "橡皮" and its pronunciation. Therefore, in the case of children with little knowledge, there is a risk that the conventional search system will not be useful, and there is also a problem that the conventional search system is not suitable for the purpose of learning one's native language or a foreign language by such a person with little knowledge.

[0006] Furthermore, as mentioned above, typically one dictionary only converts information between two languages, so a system that can convert information between more languages ​​is desired.

[0007] Therefore, in order to solve these problems, an information retrieval system using images such as hierarchical spaces has been proposed (see Patent Document 1). The applicant of the present application has been developing and selling this information retrieval system.

Prior Art Documents

Patent Documents

[0008]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0009] In the information retrieval system of Patent Document 1 described above, the search is performed according to the hierarchy predetermined by the system designers and developers. That is, since the hierarchical information is created based on a space concept common to many people, the search according to the standard space concept is performed.

[0010] However, the space concepts held by individuals are not uniform. Since the experiences and their order that a person has from birth are different, there are naturally individual differences in the connection between spaces and the strength of the connection between a space and the objects belonging to it. For example, when there is a tennis ball, as the space to which the tennis ball belongs, some people first associate the tennis court, while others associate the tennis shop or the tennis goods section of a sports store. Therefore, the search according to the hierarchical information in the information retrieval system of Patent Document 1 described above may not be efficient for some people.

[0011] Also, in the information retrieval system of Patent Document 1 described above, the search starts from the topmost hierarchy. However, as the user's language learning progresses to a certain extent or, in the case of a child user, as they grow, the search and learning purposes become more focused, and for each user, favorite spaces or frequently visited spaces emerge. Therefore, it is not always necessary to start the search from the topmost hierarchy.

[0012] An object of the present invention is to provide an information retrieval system and a program capable of realizing efficient information retrieval according to the experience and preferences of individual users.

Means for Solving the Problems

[0013] The present invention is an information retrieval system configured by a computer that retrieves information related to words on the screen of a display means, image data storage means for storing a plurality of image data for respectively displaying images of a plurality of spaces and a plurality of image data for respectively displaying images of a plurality of objects belonging to at least one of these spaces or their constituent articles; explanation data storage means for storing explanation data of words indicating spaces and explanation data of words indicating objects or their constituent articles for one or more languages selected by the user on a setting screen displayed on the screen of the display means; hierarchical information storage means for storing hierarchical information for hierarchically arranging each image data in a state of passing through the image of the space at least once from the highest layer to the lowest layer according to any one of the subordinate relationships between spaces, the subordinate relationship between a space and an object, or the subordinate relationship between an object and its constituent articles; selection reception processing means for receiving a selection operation of an image selection part by the user on the screen of the display means on which an image of a space including an image selection part for selecting an image of a space or an object belonging to the immediately lower layer or an image of an object including an image selection part for selecting an image of a constituent article is displayed; screen switching processing means for switching the screen displayed on the display means to a screen for displaying an image of a space or an object or its constituent article corresponding to the image selection part selected by the user's selection operation, using the image data stored in the image data storage means according to the hierarchical information stored in the hierarchical information storage means; In accordance with the screen switching process by this screen switching processing means, using the explanatory data stored in the explanatory data storage means, on the same screen as the switched screen where an image of a space or object or its component articles corresponding to the image selection section selected by the user's selection operation is displayed, there is provided an explanatory display processing means for displaying an explanation of a word indicating the space or object or its component articles corresponding to the image selection section selected by the user's selection operation. The processing by the selection reception processing means, the processing by the screen switching processing means, and the processing by the explanatory display processing means are configured to be repeated such that, as the user repeatedly selects the image selection section, in accordance with the hierarchical information stored in the hierarchical information storage means, images of spaces or objects or their component articles, as well as explanations of words indicating the spaces or objects or their component articles, are repeatedly displayed on the screen of the display means in a state of proceeding from a higher-level hierarchy to a lower-level hierarchy. Furthermore, in order to realize a transition to a designated transition destination image that is not limited to a transition to an image of the next lower level of the reached image by a selection operation of the image selection section included in this reached image, and a transition to the image immediately preceding the reached image in the search path to the reached image, from any reached image while transitioning from a higher-level hierarchy to a lower-level hierarchy, there is provided a designated transition destination image registration means for receiving registration of a designated transition destination image by the user and storing the identification information of the designated transition destination image in the designated transition destination image storage means in association with the identification information of the reached image. The selection reception processing means is also configured to receive a selection operation of the user for a transition to a designated transition destination image. The screen switching processing means is characterized in that it is also configured to execute a process of switching the screen for displaying the reached image to the screen for displaying the designated transition destination image in accordance with the selection operation of the user for a transition to the designated transition destination image received by the selection reception processing means, using the designated transition destination image in which the identification information is stored in the designated transition destination image storage means in association with the identification information of the reached image.

[0014] In such an information retrieval system of the present invention, not only is the search for information related to a word advanced from a higher-level hierarchy to a lower-level hierarchy according to the hierarchical information stored in the hierarchical information storage means, but also a transition to a designated transition destination image registered by the user in association with this reached image is made from any reached image reached by the search. For this reason, the user can construct on the system the association between spaces, the association between a space and an object, or the association between an object and its constituent articles according to their own associations, and perform a search according to the association. The registration of these associations by the user can be considered in parallel with the process in which synapses are formed in the user's brain. Therefore, it becomes possible to realize an efficient information search according to the experience and preferences of each user.

[0015] In addition, the present invention can also inherit the effects obtained by the conventional information retrieval system described in the above-mentioned Patent Document 1. That is, in order to obtain target information (information related to a word), the user performs a selection operation on the image selection part of the immediately lower hierarchy included in the image on the screen of the display means, thereby passing through an image of a space at least once and sequentially switching to a screen for displaying images of lower hierarchies. Then, such a selection operation by the user and the screen switching according to the operation are repeated until the target information is obtained. At this time, the selection operation of the user is received by the selection reception processing means, and the screen switching is performed by the screen switching processing means according to the hierarchical information stored in the hierarchical information storage means.

[0016] For this reason, the user can obtain target information by a visual action using the space as a clue by performing a search that passes through an image of a space at least once. Therefore, just by performing a selection operation while referring to the image of the space, the user can reach the image that is the target of obtaining the target information. Compared with the case of conventional searches using the number of strokes, pronunciation, radicals, triangular codes, square codes, etc., for example, the labor of specifying the number of strokes and the knowledge for counting the number of strokes can be made unnecessary, and thus it becomes possible to easily obtain the target information, and the above object is achieved by these.

[0017] <Aspects of the registration process of the designated transition destination image (warp destination) by the designated transition destination image registration means>

[0018] The designated transition destination image registration means in the present invention described above can have the following various configurations.

[0019] <Configuration in which the image of the layer immediately above the arrival image is used as a candidate for the designated transition destination image (warp destination)>

[0020] That is, in the information search system of the present invention described above, the designated transition destination image registration means can be configured to display, on the screen of the display means, as a candidate for the designated transition destination image, the image of the layer immediately above the arrival image stored in the layer information storage means for the user, and accept a selection from the displayed candidate images.

[0021] When configured to display the image of the layer immediately above the arrival image as a candidate for the designated transition destination image in this way, when there are multiple images of the layer immediately above for a certain arrival image, not only can one return to the image immediately before in the search path that has passed, but it is also possible to transition to other images of the layer immediately above.

[0022] Here, "when there are multiple images of the layer immediately above" means, for example, when there are an image of the tennis court space and an image of the tennis shop space as images of the space of the layer immediately above for an image of an object such as a tennis ball. Therefore, assuming that one has reached an image of an object such as a tennis ball along a search path from the tennis court to the tennis ball, when returning to the image immediately before in this search path, one will return to the image of the tennis court space and will not transition from the tennis ball to the image of the tennis shop space. However, if the user feels that the association with the tennis ball is stronger with the space of the tennis shop than with the space of the tennis court in their own association, and registers the image of the tennis shop space as the designated transition destination image, it is possible to realize the transition from the tennis ball to the image of the tennis shop space.

[0023] Conversely, if the user feels that the tennis ball has a strong association with the space of the tennis court, an image of the space of the tennis court may be registered as the designated transition destination image. In this case, the image immediately preceding in the search path and the designated transition destination image may happen to match, and the transition destinations of both will be the image of the space of the tennis court. However, they do not always match. In this case, when an image (destination image) of an object called a tennis ball is reached through another search path (the search path of the transition from the tennis shop to the tennis ball), the image of the space of the tennis shop becomes the image immediately preceding in the search path. Therefore, it is possible to return to the image of the space of the tennis shop, and it is also possible to transition to the image of the space of the tennis court registered as the designated transition destination image.

[0024] <Configuration in which the user-specified image is used as a candidate for the designated transition destination image (warp destination)>

[0025] Also, in the information search system of the present invention described above, a user-specified image registration means is provided that accepts an input designation by the user regarding using, as the user-specified image, an image that is the object of word explanation on the screen displayed by the display means, and stores the identification information of the user-specified image in the user-specified image storage means. The designated transition destination image registration means can be configured to display, on the screen of the display means, the user-specified image whose identification information is stored in the user-specified image storage means as a candidate for the designated transition destination image to the user, and accept a selection from the displayed candidate images.

[0026] When configured to display the user-specified image as a candidate for the designated transition destination image in this way, if an image preferred by the user, an often-used image, an image that the user feels will have high utility value in the future, etc. are registered as the user-specified image (favorite) during the search, it becomes possible to register that user-specified image as the designated transition destination image.

[0027] For example, there is a person who has tennis and baseball as hobbies. For that person, if the association between the space of a tennis court and the space of a baseball field is strong, if that person registers an image of the space of a tennis court or an image of the space of a baseball field as a user-specified image (favorite), when reaching an image of the space of a baseball field in a certain search, it becomes possible to easily register the image of the space of a tennis court as a specified transition destination image, and when reaching an image of the space of a tennis court in another search, it becomes possible to easily register the image of the space of a baseball field as a specified transition destination image. Note that usually, there are no baseball supplies in the space of a tennis court and no tennis supplies in the space of a baseball field, so an association unique to that person is constructed.

[0028] <Configuration of using the image obtained by keyword search or the image in the immediately upper layer thereof as a candidate for the specified transition destination image>

[0029] Furthermore, in the information search system of the present invention described above, the specified transition destination image registration means immediately attaches and accepts the input of a keyword for searching the specified transition destination image by the user, uses the received keyword for searching, searches for a word that matches or is similar to the keyword from among the words indicating the images included in the hierarchical information stored in the hierarchical information storage means, and displays on the screen of the display means, as candidates for the specified transition destination image, the images corresponding to the words obtained by the search and the images in the immediately upper layer of this image, and can be configured to accept a selection from the displayed candidate images.

[0030] When configured to display the image obtained by keyword search or the image in the immediately upper layer thereof as a candidate for the specified transition destination image in this way, it becomes possible to find a desired image relying on the word and register it as the specified transition destination image.

[0031] For example, there is a person who has tennis and baseball as hobbies. For that person, if the association between the space of a tennis court and the space of a baseball field is strong, when that person reaches an image of the space of a baseball field in a certain search, if the person searches using a tennis court or tennis as a search keyword, it becomes possible to easily register an image of the space of a tennis court as a designated transition destination image. When reaching an image of the space of a tennis court in another search and searching using a baseball field or baseball as a search keyword, it becomes possible to easily register an image of the space of a baseball field as a designated transition destination image.

[0032] <Configuration for accepting a user's selection of a designated transition destination image within a tree-shaped hierarchical display>

[0033] Also, in the information search system of the present invention described above, The designated transition destination image registration means can be configured to display the hierarchy of each image in a tree form for the user using the hierarchy information stored in the hierarchy information storage means, and accept the user's selection of a designated transition destination image within this tree-shaped hierarchical display.

[0034] When configured to accept the user's selection of a designated transition destination image within a tree-shaped hierarchical display in this way, the user can select a designated transition destination image to be registered while referring to the hierarchy information. Therefore, it becomes possible to register the association between personal images that match the user's association while confirming the association (that is, the standard association) between the images predetermined as the hierarchy information.

[0035] <Configuration for disabling the registration of a designated transition destination image (warp destination)>

[0036] Furthermore, in the information search system of the present invention described above, In the designated transition destination image storage means, The final use date and time using the transition from the arrival image to the designated transition destination image, or the registration date and time of the designated transition destination image when the use of the transition from the arrival image to the designated transition destination image has never occurred, are stored in association with the identification information of the arrival image, together with the identification information of the registered designated transition destination image. The designated transition destination image registration means may be configured to determine whether a predetermined time has elapsed from the final use date and time or the registration date and time when there has been no use, and if so, invalidate the registration of the designated transition destination image.

[0037] When configured to invalidate the registration of the designated transition destination image in this way, it becomes possible to invalidate the associations between unused spaces, the association between a space and an object, and the association between an object and its component articles, even though the user has registered them, for a predetermined period. This invalidation of registration can be considered in conjunction with the fact that once-formed synapses are severed due to long-term non-use.

[0038] <Configuration that also accepts registration of reverse transitions>

[0039] And in the information retrieval system of the present invention described above, the designated transition destination image registration means When registering the designated transition destination image to enable the transition from the arrival image to the designated transition destination image, accepts the user's selection to also register the reverse transition from the reverse designated transition destination image to the arrival image. When the selection of this reverse transition is accepted, it is assumed that the image currently being registered as the designated transition destination image becomes the temporary arrival image at another search time, and the image currently being the arrival image becomes the temporary designated transition destination image at another search time. The identification information of the temporary designated transition destination image in this assumed state may be stored in the designated transition destination image storage means in association with the identification information of the temporary arrival image.

[0040] When configured to also accept registration of reverse transitions in this way, when the user feels that the reverse association is also strong, that is, when the user feels that the bi-directional association is strong, the user can easily register the reverse transition without having to perform another search and the operation of reaching the image that is currently being attempted to be registered as the designated transition destination image.

[0041] In the present invention, the association between images (the association between spaces, the association between a space and an object, the association between an object and its constituent articles) is predefined as hierarchical information with directionality. Therefore, the transition from the reached image to the designated transition destination image is also registered with directionality. Accordingly, during a search, when reaching a certain image (the reached image), even if the designated transition destination image from that image is registered, this alone does not mean that a bi-directional transition has been registered. However, as described above, it is also possible to register a bi-directional transition by the user's selection. Further, since the association of the identification information of each image in the designated transition destination image storage means is also stored with directionality, when registering a bi-directional transition, the identification information of each image is associated with the designated transition destination image storage means with the positions of the respective images swapped.

[0042] <Configuration including start image selection means>

[0043] Also, in the information retrieval system of the present invention described above, In order to set a start image for a search in place of the image at the highest level of the hierarchy, (1) Add and accept the input of a search keyword for the start image by the user, and using the received search keyword, search for words that match or are similar to the search keyword among the words indicating images included in the hierarchical information stored in the hierarchical information storage means, and display, on the screen of the display means, as candidates for the start image, the images corresponding to the words obtained by the search and the images at the level immediately above this image, and accept a selection from the displayed candidate images, a keyword search utilization start image selection process, (2) On the screen displayed by the display means, accept the user's input designation regarding setting the image to be the object of word explanation as the user-designated image, store the identification information of the user-designated image in the user-designated image storage means, display the user-designated image whose identification information is stored in the user-designated image storage means on the screen of the display means as a candidate for the start image for the user, and perform a user-designated image utilization start image selection process for accepting a selection from the displayed candidate images, (3) Store the cumulative number of passes of each image passed in past searches in the cumulative number of passes storage means, and except for the images excluded in advance from the object of the start image, display on the screen of the display means the top predetermined number of images with a large cumulative number of passes as candidates for the start image, and perform a cumulative number of passes utilization start image selection process for accepting a selection from the displayed candidate images It is desirable to adopt a configuration including start image selection means for executing at least one of the above processes.

[0044] When configured with such start image selection means in this way, the user does not necessarily have to start the search from the image in the topmost layer, and can change the start image of the search and start the search according to the progress of the user's learning or the growth of a child user, etc. For this reason, the convenience of the system for the user can be improved.

[0045] <Configuration enabling selection of a start image by two or more processes including a start image selection process using keyword search>

[0046] Also, in the case of adopting a configuration including the above-described start image selection means, the start image selection means (1) executes the keyword search utilization start image selection process, and is desirably configured to execute the user-designated image utilization start image selection process of (2) and / or the cumulative number of passes utilization start image selection process of (3).

[0047] When the start image can be selected by two or more processes including the selection process of the start image using keyword search as described above, the selection of the start image by the user becomes easy, and the convenience of the system for the user can be improved.

[0048] <A configuration for posing a question about an image (the main image to be described by words) displayed on the screen of the display means>

[0049] Also, in the information retrieval system of the present invention described above, Question data including an image schema learning question using an object included in an image of a space or a constituent article included in an image of an object as a subject, and a problem object to be moved indicating the correct answer and its movement destination area are stored in association with the identification information of the source image data of the question. A problem storage means, Using the question data stored in the question storage means in association with the identification information of the source image data of the question to be described by words on the screen displayed on the display means, a question including an image schema learning question is displayed on the screen of the display means for the user, the user's answer is received, the correctness of the answer is judged using the problem object to be moved and its movement destination area stored in the question storage means, and the judgment result is displayed on the screen of the display means and stored in the question storage means in association with the identification information of the source image data of the question. It is desirable to have a configuration including problem processing means.

[0050] When configured to pose a question about an image (which is the main image to be described by words and not the symbol part of the image selection section for transitioning to the image of the immediately lower layer included in the image) displayed on the screen of the display means as described above, the user can learn a language in the space and objects of the image, so that it is possible to have a feeling of learning a language in the space and objects (that is, the image reached by the search) that the user has visited. For this reason, a situation similar to language acquisition in the real world can be created, so that the user can perform effective learning.

[0051] <Configuration provided with evaluation means for recommending the field of the workplace using the judgment result of the correctness of the answer to the question>

[0052] Furthermore, in the case of adopting a configuration in which questions regarding the images displayed on the screen of the above-described display means are posed, Using the judgment result of the correctness of the answer stored in the question storage means and the employment recommendation field stored in the question image data storage means in association with the identification information of the image data, it is desirable to adopt a configuration provided with evaluation means for displaying, on the screen of the display means, information on the field of the workplace to be recommended to the user.

[0053] When configured with evaluation means for recommending the field of the workplace using the judgment result of the correctness of the answer to the question in this way, it becomes possible to recommend the field of the workplace using the results of language learning in the space or object visited by the user (that is, the image reached by the search). For this reason, it becomes possible to perform a recommendation in a situation close to a recommendation based on experience in the real society.

[0054] <Configuration provided with evaluation means for judging whether the user understands the image schema>

[0055] Also, in the case of adopting a configuration in which questions regarding the images displayed on the screen of the above-described display means are posed, For each of the plurality of image data, using the judgment result of the correctness of the answer to the question for learning the image schema of the same question type stored in the question storage means, it is desirable to adopt a configuration provided with evaluation means for judging whether the user understands the image schema of the question type and displaying the judgment result on the screen of the display means.

[0056] When configured with evaluation means for judging whether the user understands the image schema in this way, it becomes possible to judge whether the user is learning the image schema so as to be applicable to other scenes or other fields, and if there is a possibility that the user has not acquired the image schema, to convey that fact to the user and promote further learning.

[0057] <Configuration for displaying problem instructions in multiple languages>

[0058] Furthermore, when configured to pose problems regarding the images displayed on the screen of the above-described display means, In the problem storage means, problem instructions in multiple languages are stored as the problem instructions constituting the problem data. The problem processing means may be configured to display on the screen of the display means the problem instructions in multiple languages stored in the problem storage means.

[0059] When configured to display problem instructions in multiple languages in this way, the user can simultaneously learn multiple languages in the space or object of the image (which is the main image for which the word is to be explained and not the symbol part of the image selection section for transitioning to the image in the immediately lower layer included in that image) displayed on the screen of the display means. Therefore, it becomes possible to have the feeling of learning each language in the space or object (i.e., the image reached by the search) one has visited. For this reason, a situation similar to the acquisition of each language in the real world can be created, and thus the user can perform effective learning.

[0060] <Configuration for displaying a manufacturing method manual>

[0061] Also, in the information retrieval system of the present invention described above, Manufacturing method manual storage means for storing manufacturing method manual data indicating the manufacturing method of the object forming the space indicated by the image data, or the object indicated by the image data or its constituent articles, in association with the identification information of the image data, Manufacturing method manual display processing means for displaying on the screen of the display means the manufacturing method manual data stored in the manufacturing method manual storage means in association with the identification information of the image data for which a word is to be explained on the screen displayed by the display means It is desirable to have a configuration including

[0062] Here, the "thing that forms a space" refers to the space when the space indicated by the image can be regarded as a thing. For example, the image of "the space on the desk" includes patterns (images) such as pencils and erasers. Although the attribute of the image is a space, since the desk itself is a thing and there is a manufacturing method for the desk, it means that a manufacturing manual can be prepared.

[0063] Also, the "manufacturing manual data" is preferably video data (which may include audio data), but it may also be still image data.

[0064] When configured to display the manufacturing manual in this way, the user can learn a language in the space or object (i.e., the image reached by the search) the user visited, and can also learn the manufacturing method of the thing related to that image. Therefore, it becomes easier to fix the memory of the word indicated by that image.

[0065] For example, if the user goes from the image of "the space on the desk" to the image of "an object called an eraser", the user can learn the spelling, meaning content, and pronunciation of the word corresponding to "eraser" in Japanese, and can also learn the manufacturing method of the "eraser" (manufacturing process in the factory). Also, if the user goes from the image of "an object called a personal computer" to the image of "a component article called a CPU (central processing unit)", the user can learn the spelling, meaning content, and pronunciation of the word corresponding to "CPU", and can also learn the manufacturing method of the CPU (which may include the manufacturing method of a semiconductor which is a further component article).

[0066] <Configuration for displaying a recipe for food>

[0067] Furthermore, in the case of the configuration for displaying the above-described manufacturing manual, the manufacturing manual data may be recipe data when the object or its component article is food.

[0068] When configured to display food recipes in this way, the user can learn a language from the images of the objects they visit and their component articles (i.e., the images reached through the search), and can also learn the manufacturing methods of the foods related to those images. For this reason, it becomes easier to fix the memory of the words pointed to by those images.

[0069] For example, if the user reaches the image of the object (food) "ramen" from the image of the "space of a Chinese restaurant", they can learn the spelling, meaning content, and pronunciation of the word corresponding to "ramen" in Japanese, and can also learn how to make ramen. Also, if the user reaches the images of the component articles (foods) "noodles" and "menma" from the image of the object (food) "ramen", they can learn the spelling, meaning content, and pronunciation of the words corresponding to "noodles" and "menma", and can also learn how to make noodles and menma.

[0070] <Configuration for accepting the choice of the times>

[0071] And in the information retrieval system described above, it is provided with time switching means for accepting the user's selection of a time in the time selection section displayed on the screen of the display means, the image data storage means, the explanation data storage means, the hierarchical information storage means, and the designated transition destination image storage means are provided for each time, the selection acceptance processing means, the screen switching processing means, the explanation display processing means, and the designated transition destination image registration means may be configured to execute their respective processes using the data stored in the image data storage means, the explanation data storage means, the hierarchical information storage means, and the designated transition destination image storage means for the time selected by the time switching means.

[0072] When configured to accept the choices of the times in this way, the user can perform various information searches across multiple times, that is, across multiple eras (where the width of the time axis division when defining the eras prepared in this system is arbitrary and can be, for example, a relatively short time period such as several years or 10 years, or a relatively long time period such as the Stone Age). The user can learn not only current terms and their explanations (including usage methods, etc.), but also terms and their explanations in history (including recent past), manufacturing methods (including recipes) of objects that existed in history (including food), etc., or terms and their explanations in the predicted or imagined future, manufacturing methods of future objects, etc.

[0073] <Invention of Program>

[0074] Also, the program of the present invention is for causing a computer to function as the information search system described above.

[0075] Note that the above program or a part thereof can be recorded and stored, distributed, etc. on a recording medium such as a magneto-optical disk (MO), compact disk (CD), digital versatile disk (DVD), flexible disk (FD), magnetic tape, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM), flash memory, random access memory (RAM), hard disk drive (HDD), solid state drive (SSD), flash disk, etc. It can also be transmitted using a transmission medium such as a wired network such as a local area network (LAN), metropolitan area network (MAN), wide area network (WAN), Internet, intranet, extranet, etc., or a wireless communication network, or a combination thereof, and can also be carried on a carrier wave. Further, the above program may be a part of another program, or may be recorded on a recording medium together with a separate program.

Effects of the Invention

[0076] As described above, according to the present invention, not only is the search for information related to words advanced from the upper layer to the lower layer according to the hierarchical information stored in the hierarchical information storage means, but also the transition from an arbitrary reached image reached by the search to the designated transition destination image registered by the user in association with this reached image is performed. Therefore, the user can construct on the system the association between spaces, the association between a space and an object, or the association between an object and its constituent articles according to his or her own association, and perform a search according to the association. As a result, it is possible to realize an efficient information search according to the experience and preference of each user.

Brief Description of Drawings

[0077]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Mode for Carrying Out the Invention

[0078] An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows the overall configuration of the information retrieval system 10 of this embodiment. FIG. 2 shows, in a flowchart, the flow of operations performed by a user who uses the information retrieval system 10. FIG. 3 shows an example of a setting screen 100 such as the language to be used. FIGS. 4 to 8 show example screens from the topmost layer to the second to fifth layers. FIG. 9 shows an example of a setting change screen 200. Further, FIG. 10 shows the configuration of the hierarchical information storage means 44, FIG. 11 shows the configuration of the image coordinate information storage means 45, FIG. 12 shows the configuration of the cumulative passage count storage means 47, FIG. 13 shows the configuration of the user-specified image storage means 48, FIG. 14 shows the configuration of the designated transition destination image storage means 49, FIG. 15 shows the configuration of the problem image data storage means 50, and FIG. 16 shows the configuration of the problem storage means 51. Furthermore, FIG. 17 shows an example of a start image selection screen 300, FIG. 18 shows a specific example of candidates for the designated transition destination image (warp destination), FIG. 19 shows the transition destination from the arrival image, FIG. 20 shows an example of a designated transition destination image (warp destination) registration screen 400, FIG. 21 shows an example of a problem screen 500, and FIG. 22 shows an example of an evaluation report screen 600.

[0079] <Overview of the information retrieval system 10 and meanings of terms>

[0080] In FIG. 1, the information retrieval system 10 is a system constituted by one or a plurality of computers for learning a language other than the mother tongue by retrieving target information (information related to words) using hierarchical images (images of spaces, objects, constituent articles) constructed on the system. Note that in the case of a child user, the information retrieval system 10 can be used even if the user has no knowledge of language at all including the mother tongue or the mother tongue is incomplete.

[0081] Here, the "image (image data)" includes, as described above, an image of space, an image of an object, and an image of a constituent article, and any of the image data may be planar image data or three-dimensional image data. Further, the image data may be illustration data or actual photograph data. Furthermore, the image data is not limited to still image data, but may also be moving image data, or data combining a still image and a moving image (data composed of a still image layer and a moving image layer). In the case of the bottommost image (see Fig. 8), since it is not necessary to provide an image selection section for transitioning to the image of the immediately lower layer, it can be moving image data. In the case of the images of other layers, since it is necessary to provide an image selection section (for example, the image selection sections 131, 132, 133, etc. in Fig. 4), for example, the symbol portion of the image selection section can be provided on the still image layer, and the portion to be moved can be made into the moving image layer. These image data are stored in the image data storage means 41.

[0082] In addition, the "purpose information (information related to a word)" includes, for example, the pronunciation, phonetic symbols, spelling, meaning, usage, origin, equivalent word, antonym, related phrases, etc. of a word indicating a space, an object, or its component articles. In addition, other things corresponding to the word (including foods and animals and plants), that is, things forming the space indicated by an image, or things related to the object indicated by the image or its component articles, such as the name, address, telephone number, facsimile number, homepage address, map, handling scale, cheapness ranking of a store, prices in various countries and each store (individual prices for each store, average, standard, national prices), bargain information, advertisements of products and goods, the inventory quantity of one's own home or company, business partners, types (manufacturer, quality, performance, specifications, etc.), location, habitat, logos and trademarks of products and goods, logos and trademarks of manufacturers and sellers, etc. are also included. In addition, the title, price, publication date, etc. of magazines and books with descriptions of things corresponding to the word (including foods and animals and plants) are also included. Among these purpose information (information related to a word), the pronunciation of a word is voice data, and the remaining data is explanatory data of the word if it is text data (however, including logos and trademarks. Also including icons such as emojis). If it is the voice reading of the text, it is voice data, and it is stored in the explanatory data storage means 41 and the voice data storage means 42.

[0083] In addition, the "purpose information (information related to a word)" also includes manufacturing method manual data indicating the manufacturing method of things corresponding to the word (including foods and animals and plants). This manufacturing method manual data includes recipe data when the word indicates food and breeding manual data when the word indicates animals and plants. These manufacturing method manual data are stored in the manufacturing method manual storage means 46.

[0084] And the "word" includes words in various languages such as Chinese, French, Russian, Japanese, German, Spanish, English, etc. In addition, if it is Japanese, for example, Tohoku dialect, Nagoya dialect, Kansai dialect, etc., and if it is English, various dialects such as American English and British English are also included.

[0085] Furthermore, the "space" refers to something that can include an image of an object within an image of that space, such as the space of a house, urban space, natural space, the space of a study, the space of a toilet, the space of a dining room, the space on or within a desk, and so on.

[0086] And the "object" includes living organisms in a broad sense such as humans, animals, plants, insects, birds, microorganisms, bacteria, etc. It also includes not only solids but also liquids such as water and alcohol, and gases such as air and propane gas. Furthermore, for example, a human or animal with a characteristic facial expression, a human or animal with characteristic gestures, or a symbolic object, etc. are also included in the "object". For example, when the facial expression of an object of "human" belonging to the "dining room space" belonging to the "space of a house" shows an expression of "delicious" or "tasteless", the adjectives "delicious" or "tasteless" can be used as the pronunciation, spelling, etc. of the word as the target information. Also, for example, when the gesture of an object of "dog" belonging to the "garden space" belonging to the "space of a house" is a gesture of "sniffing" an odor, the verb "sniff" can be used as the pronunciation, spelling, etc. of the word as the target information. Also, for example, when the gesture of an object of "human" belonging to the "toilet space" belonging to the "space of a house" is a gesture of pinching the nose and showing a "smelly" gesture, the adjective "smelly" can be used as the pronunciation, spelling, etc. of the word as the target information. And, for example, when an object of "car" belonging to the "space on the road" belonging to the "urban space" is a racing car with speed, the adjective "fast" can be used as the pronunciation, spelling, etc. of the word as the target information. Also, for example, when an object of "seawater" belonging to the "space of the sea" belonging to the "natural space" is seawater in the deep sea part, the adjective "deep" can be used as the pronunciation, spelling, etc. of the word as the target information. Therefore, the "object" in the present invention includes not only the case of referring to the "object" itself but also the case of referring to what is expressed by that "object".

[0087] Also, when registering something that can be regarded as an object in the system, it may be registered by regarding it as a space. For example, an aircraft may be registered in the system as an object belonging to the "space of the airport" which belongs to the "urban space", or alternatively, it may be registered as the "space inside the aircraft" including the image of the pilot or the image of the cabin attendant which are images of a lower hierarchy. However, the image of the aircraft and the images of the pilot and the cabin attendant may also be registered in the system as objects belonging to the "space of the airport" as images of the same hierarchy.

[0088] Also, among the "hierarchical images", there are images of a plurality of spaces, and there are also images of a plurality of objects or their constituent articles belonging to at least one of these spaces. "Belonging to at least one" means that one object may belong to a plurality of different spaces. For example, a car may be registered in the system as an object belonging to the "space of the garage" which belongs to the "space of the house", or as an object belonging to the "space on the road" which belongs to the "urban space", or as an object belonging to both. When registered as an object belonging to both, there will be a plurality of search paths leading to the car.

[0089] Furthermore, "or its constituent articles" in the above "a plurality of objects or their constituent articles belonging to at least one of these spaces" means that when an object is registered in the system, its constituent articles do not necessarily have to be registered. For example, it is optional whether to end the search with the "car" as the lowest hierarchy or to be able to search up to the "tire", "bumper", etc. which are constituent articles of the "car" as the lowest hierarchy. However, the search referred to as "ending the search" here is a search from the upper hierarchy to the lower hierarchy according to the hierarchical information, excluding the case of returning the search path and the case of transitioning to the designated transition destination image (loop destination) of the present invention.

[0090] Note that, it is also possible to register "brake pedal", "accelerator pedal", "steering wheel", etc. as constituent articles of a "motor vehicle", and consider that there is a subordinate relationship between the object and its constituent articles, or register these as objects belonging to the "space inside the motor vehicle", and consider that there is a subordinate relationship between the space and the object.

[0091] In addition, as will be described in detail later, the hierarchical information for hierarchizing a plurality of images (images of spaces, images of objects, images of constituent articles) is stored in a hierarchical information storage means 44 (see FIG. 10). This hierarchical information is information for grasping what the image in the lower layer of a certain image is, or what the image in the upper layer of a certain image is. The image in the layer immediately below the image of the space is an image of the space or an image of an object. The image of the object is either the image in the lowest layer, or a constituent article image is subordinate to the object image as its lower layer image. Therefore, for example, like a search path of space → space → space → object, a search path for obtaining target information by displaying an object image via at least one space image, or like a search path of space → space → space → object → constituent article, there are search paths such as obtaining target information by passing through at least one space image and passing through the object image to display the constituent article image thereof. Also, when the target information is information targeting a space, the search may be terminated at the stage where the space image is displayed on the screen of the display means 60, like a search path of space → space → space. In short, it suffices that the configuration is such that a search via at least one space image is performed.

[0092] <Overall Configuration of Information Retrieval System 10>

[0093] In FIG. 1, the information retrieval system 10 includes a computer main body 20, a display means 60 and an input means 61 connected to the main body 20. Also, a data providing server 70 is connected to the information retrieval system 10 via a network 1.

[0094] Here, network 1 is an external network mainly composed of the Internet, and it may be a combination of the Internet and internal networks such as LANs and intranets. Whether it is wired, wireless, or a mixed type of wired and wireless, in short, as long as it can transmit information at a certain speed between multiple locations (regardless of the length of the distance).

[0095] The main body 20 is configured to include processing means 20A that executes various processes for searching information related to words, and storage means 40 that stores various data necessary for the processing by the processing means 20A.

[0096] The processing means 20A includes selection reception processing means 21, screen switching processing means 22, explanation display processing means 23, voice output processing means 24, explanation data acquisition means 25, voice data acquisition means 26, setting means 27, user-specified image registration means 28, start image selection means 29, designated transition destination image registration means 30, manufacturing method manual display processing means 31, manufacturing method manual acquisition means 32, problem processing means 33, evaluation means 34, and problem image data / problem data acquisition means 35. Among these, each of the means 21 to 26 has the same functions as the information search system described in Patent Document 1 mentioned above. However, functions have been added to the selection reception processing means 21 and the screen switching processing means 22.

[0097] Here, each of the means 21 to 35 included in the processing means 20A is realized by a central processing unit (CPU) provided inside the computer constituting the main body 20, one or more programs that define the operation procedures of this CPU, and a working memory such as a main memory and a cache memory. The details of each of these means 21 to 35 will be described later.

[0098] The memory means 40 includes an image data memory means 41, an explanatory data memory means 42, an audio data memory means 43, a hierarchical information memory means 44, an image coordinate information memory means 45, a manufacturing method manual memory means 46, a cumulative passage count memory means 47, a user-specified image memory means 48, a specified transition destination image memory means 49, a problem-use image data memory means 50, and a problem memory means 51. Among these, each of the memory means 41 to 45 is the same as the information retrieval system described in Patent Document 1 mentioned above.

[0099] Here, as each of the memory means 41 to 51 included in the memory means 40, for example, a non-volatile memory such as a hard disk drive (HDD) or a solid state drive (SSD) can be adopted. The details of each of these memory means 41 to 51 will be described later.

[0100] As the display means 60, for example, a liquid crystal display, a CRT display, a projector and a screen, or a combination thereof can be adopted.

[0101] As the input means 61, for example, various ones such as a mouse, a keyboard, a touch panel, a trackball, a light pen, a track pad, a track point, a joystick, a voice recognition device, or a combination thereof can be adopted.

[0102] The data providing server 70 is constituted by one or a plurality of computers, stores the program of the present invention executed in the information retrieval system 10, explanatory data created by each publisher, audio data, and manufacturing method manual data (including recipe data), and problem-use image data and problem data (including voice reading data), and executes a process of transmitting these data to the information retrieval system 10 via the network 1 in response to a download request from the user of the information retrieval system 10.

[0103] <Configuration of the processing means 20A / selection reception processing means 21>

[0104] The selection reception processing means 21 executes a process of receiving a selection operation of each image selection unit by the user on the screen of the display means 30 on which an image of a space including an image selection unit for selecting an image of a space or an object belonging to the immediately lower layer or an image of an object including an image selection unit for selecting an image of a constituent article belonging to the immediately lower layer is displayed.

[0105] Further, the selection reception processing means 21 also executes a process of receiving a selection operation of a "return" button 125 (see FIGS. 4 to 8) by the user for returning to the screen including the previous image in the search path that has passed.

[0106] Furthermore, the selection reception processing means 21 also executes a process of receiving a selection operation of a "warp" button 128A (see FIGS. 4 to 8) by the user for transitioning to a designated transition destination image (warp destination) registered by the user himself. This function is an added function in the present invention.

[0107] Note that the "selection operation" of the user received by the selection reception processing means 21 is, for example, a click operation of a mouse, a touch operation by a finger or a stylus pen (touch pen), etc. In the following description, clicking or touching may be expressed as "pressing" or "depressing", and the description of "click" may represent all modes of selection operations including touches by a finger or a pointing device.

[0108] <Configuration of processing means 20A / screen switching processing means 22>

[0109] The screen switching processing means 22 switches the screen displayed on the display means 60 to a screen for displaying an image of a space or an object corresponding to the image selection unit selected by the selection operation of the user received by the selection reception processing means 21 or an image of its constituent article according to the hierarchical information stored in the hierarchical information storage means 44 (see FIG. 10) using the image data stored in the image data storage means 41.

[0110] Note that the screen switching processing means 22 also executes the process of displaying the screen of the topmost layer. That is, in the present embodiment, the screen switching processing means 22 also executes the switching process from the setting screen 100 in FIG. 3 to the screen on which the image 130 of the "Comprehensive Space" in FIG. 4 is displayed.

[0111] In addition, when the selection reception processing means 21 receives a selection operation of the "Return" button 125 (see FIGS. 4 to 8), the screen switching processing means 22 also executes a process of returning the screen that displays the currently displayed image (arrival image) reached by the search to the screen that displays the previous image in the search path through which it has passed.

[0112] Furthermore, when the selection reception processing means 21 receives a selection operation of the "Warp" button 128A (see FIGS. 4 to 8), the screen switching processing means 22 uses the specified transition destination image (warp destination) in which the identification information (image ID) is stored in the specified transition destination image storage means 49 (see FIG. 14) associated with the identification information (image ID) of the currently displayed image (arrival image) reached by the search, and switches the screen (the screen that displays the arrival image) displayed on the display means 60 to the screen that displays the specified transition destination image (warp destination). This function is an added function in the present invention.

[0113] And when the screen switching processing means 22 performs the screen switching process from the arrival image to the specified transition destination image (warp destination), it updates the data in the column of the latest use date and time (initially the registration date and time) stored in the specified transition destination image storage means 49 (see FIG. 14). If the latest use date and time has already been stored, it is the update registration process of the latest use date and time, and if the registration date and time is stored, it is the update registration process from the registration date and time to the latest use date and time due to the first use. This function is also an added function in the present invention.

[0114] In addition, when the screen switching processing means 22 performs screen switching processing, the cumulative passage count update process that increases the cumulative passage count stored in the cumulative passage count storage means 47 (see FIG. 12) associated with the image ID of the image (the main image for which the word is to be explained) included in the screen after switching by 1 is also executed. This function is an added function in the present invention.

[0115] <Configuration of processing means 20A / description display processing means 23>

[0116] The description display processing means 23 executes processing to display an explanation of a word indicating these spaces, objects, or constituent articles of these on the same screen as the screen on which an image of a space, object, or constituent article thereof is displayed, using the explanation data stored in the explanation data storage means 42. In the present embodiment, an explanation of a word (phonetic symbols, meaning content, etc.) is displayed in the explanation display section 122 of the search window 120 which is a search screen (see FIGS. 5 to 8).

[0117] In addition, when a user (a person who performs a search) who is learning a foreign language downloads explanation data of words in a plurality of languages from the data providing server 70 by the explanation data acquisition means 25, the explanation display processing means 23 is configured to display explanations of words in a plurality of languages in parallel. Since the display within the explanation display section 122 of the search window 120 can be scrolled, even an explanation that cannot fit within the explanation display section 122 can be referred to.

[0118] <Configuration of processing means 20A / audio output processing means 24>

[0119] The audio output processing means 24 executes processing to output the pronunciation of a word indicating the space, object, or constituent article of the object corresponding to the image selection section selected by the user's selection operation, using the audio data stored in the audio data storage means 43 when the screen is switched by the screen switching processing means 22.

[0120] Also, when a user (the person performing the search) who is learning a foreign language downloads voice data of words in a plurality of languages from the data providing server 70 by the voice data acquisition means 26, the voice output processing means 24 outputs the pronunciation of the language first selected on the setting screen 100 (see FIG. 3). However, it is also possible to change the language for which the pronunciation is output by changing the display order of each language by the explanation display processing means 23.

[0121] <Configuration of processing means 20A / Explanation data acquisition means 25>

[0122] Based on the user's selection operation on the setting screen 100 (see FIG. 3), the explanation data acquisition means 25 downloads and acquires any one of the explanation data created by each publisher from the data providing server 70 connected via the network 1, and stores this in the explanation data storage means 42 in association with an image ID (identification information of image data). Explanation data is stored in the data providing server 70 in association with the image ID. Also, the explanation data acquisition means 25 can download explanation data of words in a plurality of languages in response to the request of a user (the person performing the search) who is learning a foreign language.

[0123] <Configuration of processing means 20A / Voice data acquisition means 26>

[0124] Based on the user's selection operation on the setting screen 100 (see FIG. 3), the voice data acquisition means 26 downloads and acquires any one of the voice data created by each publisher from the data providing server 70 connected via the network 1, and stores this in the voice data storage means 43 in association with an image ID (identification information of image data). Voice data is stored in the data providing server 70 in association with the image ID.

[0125] <Configuration of processing means 20A / Setting means 27>

[0126] The setting means 27 executes a process of accepting, on the setting screen 100 shown in FIG. 3, selections by the user of the language to be used and its display order, the explanatory data to be used, the voice data, the publisher of the manufacturing method manual data (including recipe data and breeding manual data), the addition or deletion of languages, and the change of the display order of languages, and storing the accepted setting information in a setting information storage means (not shown).

[0127] Also, the setting means 27 executes a process of accepting a data set (setting change) by the user on the setting change screen 200 shown in FIG. 9, and storing the accepted setting information in the hierarchical information storage means 44 (see FIG. 10) and the image coordinate information storage means 45 (see FIG. 11).

[0128] Furthermore, although not shown in the figure, the setting means 27 executes a process of clearing the accumulated passing times stored in the accumulated passing times storage means 47 (see FIG. 12) to zero, and a process of accepting the setting and change of a flag indicating whether or not it corresponds to an exclusion start image (an image to be excluded from the candidates for the start image) stored in the accumulated passing times storage means 47.

[0129] <Configuration of the processing means 20A / user-specified image registration means 28>

[0130] The user-specified image registration means 28 executes a process of accepting an input specification by the user regarding setting an image that is the object of word explanation as a user-specified image (favorite) on the screen displayed on the display means 60, and storing the image ID (identification information of the image data) of the accepted user-specified image in the user-specified image storage means 48 (see FIG. 13). The user-specified images registered in the user-specified image storage means 48 are used in the start image selection process by the start image selection means 29 and the registration process of the designated transition destination image (warp destination) by the designated transition destination image registration means 30.

[0131] Specifically, when the user clicks the "Register" button 128B of the user-specified image (favorite) provided in the search window 120 (see FIGS. 4 to 8), the user-specified image registration means 28 registers the image ID of the image included in the currently displayed screen (which is the main image to be described by the word, not the pattern part which is the image selection part included in the image) as the image ID of the user-specified image in the user-specified image storage means 48 (see FIG. 13). On the other hand, when the user clicks the "Cancel" button 128C of the user-specified image (favorite), if the image included in the currently displayed screen has already been registered in the user-specified image storage means 48 as a user-specified image, the image ID of that user-specified image is deleted from the user-specified image storage means 48 by the user-specified image registration means 28.

[0132] <Configuration of Processing Means 20A / Start Image Selection Means 29>

[0133] As shown in FIG. 17, in order to set a start image for the search in place of the top-level image (in this embodiment, the image of the integrated space in FIG. 4), the start image selection means 29 executes any one of the following processes according to the user's selection operation: (1) keyword search start image selection process, (2) user-specified image start image selection process, (3) cumulative passing count start image selection process, and (4) standard setting process, and executes a process of storing the image ID of the selected start image (however, not only this time but also when starting from the selected image in subsequent starts) in a specified start image storage means (not shown).

[0134] (1) The keyword search start image selection process is to immediately prompt the user to input a keyword for searching the start image, and use the received search keyword to search for words (the names of words which are text data) that match or are similar to the search keyword from among the words included in the hierarchical information stored in the hierarchical information storage means 44 (see FIG. 10). The images corresponding to the words obtained by the search and the images of the hierarchy immediately above this image are displayed on the screen of the display means 60 as candidates for the start image, and the process is to accept the user's selection from the displayed candidate images. Note that the candidates for the start image displayed on the screen of the display means 60 are not only the names of the words indicated by the images, but also the images themselves, although they are reduced images.

[0135] (2) The user-specified image start image selection process is to accept the user's input designation on the screen displayed on the display means 60 regarding designating the image to be the subject of the word explanation as the user-specified image, store the image ID (identification information of the image data) of the user-specified image in the user-specified image storage means 48 (see FIG. 13), and display, on the screen of the display means 60 as a candidate for the start image, the user-specified image in which the image ID (identification information) is stored in the user-specified image storage means 48, and the process is to accept the user's selection from the displayed candidate images. Note that the candidates for the start image displayed on the screen of the display means 60 are not only the names of the words indicated by the images, but also the images themselves, although they are reduced images.

[0136] (3) The cumulative pass count-based start image selection process stores the cumulative pass count of each image passed through in past searches in the cumulative pass count storage means 47 (see FIG. 12). Excluding images that have been excluded in advance from the target of the start image (images for which the “exclusion start image” column in FIG. 12 is “1: applicable”), the top predetermined number (e.g., the top 3) of images with a high cumulative pass count are displayed on the screen of the display means 60 as candidates for the start image, and the process accepts the user's selection from the displayed candidate images. The topmost layer image (in this embodiment, the image of the comprehensive space in FIG. 4) is originally set as the start image by default (standard setting), so it is excluded in advance from the candidates for the start image here (see FIG. 12). Also, the images of the second layer from the top (in this embodiment, the images of the “city,” “house,” and “nature” spaces) will inevitably have a high pass count. So, if the user has no desire to start a search from the images of the second layer, the setting means 27 may make a setting to exclude the images of the second layer in advance (see FIG. 12). Note that the candidates for the start image displayed on the screen of the display means 60 are not only the names of the words indicated by the images but also the images themselves, although they are thumbnail images.

[0137] (4) The standard setting process is a process of returning the start image changed in the processes (1) to (3) to the image of the topmost layer of the default setting (standard setting) (in this embodiment, the image of the comprehensive space in FIG. 4).

[0138] Specifically, when the “start image change” button 128D (see FIGS. 4 to 8) provided in the search window 120 is clicked by the user, the start image selection means 29 displays the start image selection screen 300 (window) shown in FIG. 17 on the screen of the display means 60 and accepts the user's selection operation on this screen 300.

[0139] In FIG. 17, the start image selection screen 300 is provided with (1) a selection part 301 for the keyword search-based start image selection process, (2) a selection part 302 for the user-specified image-based start image selection process, (3) a selection part 303 for the cumulative pass count-based start image selection process, and (4) a selection part 304 for the standard setting process.

[0140] When the user clicks on the selection part 301 of the process in (1), the keyword search start image selection screen 310 (window) shown in FIG. 17 is displayed on the screen of the display means 60. In FIG. 17, on the keyword search start image selection screen 310, there are provided an input part 311 for search keywords, a “start image candidate search” button 312 for executing a search using the input search keywords, a start image selection part 313 for displaying the searched images and the images of the immediately upper hierarchy as candidates for the start image and accepting the selection of those images, a subsequent continuation selection part 314 for selecting to start the search from the image selected here also at the start of subsequent times, and an “apply” button 315 for applying the user settings (the user's selection is displayed in colors, blinking, etc.). Note that if the subsequent continuation selection part 314 is not selected, the start image setting is only valid for this time. When the subsequent continuation selection part 314 is selected, the image ID of the selected start image is stored in a designated start image storage means (not shown).

[0141] When the user clicks on the selection part 302 of the process in (2), the user-specified image utilization start image selection screen 320 (window) shown in FIG. 17 is displayed on the screen of the display means 60. In FIG. 17, on the user-specified image utilization start image selection screen 320, there is a start image selection part 321 that displays user-specified images in which image IDs (identification information) are stored in the user-specified image storage means 48 (see FIG. 13) as candidates for the start image and accepts the selection of these images, a "Do not display from next time" button 322 for selecting not to display the images displayed in this start image selection part 321 as candidates for the start image from the next time on, a subsequent continuation selection part 323 for selecting to start the search from the image selected here also at the start of subsequent times, and an "Apply" button 324 for applying the user's settings (the user's selection is displayed in colors, blinking, etc.). When the "Do not display from next time" button 322 is selected, the column of the start image candidates in the user-specified image storage means 48 (see FIG. 13) is changed to "0: Not applicable". Note that if the subsequent continuation selection part 323 is not selected, the setting of the start image is only valid for this time. When the subsequent continuation selection part 323 is selected, the image ID of the selected start image is stored in the designated start image storage means (not shown).

[0142] When the user clicks on the selection part 303 of the process (3), the cumulative passage count usage start image selection screen 330 (window) shown in Fig. 17 is displayed on the screen of the display means 60. In Fig. 17, on the cumulative passage count usage start image selection screen 330, an start image selection part 331 is provided which displays, as candidates for the start image, the top predetermined number (for example, the top 3) of images having a large cumulative passage count stored in the cumulative passage count storage means 47 (see Fig. 12) excluding the excluded images, and accepts the selection of these images; an "Exclude (do not display from the next time)" button 332 for selecting not to display the images displayed in this start image selection part 331 as candidates for the start image from the next time; a subsequent continuation selection part 333 for selecting to start the search from the image selected here also at the start of subsequent times; and an "Apply" button 334 for applying the user settings (where the user's selection is displayed in terms of color, blinking, etc.). When the "Exclude (do not display from the next time)" button 332 is selected, the column of the excluded start image in the cumulative passage count storage means 47 (see Fig. 12) is changed to "1: applicable". Note that if the subsequent continuation selection part 333 is not selected, the setting is for the start image only for this time. When the subsequent continuation selection part 333 is selected, the image ID of the selected start image is stored in the designated start image storage means (not shown).

[0143] When the user clicks on the selection part 304 of the process (4), in order to return to the standard settings, the image ID of the start image related to the user's selection designation stored in the designated start image storage means (not shown) is deleted.

[0144] <Configuration of the processing means 20A / designated transition destination image registration means 30>

[0145] As shown in FIGS. 18, 19, and 20, the designated transition destination image registration means 30 accepts registration of a designated transition destination image (warp destination) by the user in order to realize a transition to a designated transition destination image that is not limited to a transition to an image in the layer immediately below the reached image by a selection operation of an image selection section included in the reached image and a transition to the image immediately before the reached image in the search path to the reached image (transition by a selection operation of the "back" button 125) while transitioning from the upper layer to the lower layer. The designated transition destination image registration means 30 executes a process of associating the image ID (identification information of image data) of the designated transition destination image with the image ID (identification information of image data) of the reached image and storing it in the designated transition destination image storage means 49 (see FIG. 14).

[0146] As shown in FIG. 18, for an image of an object "eraser", for example, assume that there are images in the three immediately upper layers, namely, "space on the desk", "stationery store space", and "pencil case space" (the same as the example in FIG. 10). In this case, in the search path from the image of the "general space" at the topmost layer to the image of the "eraser", there are three search paths passing through "space on the desk", "stationery store space", and "pencil case space" respectively. If the user reaches the image of the "eraser" through one of these three search paths, that is, the search path "general space → home space → study space → space on the desk → eraser" (the path shown by the solid line in FIG. 18), then the image of the "eraser" is the reached image in this example. Since the image immediately before the reached image (image of the "eraser") in this search path is the image of the "space on the desk", when the user clicks the "back" button 125 in the search window 120 of FIG. 8 where the image of the "eraser" is displayed, the user will return to the image of the "space on the desk" (the return path shown by the dashed line in FIG. 18).

[0147] However, as a user, assume that among the three spaces of "the space on the desk", "the space in the stationery store", and "the space in the pencil case", the one with the strongest association with the "eraser" is not "the space on the desk" but "the space in the stationery store". At this time, the user should register the image of "the space in the stationery store" as the designated transition destination image (warp destination) in order to form a transition route (transition path) from the "eraser" to the "space in the stationery store" according to their own associations, and let it be stored in the designated transition destination image storage means 49 (see FIG. 14). In the example of FIG. 14, the image ID of the image of "the space in the stationery store", which is the designated transition destination image (warp destination), is stored in association with the image ID of the image of the "eraser", which is the arrival image. Therefore, a new association between images is formed so that new synapses are formed in the brain. However, the registration in the designated transition destination image storage means 49 in this example is only the formation of a transition route from the "eraser" to the "space in the stationery store", and the reverse transition route is not formed. Thus, the registration of the designated transition destination image (warp destination) is the formation of a transition route (transition path) with directionality.

[0148] Also, the images of the three spaces, namely, the "space on the desk", the "stationery store space", and the "pencil case space", which are the images of the layer directly above the arrival image (the image of the "eraser"), are all candidates for the designated transition destination images (warp destinations) from the arrival image (the image of the "eraser") (the transition path indicated by the thick two-dot chain line in Fig. 18). Therefore, even when the user reaches the image of the "eraser" through the search path of the integrated space → home space → study space → space on the desk → eraser (the path indicated by the solid line in Fig. 18) as in the above example, the user may register the image of the "space on the desk" as the designated transition destination image (warp destination). In this case, a transition path will be formed in which the transition destination image by pressing the "Back" button 125 to return to the passed search path and the designated transition destination image (warp destination) happen to match. That is, in the search window 120 where the image of the "eraser" in Fig. 8 is displayed, whether the "Back" button 125 is pressed or the "Warp" button 128A is pressed, the search window 120 displaying the same image (the image of the "space on the desk") will be transitioned to. However, even when such registration is performed, if another search is performed on another occasion and the search path at that time is the search path of the integrated space → urban space → shopping street space → stationery store space → eraser (the path indicated by the dotted line in Fig. 18), pressing the "Back" button 125 will return to the image of the "stationery store space", while pressing the "Warp" button 128A can transition to the image of the "space on the desk". Therefore, since the transition destination images will not match, there is a meaning in the registration.

[0149] Also, as shown in Fig. 18, the candidates for the designated transition destination images (warp destinations) are not only the images of the layer directly above the arrival image (the image of the "eraser"). Since the user-specified images (favorites) registered in advance considering their future utility value can also be the images that become the center of the user's association, there is a possibility that the user may want to register them as the designated transition destination images (warp destinations). In the example of Fig. 18, the image of the "classroom space" is registered as the user-specified image (favorite), and this image of the "classroom space" is an image that the user wants to associate with the image of the "eraser".

[0150] In addition, as shown in FIG. 18, images obtained by keyword search from the hierarchical information are also described as candidates for the designated transition destination image (warp destination). If the user directly uses the name of a word associated with "eraser" as a search keyword, the image obtained by this search may be an image that the user wants to register as the designated transition destination image (warp destination). Also, the user may search for an image that they want to register as the designated transition destination image (warp destination) while referring to the hierarchical information.

[0151] FIG. 19 shows an image that can be transitioned from any reachable image reached by search after the user newly registers a designated transition destination image (warp destination). When the reachable image is not an image in the lowest layer, the reachable image includes several image selection parts for transitioning to images in the layer directly below it. Therefore, if the user selects any of these image selection parts, they can transition to the image in the layer directly below. Also, when the "Back" button 125 is pressed, the user can return to the previous image in the search path. Furthermore, when the "Warp" button 128A is pressed, the user can transition to the designated transition destination image (warp destination) newly registered by the user.

[0152] Specifically, when the user clicks the "Register Designated Transition Destination Image (Warp Destination)" button 128E (see FIGS. 4 to 8) provided in the search window 120, the designated transition destination image registration means 30 displays the designated transition destination image registration screen 400 (window) shown in FIG. 20 on the screen of the display means 60 and accepts the user's selection operation on this screen 400.

[0153] In FIG. 20, on the designated transition destination image (warp destination) registration screen 400, there is a display section 401 that displays the name of the currently displayed image, that is, the arrival image reached by the search (in the example of FIG. 20, the image of "eraser"), and a display section 402 for images that have already been registered as the designated transition destination image (warp destination) from the currently displayed image (arrival image) (in the example of FIG. 20, none). The display in the display section 401 is only the name, while the display in the display section 402 is the image itself (a reduced image). When the display in the display section 402 is "none", it is a new registration of the designated transition destination image (warp destination), and when an already registered designated transition destination image (warp destination) is displayed, it is a change registration of the designated transition destination image (warp destination).

[0154] Also, on the designated transition destination image (warp destination) registration screen 400, there is provided a designated transition destination image selection section 410 that displays the images in the layer immediately above the currently displayed image (arrival image) as candidates for the designated transition destination image (warp destination) and accepts the selection of those images. The images displayed in this selection section 410 are not only the names of the words but also the images themselves (reduced images). For example, in the examples of FIGS. 18 and 10, if the arrival image is the image of "eraser", then the images of "space on the desk", "space in the stationery store", and "space in the pencil case", which are the images in the layer immediately above it, are displayed as candidates.

[0155] Furthermore, on the designated transition destination image (warp destination) registration screen 400, there is a designated transition destination image selection unit 420 that displays user-specified images (favorites) in which image IDs (identification information of image data) are stored in the user-specified image storage means 48 (see FIG. 13) as candidates for the designated transition destination image (warp destination) and accepts selection of these images, and a "Do not display next time" button 421 for selecting not to display the images displayed in this designated transition destination image selection unit 420 as candidates for the designated transition destination image (warp destination) next time. The images displayed in the designated transition destination image selection unit 420 are not only the names of the words but also the images themselves (thumbnail images). When the "Do not display next time" button 421 is selected, the column of candidates for the designated transition destination image (warp destination) in the user-specified image storage means 48 (see FIG. 13) is changed to "0: Not applicable".

[0156] Also, on the designated transition destination image (warp destination) registration screen 400, there is an input unit 430 for search keywords, a "Search" button 431 for executing a search for candidates for the designated transition destination image (warp destination) using the input search keywords, and a designated transition destination image selection unit 432 that displays the searched images and the images of the immediately upper hierarchy as candidates for the designated transition destination image (warp destination) and accepts selection of these images. The images displayed in the designated transition destination image selection unit 432 are not only the names of the words but also the images themselves (thumbnail images). The search and display process using the search keywords by this designated transition destination image registration means 30 searches for words (names of words that are text data) that match or are similar to the search keywords from among the words (names of words that are text data) indicating the images included in the hierarchy information stored in the hierarchy information storage means 44 (see FIG. 10), and displays the images corresponding to the searched words and the images of the immediately upper hierarchy of this image.

[0157] Furthermore, on the designated transition destination image (warp destination) registration screen 400, there is a designated transition destination image selection unit 440 that displays the hierarchy of each image in a tree form using the hierarchy information stored in the hierarchy information storage means 44 (see FIG. 10) and accepts selection of the designated transition destination image (warp destination) by the user within this tree-form hierarchy display.

[0158] And at the bottom of the designated transition destination image (warp destination) registration screen 400, there is a bidirectional transition selection unit 450 for selecting to register the bidirectional transition path between the currently displayed image (arrival image) and the designated transition destination image (warp destination) selected on this designated transition destination image (warp destination) registration screen 400 of FIG. 20, and an "Apply" button 451 for applying the user settings (the user's selections are displayed in colors, blinking, etc.) on this designated transition destination image (warp destination) registration screen 400 of FIG. 20.

[0159] When the bidirectional transition selection unit 450 is not selected, only the one-way transition path from the currently displayed image (arrival image) to the designated transition destination image (warp destination) selected on this designated transition destination image (warp destination) registration screen 400 of FIG. 20 is registered. Therefore, the registration of the designated transition destination image (warp destination) of the present invention, that is, the registration of a new transition path by the user, is the registration of a directional transition path. For this reason, in the example of the designated transition destination image storage means 49 shown in FIG. 14, there is a record in which G00059132 (eraser) is stored in the column of the image ID of the arrival image and G00025784 (stationery store) is stored in the column of the image ID of the designated transition destination image (warp destination). However, the registration of these data only means the registration of the one-way transition path from the image of the "eraser" to the image of the "space of the stationery store". When registering the bidirectional transition path, in the record in which G00025784 (stationery store) is stored in the column of the image ID of the arrival image, G00059132 (eraser) must be stored in the column of the image ID of the designated transition destination image (warp destination) within that record.

[0160] Further, the designated transition destination image registration means 30 uses the date and time data in the final use date and time (initially the registration date and time) column stored in the designated transition destination image storage means 49 (see FIG. 14) and the system time of the computer to determine whether a predetermined time (for example, a period of six months, one year, etc.) has elapsed since the registration date and time when the final use date and time or use (use of the transition path from the arrival image to the designated transition destination image) has never occurred. If the time has elapsed, the registration of the designated transition destination image (warp destination) is invalidated, that is, a process of setting the valid flag indicating whether the designated transition destination image storage means 49 is valid or invalid to "0 (invalid)" is executed. At this time, before performing the invalidation process, the designated transition destination image registration means 30 may give a warning to the user that it will soon become invalid and then invalidate it. A user who does not want to invalidate it may use the transition path to the designated transition destination image (warp destination).

[0161] Furthermore, when the designated transition destination image registration means 30 registers a designated transition destination image for enabling a transition from the arrival image to the designated transition destination image, it accepts the user's selection to also register a transition from the reverse-direction designated transition destination image to the arrival image (the selection of checking the two-way transition selection unit 450 in FIG. 20). When the selection of this reverse-direction transition is accepted, it is assumed that the image currently intended to be registered as the designated transition destination image becomes a temporary arrival image in another search, and the image currently serving as the arrival image becomes a temporary designated transition destination image in another search. Then, it executes a process of associating the image ID (identification information of image data) of the temporary designated transition destination image in this assumed state with the image ID (identification information of image data) of the temporary arrival image and storing it in the designated transition destination image storage means 49 (see FIG. 14). Therefore, as already described in detail, in the example of the designated transition destination image storage means 49 in FIG. 14, since a transition path with directionality is registered in one record, when registering the reverse-direction transition path simultaneously, data registration to two records will be performed. Note that the registration of the reverse-direction transition path can also be achieved by performing the registration of the one-direction transition path twice. In that case, in the first search, the image registered as the designated transition destination image is reached in a subsequent another search (the second search), and at the reached image, the registration of the transition path in the reverse direction to the transition path registered in the first search can be performed. That is, the arrival image in the first search may be set as the designated transition destination image in the second search, and the designated transition destination image in the first search may be set as the arrival image in the second search. Performing the registration of the two-way transition path in a batch corresponds to performing the registration processes in these two searches (the first search and the second search) simultaneously. However, since the second search is an assumed another search and not an actual search, as described above, the explanation uses the terms "temporary arrival image" and "temporary designated transition destination image".

[0162] <Configuration of the processing means 20A / manufacturing method manual display processing means 31>

[0163] When the user clicks the "Reference Manufacturing Manual" button 128G (see FIGS. 4 to 8) provided in the search window 120, the manufacturing method manual display processing means 31 uses the image ID (identification information of image data) of the reached image obtained by the search and the manufacturing method manual data (including recipe data and breeding manual data) stored in the manufacturing method manual storage means 46, which is associated with the image ID, to execute a process of displaying a manufacturing method manual display screen (a window not shown) indicating the manufacturing method of an object corresponding to the word pointed to by the image on the screen of the display means 60. The manufacturing method manual data may be moving image data (including moving image data including audio data), still image data, planar image data, or three-dimensional image data.

[0164] <Configuration of Processing Means 20A / Manufacturing Method Manual Acquisition Means 32>

[0165] Based on the user's selection operation on the setting screen 100 (see FIG. 3), the manufacturing method manual acquisition means 32 downloads and acquires any one of the manufacturing method manual data (including recipe data and breeding manual data) created by each publishing house from the data providing server 70 connected via the network 1, and stores this in the manufacturing method manual storage means 46 in association with the image ID (identification information of image data). The data providing server 70 stores the manufacturing method manual data in association with the image ID.

[0166] <Configuration of Processing Means 20A / Problem Processing Means 33>

[0167] The problem processing means 33 uses the problem data stored in the problem storage means 51 (see FIG. 16) associated with the image ID (identification information) of the image data of the question source for which a word is to be explained on the screen displayed on the display means 60, displays a question including an image schema learning question for the user on the screen of the display means 60, accepts the user's answer, determines the correctness of the answer using the information indicating the correct answer (the object for the question to be moved and its destination area) stored in the problem storage means 51, displays the determination result on the screen of the display means 60, and executes a process of storing it in the problem storage means 51 in association with the image ID (identification information) of the image data of the question source.

[0168] Specifically, when the "Learn by Solving Problems" button 128F (see FIGS. 4 to 8) provided in the search window 120 is clicked, the problem screen 500 (window) shown in FIG. 21 is displayed on the screen of the display means 60.

[0169] In FIG. 21, on the problem screen 500, there are provided an image display section 501 for displaying a problem background image and several problem objects superimposed thereon, a problem display section 510 for displaying an instruction text of the problem using the image displayed in the image display section 501, a correct / incorrect determination result display section 511 for displaying the correctness of the user's answer to the problem (a circle "〇" for correct and a cross "×" for incorrect), a "Return to Search Screen" button 520 for ending the learning of solving the problem and returning to the search window 120 (search screen), a "Next Problem" button 521 for moving to the next problem, and an "Evaluate" button 522 for evaluating the user using the correct / incorrect information of the user's answer to the problem and the challenge status of the problem. Note that the "Evaluate" button may be provided on the setting screen 100 in FIG. 3.

[0170] The image display unit 501 displays a problem background image corresponding to the image of the question source (the image displayed when the user presses the "Solve problems and learn" button 128F) and several problem objects superimposed thereon. Therefore, if the image of the question source is an image of a space, the problem object corresponds to an object belonging to that space, and if the image of the question source is an image of an object, the problem object corresponds to a component of that object. However, it is not necessary for the patterns to match exactly, nor is it necessary for the number of image selection parts in the image of the question source to match the number of problem objects. Since the user presses the "Solve problems and learn" button 128F with the intention of learning with the image (the space or object shown in the image) of the question source, it is sufficient if the image of the question source, the problem background image, and the problem objects superimposed thereon are similar to the extent that they do not betray that expectation.

[0171] In the example of FIG. 21, the image display unit 501 includes a problem background image 160A corresponding to the image 160 of the "space on the desk" in FIG. 7, a problem object 161A corresponding to the image selection part 161 shown by the pattern of the "pencil" in the image 160 of the "space on the desk" in FIG. 7, a problem object 162A corresponding to the image selection part 162 shown by the pattern of the "eraser" in FIG. 7, a problem object 163A corresponding to the image selection part 163 shown by the pattern of the "phone" in FIG. 7, a problem object 164A corresponding to the image selection part 164 shown by the pattern of the "calculator" in FIG. 7, a problem object 165A corresponding to the image selection part 165 shown by the pattern of the "dictionary" in FIG. 7, problem objects 166A and 166B corresponding to the image selection part 166 shown by the pattern of the "notebook" in FIG. 7, a problem object 167A corresponding to the image selection part 167 shown by the pattern of the "photo frame" in FIG. 7, and a problem object 168A corresponding to the image selection part 168 shown by the pattern of the "reference book" in FIG. 7.

[0172] Here, the problem object corresponding to the "notebook" is divided into the problem object 166A on the left side of the notebook and the problem object 166B on the right side of the notebook in order to teach the user the operation of closing the notebook in the instruction text of the problem "Please close the open notebook."

[0173] In addition, the image display unit 501 is provided with a destination area 530 indicating "inside the drawer" in the open state. In the example of FIG. 21, the "drawer" is not the problem object to be moved, but since a destination area for another problem object (here, an eraser) is required, the destination area 530 is provided. When preparing an instruction text for the problem "Please open the drawer.", the "drawer" may be set as the problem object to be moved and its destination area may be prepared.

[0174] Furthermore, the image display unit 501 is provided with a destination area 531 indicating "on the notebook". The meaning of "on the notebook" in the instruction text of the problem "Please place the pencil from under the phone onto the notebook." has two meanings: placing it on top of the notebook (meaning the upper position in the vertical direction of the screen) and the upper position in the planar spread of the screen displayed by the display means 60. Therefore, in order to consider both cases as correct answers, the destination area 531 is provided wider than the picture part of the notebook.

[0175] In addition, the image display unit 501 is provided with a destination area 532 indicating "the left side of the notebook" and a destination area 533 indicating "the right side of the notebook". In order to consider both cases as correct answers: moving the problem object 166B on the right side of the notebook to the destination area 532 indicating the left side of the notebook and moving the problem object 166A on the left side of the notebook to the destination area 533 indicating the right side of the notebook in response to the instruction text of the problem "Please close the open notebook.", the destination areas 532 and 533 are provided.

[0176] Furthermore, the image display unit 501 is provided with a destination area 534 indicating "to the right of the phone". This is an area that gives the correct answer to the instruction text of the problem "Please place the calculator to the right of the phone."

[0177] As shown in FIG. 21, the problem processing means 33 uses the problem background image stored in the problem image data storage means 50 (see FIG. 15) associated with the image ID of the image data of the source of the problem (the image data displayed when the user presses the "Solve problems and learn" button 128F) and a plurality (usually a plurality, but one may also be sufficient) of problem objects to be superimposed thereon on the image display unit 501 of the problem screen 500 to display the image to be used for the problem (in the example of FIG. 21, the image of "the space on the desk"). Therefore, in the example of FIG. 21, the user searches and reaches the image 160 of "the space on the desk" in FIG. 7, and hopes to acquire the language with the reached image of "the space on the desk".

[0178] Then, the problem processing means 33 displays the problem instruction text in a foreign language such as Chinese that the user wants to acquire, using the problem data of each record stored in the problem storage means 51 (see FIG. 16), on the problem display section 510 of the problem screen 500. In the example of FIG. 21, it displays the foreign language such as Chinese corresponding to the Japanese sentence "Please put the eraser into the drawer." with the problem number = Q1. The language of the problem instruction text displayed on this problem display section 510 is the language set on the setting screen 100 of FIG. 3 (when multiple languages are set, there will be multiple). The problem instruction text is displayed in the display order of the languages set on the setting screen 100 (that is, the selection order of the languages in each language selection section 101, or the order changed by pressing the "Change Language Display Order" button 106). For example, if Chinese, German, and French are selected in this order on the setting screen 100 of FIG. 3, the problem instruction texts will be displayed in parallel in this order on the problem display section 510 of the problem screen 500 in FIG. 21. That is, German and French are displayed below Chinese (or at the position following Chinese). This is the same as the parallel display of word explanations in multiple languages on the explanation display section 122 (see FIGS. 5 to 8) of the search window 120. Also, when performing voice reading for the problem instruction text displayed on the problem display section 510, the reading is performed in the language first selected on the setting screen 100 of FIG. 3 (if it is changed by pressing the "Change Language Display Order" button 106, the first language after the change).

[0179] At this time, the user grasps the meaning of the problem instruction text (foreign language such as Chinese), and according to the instruction, drags the problem object 162A of the "eraser" that is the object to be moved, moves it to the destination area 530 indicating "inside the drawer", and drops it there.

[0180] The same applies to other problems. When the foreign language such as Chinese corresponding to the Japanese sentence "Please place the pencil on top of the notebook, under the phone." with the problem number = Q2 is displayed, the user drags the problem object 161A of the "pencil" and moves it to the destination area 531 indicating "on top of the notebook", and drops it there.

[0181] Also, when a foreign language such as Chinese corresponding to the Japanese sentence "Please close the open notebook." with question number = Q3 is displayed, the user drags the problem object 166A at the "left side of the notebook" to the destination area 533 indicating the "right side of the notebook" and drops it there, or drags the problem object 166B at the "right side of the notebook" to the destination area 532 indicating the "left side of the notebook" and drops it there.

[0182] Furthermore, when a foreign language such as Chinese corresponding to the Japanese sentence "Please place the calculator to the right of the phone." with question number = Q4 is displayed, the user drags the problem object 164A of the "calculator" to the destination area 534 indicating the "right of the phone" and drops it there.

[0183] Then, the problem processing means 33 uses the information indicating the correct answer (the problem object to be moved and the destination area) stored in the problem storage means 51 (see FIG. 16) to determine whether the user's answer is correct or incorrect, and stores the result of the correct / incorrect determination in the problem storage means 51. That is, if it is correct, the data of the cumulative correct answer count is incremented by 1, the data of the most recent correct / incorrect is updated to "1: correct answer", and the data of the most recent correct / incorrect until then is copied and transferred to the data of the previous correct / incorrect. If it is incorrect, the data of the cumulative incorrect answer count is incremented by 1, the data of the most recent correct / incorrect is updated to "0: incorrect answer", and the data of the most recent correct / incorrect until then is copied and transferred to the data of the previous correct / incorrect.

[0184] The questions with these question numbers Q1 to Q4 are questions for image schema learning. This image schema is a term used in cognitive linguistics and forms the basis of language. Therefore, if a user does not learn and correctly understand the image schema, they cannot reach the ideal state of using the learned language while applying it in many scenarios. For example, entering a room, exiting a room. Putting money into a wallet, taking money out of a wallet. The common concept in these is the image of a container, where the room and the wallet are containers. Therefore, if one has acquired this image schema, in the scenarios of entering and exiting a room and the scenarios of putting money in and taking money out of a wallet, one can understand them by applying language without having to learn each of them individually.

[0185] <Configuration of Processing Means 20A / Evaluation Means 34>

[0186] The evaluation means 34 uses the judgment result of the correctness of the answer stored in the question storage means 51 (see FIG. 16) and the image ID (identification information) of the image data of the question source (the image data displayed when the user presses the "Solve the problem and learn" button 128F), and the employment recommendation field stored in the question image data storage means 50 (see FIG. 15) associated therewith, and executes a process of displaying information on the field of the employment destination recommended to the user on the screen of the display means 60.

[0187] Also, the evaluation means 34 uses the judgment result of the correctness of the answer of the image schema learning question of the same question type stored in the question storage means 51 (see FIG. 16) for each of the plurality of image data, determines whether the user understands the image schema of the question type, and also executes a process of displaying the judgment result on the screen of the display means 60.

[0188] Specifically, when the "Evaluate" button 522 on the question screen 500 in FIG. 21 or the "Evaluate" button (not shown) on the setting screen 100 in FIG. 3 is pressed, the evaluation report screen 600 (window) shown in FIG. 22 is displayed on the screen of the display means 60.

[0189] In FIG. 22, on the evaluation report screen 600, a [Recommended Employment Destination] display section 610 and a [Recommended Future Learning] display section 620 are provided.

[0190] In the [Recommended Employment Destination] display section 610, a plurality of recommended fields can be displayed, and they are displayed in a divided state such as <First Recommended Field>, <Second Recommended Field>,.... For example, "Your employment destination is suitable for a 'General Affairs Position'." etc. are displayed. This "General Affairs Position" is stored in the employment recommendation field column of the problem image data storage means 50 (see FIG. 15). In the display section of each recommended field, the reason for recommendation is described, and (A) the reason for recommendation from the perspective of the presence or absence of interest and (B) the reason for recommendation from the perspective of the level of correct answer rate are described.

[0191] As the reason for recommendation from the perspective of (A) the presence or absence of interest, for example, "You are actively working on problems of images related to the 'General Affairs Position'." etc. are displayed. And the image data associated with the "General Affairs Position" is not limited to one, and usually there are a plurality of them. Therefore, the total number of the cumulative correct answer count and the cumulative incorrect answer count stored in the problem storage means 51 (see FIG. 16) associated with the image IDs (identification information) of those multiple image data is calculated and displayed as the cumulative number of times the problem has been answered. That is, there are multiple image data associated with the "General Affairs Position", and for each of those image data, there are multiple problems. So, all the data in the columns of their cumulative correct answer count and cumulative incorrect answer count are totaled. Furthermore, for the image data with a particularly large total number among them, "In particular, you are actively working on problems in the image of 'the space on the desk'." etc. are displayed, and the total number of the cumulative correct answer count and the cumulative incorrect answer count associated with the image ID of that image data is displayed as the cumulative number of times the problem has been answered. Note that the image data with a particularly large total number may be the image data whose total number is equal to or exceeds the threshold value, or may be the top predetermined number of pieces (for example, the top 3 pieces) of image data.

[0192] (B) As a reason for recommendation from the perspective of the level of correct answer rate, for example, "You have a high correct answer rate for the questions in the images related to 'general affairs positions' recently (the most recent and the previous one). " etc. are displayed. And the image data associated with "general affairs positions" is not limited to one, and usually there are multiple. Therefore, using all the data in the columns of the most recent correct / incorrect and the previous correct / incorrect stored in the question storage means 51 (see FIG. 16) associated with the image IDs (identification information) of those multiple image data, divide the number of data with "1: correct" by the total number of data (the total of the number of data with "1: correct" and the number of data with "0: incorrect") to obtain and display the average correct answer rate. The reason for obtaining the average correct answer rate including the data in the column of the previous correct / incorrect is to exclude the influence in case of accidentally getting the correct answer in the most recent test. In addition, it is also possible to obtain the average correct answer rate only using the data in the column of the most recent correct / incorrect, or to set columns for the correct / incorrect two times ago or even earlier, and use all of those data to obtain the average correct answer rate. Furthermore, among them, for the image data with particularly high correct answer rate, for example, "In particular, the correct answer rate for the questions in the image of 'the space on the desk' recently (the most recent and the previous one) is high. " etc. are displayed, and using all the data in the columns of the most recent correct / incorrect and the previous correct / incorrect associated with the image ID of that image data, the average correct answer rate is obtained and displayed.

[0193] [Recommendation for future learning] The display unit 620 is provided with a display unit 621 for <image schema that may not be understood> and a display unit 622 for <fields where learning is incomplete>.

[0194] In the display section 621 of <Image Schema that may not be understood>, for example, there may be a possibility that an image schema (IS01) such as "put ~ into ~" is not understood. " etc. are displayed, and the reasons for the judgment and recommendations for learning are described. This display content is described in advance in the program in association with the problem type (IS01) of the image schema learning problem. Note that by using the title <Image Schema that may not be understood>, it is displayed as a possibility without making an assertion because it is not that the image schema is not understood and is incorrect, but rather there is a possibility that the meaning of the words included in the problem instruction text is not understood, resulting in an incorrect answer. Therefore, even if an image schema learning problem of a certain problem type has not been solved at all (even if all are incorrect), it may be because the meaning of the words is not understood, so it is displayed as a possibility without making an assertion.

[0195] And, for example, "The answer to the question (IS01) in the image of 'the space on the desk' (general office work) is correct." etc. are displayed, and the question "Q1: Please put the eraser into the drawer." etc. is displayed. Therefore, this shows that it is not the case that the image schema (IS01) is not understood at all.

[0196] On the other hand, a display is made indicating a high possibility that the image schema (IS01) is not understood. For example, "However, the answer to the question (IS01) in the image of 'the space of soccer' (sports field) is incorrect." etc. are displayed, and the question "Q4: Please put the soccer ball into the goal." is displayed. Similarly, for example, "The answer to the question (IS01) in the image of 'the space at the cash register' (accounting field) is incorrect." etc. are displayed, and the question "Q8: Please put the change into the wallet." is displayed.

[0197] As learning recommendations, for example, "Problems with the same image schema (IS01) also exist in other images, so please give it a try." etc. are displayed. And for the image of "Police station space" which is a problem of image schema (IS01) (employment recommendation field = civil servant), "Q5: Please put the prisoner in the cell.", for the image of "Park space" (employment recommendation field = administrator), "Q3: Please put the lunch box in the trash can." etc. are displayed. These problems are extracted using the data in the problem type column stored in the problem storage means 51 (see Fig. 16). Note that if the problem in that image has already been solved (if the correct / incorrect judgment result is stored), it will not be extracted.

[0198] In the display section 622 of <the field where learning is incomplete>, for example, "Learning in the image of the food and beverage industry is incomplete." etc. are displayed. And as the reason for the judgment, for example, "The average correct answer rate in the image of 'Chinese restaurant' (food and beverage industry) is low.", "The average correct answer rate in the image of 'French restaurant' (food and beverage industry) is low." etc. are displayed. The calculation method of the average correct answer rate is the same as in the case of the recommendation reason from the perspective of the high or low (B) correct answer rate in the [Employment destination recommendation] display section 610. Extract images that have been challenged with a problem at least once (images for which the correct / incorrect judgment result is stored in the problem storage means 51 (see Fig. 16)) and whose average correct answer rate is equal to or below the threshold value.

[0199] <Configuration of Processing Means 20A / Problem Image Data and Problem Data Acquisition Means 35>

[0200] Based on the user's operation, the problem image data and problem data acquisition means 35 downloads and acquires the problem image data and problem data from the data providing server 70 connected via the network 1, and stores them in the problem image data storage means 50 (see Fig. 15) and the problem storage means 51 (see Fig. 16) in association with the image ID of the image data of the question source. In the data providing server 70, the problem image data and problem data are stored in association with the image ID of the image data of the question source.

[0201] In addition, if the user does not perform learning to solve problems, the user does not have to download the problem image data and the problem data. Also, for the programs (including hierarchical information and image coordinate information) and image data that are essential for the operation of this system, the user must first download and obtain them from the data providing server 70 (see step S1 in FIG. 2 described later). However, for the problem image data and the problem data, since they only need to be downloaded and obtained before performing learning to solve problems, it is not necessary to obtain them from the beginning.

[0202] <Configuration of the storage means 40 / image data storage means 41>

[0203] The image data storage means 41 stores the image data for displaying the image of the space as an image file for each space in association with an image ID (identification information of the image data), and stores the image data for displaying the image of an object or its constituent articles belonging to at least one of these spaces as an image file for each object or each constituent article in association with the image ID. As already described in detail, these image data include, for example, not only illustration data but also actual photograph data, and include not only still image data but also moving image data. And these image data may be planar image data or three-dimensional image data.

[0204] <Configuration of the storage means 40 / description data storage means 42>

[0205] The description data storage means 42 stores the description data (here, text data) of the words referring to the space in association with an image ID (identification information of the image data) for each space, and stores the description data (here, text data) of the words referring to an object or its constituent articles in association with the image ID for each object or each constituent article. As already described in detail, these description data include, for example, the Japanese notation and reading of each word, the notation (Chinese characters in the case of Chinese) and reading (pinyin, which is the pronunciation symbol in the case of Chinese), the usage method, etc.

[0206] In addition, when a user (a person who performs a search) who conducts foreign language learning downloads explanation data of words in a plurality of languages from the data providing server 70 by the explanation data acquisition means 25, the explanation data storage means 42 stores the explanation data of words in the plurality of languages.

[0207] <Configuration of the storage means 40 / audio data storage means 43>

[0208] The audio data storage means 43 stores, as audio files for each space, audio data for outputting the pronunciation of a word indicating a space, in association with an image ID (identification information of image data), and stores, as audio files for each object or each component article, audio data for outputting the pronunciation of a word indicating an object or its component article, in association with the image ID. When a user (a person who performs a search) who conducts foreign language learning downloads audio data of words in a plurality of languages from the data providing server 70 by the audio data acquisition means 26, the audio data storage means 43 stores the audio data of words in the plurality of languages. However, the audio data used is, in principle, audio data for outputting the pronunciation of the language first selected on the setting screen 100 (see FIG. 3).

[0209] <Configuration of the storage means 40 / hierarchy information storage means 44>

[0210] As shown in FIG. 10, the hierarchy information storage means 44 stores hierarchy information for hierarchically arranging each image data according to any one of the subordinate relationships between spaces, the subordinate relationships between a space and an object, or the subordinate relationships between an object and its component articles. This hierarchy information is information for grasping what the images in the lower hierarchy of a certain image are, or what the images in the upper hierarchy of a certain image are. For example, it defines a subordinate relationship as shown by the tree in FIG. 9.

[0211] Specifically, as shown in FIG. 10, the hierarchical information storage means 44 stores, in an associated manner, an image ID (identification information of image data), the name of the word represented by the image, the attributes of the image (classification into space, object, or component article), the image ID of the first upper layer (the immediately upper layer) to which the self (image data of the image ID in the leftmost column) belongs, the image ID of the second upper layer (the immediately upper layer), the image ID of the third upper layer (the immediately upper layer), and so on.

[0212] For one piece of image data (image data identified by a certain image ID), there may be multiple immediately upper layers, namely the first, second, third, etc. This is because the immediately upper layer to which one piece of image data belongs is not limited to one and may be multiple. In the topmost layer (the general space in the example of FIG. 10), there is no immediately upper layer. In FIG. 10, for reasons of the drawing space, only the first to third immediately upper layers are illustrated, but more columns of immediately upper layers can be provided according to the total number of words.

[0213] In the example of FIG. 10, the image data with the name "eraser" (image ID = G00059132) belongs to the image data of "desk (space above the desk)" (image ID = G00042821), the image data of "stationery store (space of the stationery store)" (image ID = G00025784), and the image data of "pencil case (space of the pencil case)" (image ID = G00026705). Therefore, when the image selection part (the eraser pattern part) included in each of these space images is selected, it is possible to transition to the image of "eraser (the object called eraser)".

[0214] Also, in the example of FIG. 10, the "tennis ball" is subordinate to the space of the "tennis court" and the space of the "sports goods store". Therefore, to reach the "tennis ball", one can perform a search of general space → city → park → tennis court, or perform a search of general space → city → shopping street → sports goods store → tennis court. If there are too many items handled in the "sports goods store", one may also reach the "tennis ball" via the space of "tennis supplies (tennis supplies sales area within the sports goods store)" from the space of the "sports goods store". Additionally, one may also reach the "tennis ball" from the space of the "tennis shop".

[0215] Note that, as described above, when performing a search and reaching a certain image, since there are multiple search paths, the level (hierarchy) at which a certain image is located, counted from the topmost layer (the general space in the example of FIG. 10), varies depending on the search path passed through. For example, as shown in FIGS. 18 and 10, if one performs a search of general space → house → study → desk → eraser to reach the "eraser", the eraser is at the 5th layer from the top. Even if one performs a search of general space → city → shopping street → stationery store → eraser, the eraser is still at the 5th layer from the top. However, if one performs a search of general space → city → elementary school → schoolbag → pencil case → eraser, the eraser is at the 6th layer from the top.

[0216] Also, if it is determined which image in the immediately upper layer is the image of which layer from the top, the layer number (level of layer depth) of the own (target image) is determined by the layer number obtained by adding 1 to the layer number of the image in the immediately upper layer. However, since it is not determined which layer the immediately upper layer itself is from the top, the layer number of the target image is not determined. There are multiple search paths to reach the image in the immediately upper layer, and if the number of images passed through by these search paths is different, it cannot be determined which image in the immediately upper layer is the image of which layer from the top. For example, if there are images X, Y, and Z, and it is possible to reach image Y from image X through search paths α1, α2, α3, and further reach image Z from image Y through search paths β1, β2, then there are 3×2 = 6 search paths from image X to image Z. However, if the number of images passed through by each of the search paths α1, α2, α3 and search paths β1, β2 is different, the layer number (level of layer depth) of image Z may be six different values.

[0217] Also, in the layer information storage means 44 shown in FIG. 10, the image ID of the image data of the layer immediately above the own (image data of the said image ID) is stored in the record of the image data of a certain image ID. However, the layer information storage means of the present invention is not limited to such a configuration. For example, the image ID of the image data of the layer immediately below the own (image data of the said image ID) that is subordinate to the own may be stored in the record of the image data of a certain image ID. Further, instead of storing all the image IDs of the image data of the layer immediately above the own in one record, or storing all the image IDs of the image data of the layer immediately below the own, one record may store the image data in a subordinate relationship in a one-to-one manner.

[0218] <Configuration of the storage means 40 / image coordinate information storage means 45>

[0219] As shown in FIG. 11, the image coordinate information storage means 45 stores X and Y coordinates for designating the area of an image selection section for selecting an image in a lower layer (the immediately lower layer) included in the image displayed on the screen of the display means 60. The area designated by these X and Y coordinates becomes the image selection section. When the mouse pointer is placed on this area and clicked, or when it is touched with a finger or the like, the selection operation is received by the selection reception processing means 21, and the screen switching processing means 22 switches to a screen for displaying an image in a lower layer (the immediately lower layer) corresponding to the pattern drawn in this area. The shape of the area of the image selection section is, for example, a rectangle or a square, and is determined by designating the X and Y coordinates of the upper left position and the X and Y coordinates of the lower right position.

[0220] Specifically, as shown in FIG. 11, the image coordinate information storage means 45 stores, in association with each other, an image ID (identification information of a certain image data), the image ID of the image data in the upper layer (the immediately upper layer) to which the own (the said image data) belongs, and the position of the image selection section (the X and Y coordinates of the upper left position and the X and Y coordinates of the lower right position) in the image data of the upper layer (the immediately upper layer). In the example of FIG. 11, as the images in the immediately upper layer to which the image of the object "eraser" belongs, there are images of three spaces, namely "desk", "stationery store", and "pencil case" (see FIG. 18). Therefore, for the "eraser", there are three records. Also, considering the first column from the left in the example of FIG. 11 as the image ID of the image data in the lower layer (the immediately lower layer), the second column from the left as the own image ID, and the third column from the left as the position of the image selection section for transition to an image in the lower layer (the immediately lower layer) of the own image (one of the plurality of images in the immediately lower layer), it is the same thing. In short, FIG. 11 associates the image IDs of the image data in a subordinate relationship one-to-one.

[0221] Note that the image coordinate information storage means of the present invention is not limited to the configuration of the image coordinate information storage means 45 shown in FIG. 11. For example, in one record, all the image IDs of the image data of the layer immediately above itself may be stored, and the positions of the image selection parts (image selection parts for transition to itself) in each of the image data of the layer immediately above them may be stored. Alternatively, in one record, all the image IDs of the image data of the layer immediately below itself may be stored, and the positions of the image selection parts for transition to each of the image data of the layer immediately below them (image selection parts selected when transitioning from itself to each of the image data of the layer immediately below) may be stored.

[0222] <Configuration of the storage means 40 / manufacturing manual storage means 46>

[0223] The manufacturing manual storage means 46 stores manufacturing manual data (including recipe data and breeding manual data) for objects corresponding to words (including foods and animals and plants) in association with the image ID (identification information of the image data) of the image data indicating the word.

[0224] <Configuration of the storage means 40 / cumulative passage count storage means 47>

[0225] As shown in FIG. 12, the cumulative passage count storage means 47 stores the cumulative passage count (that is, the cumulative number of times the switching process to the screen including each image data is performed by the screen switching process means 22) of the user's search and passing through each image data, and a flag (1: corresponding, 0: not corresponding) indicating whether it corresponds to the exclusion start image (image to be excluded from the candidates for the start image) in association with the image ID of the passed image data (main image data for which the word is the object of explanation).

[0226] The update process of this cumulative passage count is automatically executed by the screen switching process means 22, and the zero clearing process of the cumulative passage count and the reception process of setting and changing the flag indicating whether it corresponds to the exclusion start image are executed by the setting means 27 based on the user's operation.

[0227] <Configuration of Memory Unit 40 / User-Specified Image Memory Unit 48>

[0228] As shown in FIG. 13, the user-specified image memory unit 48 stores by associating the image ID of the user-specified image (favorite image) specified and registered by the user, a flag indicating whether the registered user-specified image corresponds to a candidate for the start image (1: corresponding, 0: not corresponding), and a flag indicating whether it corresponds to a candidate for the specified transition destination image (warp destination) (1: corresponding, 0: not corresponding).

[0229] The registration of this user-specified image is executed by the user-specified image registration means 28, and the change of the flag indicating whether it corresponds to a candidate for the start image (change from 1 to 0) is executed by the start image selection means 29 (see the "Do not display from next time" button 322 on the user-specified image use start image selection screen 320 in FIG. 17), and the change of the flag indicating whether it corresponds to a candidate for the specified transition destination image (warp destination) (change from 1 to 0) is executed by the specified transition destination image registration means 30 (see the "Do not display from next time" button 421 on the specified transition destination image (warp destination) registration screen 400 in FIG. 20).

[0230] <Configuration of Memory Unit 40 / Specified Transition Destination Image Memory Unit 49>

[0231] As shown in FIG. 14, the specified transition destination image memory unit 49 stores by associating the image ID (identification information of image data) of the arrival image (the currently displayed image reached by searching, which is the image of the transition source), the image ID of the specified transition destination image (warp destination), the latest use date and time (initially the registration date and time) when the user used the transition path from the arrival image to the specified transition destination image (warp destination), and a valid flag indicating whether the transition path from the arrival image to the specified transition destination image (warp destination) is currently valid or invalid.

[0232] The registration process of the image ID of the specified transition destination image (warp destination) is executed by the specified transition destination image registration means 30. The registration process of the first registration date and time in the column of the final use date and time (initially the registration date and time) is executed by the specified transition destination image registration means 30. Subsequently, the update registration process of the final use date and time is executed by the screen switching process means 22. The process of changing the valid flag to invalid is executed by the specified transition destination image registration means 30 at the startup or during the operation of the system.

[0233] <Configuration of the storage means 40 / the problem image data storage means 50>

[0234] As shown in FIG. 15, the problem image data storage means 50 stores the image ID (identification information) of the image data of the question source, the employment recommendation field, the storage destination path of the file of the problem background image, the problem object 1, the problem object 2,... in association with each other.

[0235] Here, the employment recommendation field is an image (for example, an image of a certain space). If several questions are solved and learned in this image, the language knowledge in the field related to this image will increase, so it is stored for each image. For example, in the image of the "operating room space", if many questions using surgical instruments such as "scalpel" are solved and answered correctly, the medical field will become the employment recommendation field. Also, in the image of an object called "personal computer", if many questions using its component parts such as "CPU" and "main memory" are solved and answered correctly, the IT field will become the employment recommendation field.

[0236] Also, the problem background image is an image similar to the image of the question source (the image displayed in the search window 120), but it does not need to match exactly. For example, in the image of the "space on the desk" displayed in the search window 120, even if the drawer is closed, in the problem background image, the drawer may be open.

[0237] Furthermore, the image (the main image for which the word is to be explained) displayed in the search window 120 includes an image selection part which is a pattern part, and this image selection part corresponds to an object or a component article in the immediately lower layer. In contrast, the pattern part superimposed on the background image for the problem is called an object rather than an object or a component article. The image selection part is provided for transitioning to the next screen by a selection operation such as clicking or touching, whereas the object is provided for moving within the same screen by a drag operation. Note that the object of the note is divided into an object on the left side of the note and an object on the right side of the note in order to let the user learn the closing operation.

[0238] <Configuration of the storage means 40 / problem storage means 51>

[0239] As shown in FIG. 16, the problem storage means 51 stores problem data including an image schema learning problem using an object included in an image of a space or a component article included in an image of an object as a theme, and a problem object to be moved indicating a correct answer and its destination area in association with the image ID (identification information of image data) of the image of the question source.

[0240] More specifically, as shown in FIG. 16, the problem storage means 51 associates and stores the image ID (identification information) of the image data of the question source, the question number, the question type, the instruction of the question (a foreign language such as Chinese), the corresponding instruction of the question (a native language such as Japanese), the problem object to be moved, the destination area (in this embodiment, the upper left position X, Y and the lower right position X, Y of a rectangle or a square), the cumulative number of correct answers, the cumulative number of incorrect answers, the most recent correct / incorrect (1: correct answer, 0: incorrect answer), the previous correct / incorrect (1: correct answer, 0: incorrect answer), etc. Also, in the example of FIG. 16, only Chinese is described for the instruction of the question (foreign language) due to space limitations for description, but the problem storage means 51 stores instructions of questions in many other languages (foreign languages) such as German and French.

[0241] Here, the problem type is information for identifying whether it is a problem for image schema learning and further for identifying what kind of image schema the problem is for learning. With this problem type, it is possible to grasp that the same image schema is being learned in various images and in images in various fields, and it becomes possible to determine whether the user has acquired that image schema.

[0242] Also, the object for the problem to be moved is an object that the user drags on the screen according to the problem instruction, and the destination area is an area where the user drops the object for the problem according to the problem instruction. Therefore, these objects for the problem to be moved and the destination areas are information indicating the correct answer. In the record of the problem instruction "Please close the open notebook.", it is memorized such that dragging the left side of the notebook to the right area or dragging the right side of the notebook to the left area both result in the correct answer.

[0243] Furthermore, the most recent correct / incorrect and the correct / incorrect from one time ago are information indicating whether the user's answer to the most recently solved problem is correct or incorrect. When a new problem (the same problem) is solved, the correctness of the answer is memorized as the most recent correct / incorrect data, and the most recent correct / incorrect data is shifted and memorized as the correct / incorrect data from one time ago.

[0244] <Workflow of a user using the information retrieval system 10>

[0245] In such an embodiment, a user who conducts foreign language learning performs the following: while searching for information related to foreign language words using the information retrieval system 10, the user acquires the foreign language. FIG. 2 shows, in a flowchart, the operations that the user should perform at this time, and FIGS. 3 to 9 show an example of the screen transition of the information retrieval system 10 accompanying the search (mainly the flow that was also performed in the conventional information retrieval system described in Patent Document 1 mentioned above).

[0246] In FIG. 2, first, the user downloads and obtains the program of the information retrieval system 10 and the image data from the data providing server 70, or purchases a recording medium such as a CD-ROM in which the program and the image data are stored at a retail store (step S1), and installs the obtained program (including hierarchical information and image coordinate information) and the image data in the main body 20 (step S2).

[0247] Next, when the user activates the program (step S3), a setting screen 100 (window) as shown in FIG. 3 is displayed on the screen of the display means 60. In FIG. 3, on the setting screen 100, there are provided ten language selection parts 101 for selecting the language to be used, a publisher selection part 102 for selecting which publisher's created explanatory data, voice data, and manufacturing method manual data to use, and a "Download" button 103 for downloading the explanatory data, voice data, and manufacturing method manual data according to the selections in each language selection part 101 and the publisher selection part 102.

[0248] Since ten language selection parts 101 are provided in the present embodiment, it is possible to select up to ten languages and display the explanations of words on the screen. And each language selection part 101 can select all languages and dialects of all countries in the world. Also, the explanations of the words in the language selected by each language selection part 101 are displayed on the screen in the same order as the order of selection of the languages in each language selection part 101. That is, in the illustrated example, Chinese is selected by the uppermost language selection part 101, and German is selected by the second language selection part 101 from the top, so the explanation of the word is also displayed with Chinese at the uppermost position and German at the second position from the top. Further, among each language selection part 101, the pronunciation of the language selected by the uppermost language selection part 101 (Chinese in the illustrated example) is output along with the screen switching at the time of search. Note that the display of the instruction text of the problem (see FIG. 21) and its voice reading are the same.

[0249] In this embodiment, the publisher selection unit 102 is configured to select one publisher from publishers A, B, and C. Note that the publisher may be changed for each language, or the publisher of the explanatory data, the publisher of the audio data, and the publisher of the manufacturing method manual data may be changeable.

[0250] Also, on the setting screen 100, there are provided a "language addition" button 104 for adding a language to be used, a "language deletion" button 105 for deleting a language that has already been selected, a "language display order change" button 106 for changing the display order of languages (i.e., the selection order of languages in each language selection unit 101), and a "search start" button 107 for starting a search.

[0251] When the "language display order change" button 106 is clicked and the display order of languages is changed, if the uppermost language is changed, the language in which the pronunciation of words is output is also changed accordingly.

[0252] Next, the user determines whether it is the first use (step S4 in FIG. 2). If it is determined to be the first use, the user selects the language to be used in each language selection unit 101 in FIG. 3 and selects a publisher in the publisher selection unit 102 in FIG. 3. After that (step S5 in FIG. 2), the user presses the "download" button 103 in FIG. 3 to download the explanatory data, the audio data, and the manufacturing method manual data (step S6 in FIG. 2). Also, the problem image data and the problem data (including the audio reading data) are downloaded (step S6 in FIG. 2).

[0253] Here, the download of the explanatory data is performed by the explanatory data acquisition means 25. For example, in the case of the illustrated example, the explanatory data of Chinese and German words created by Publisher A is transmitted from the data providing server 70 to the information retrieval system 10. Also, the download of the voice data is performed by the voice data acquisition means 26. For example, in the case of the illustrated example, the voice data of the pronunciation of Chinese and German words created by Publisher A is transmitted from the data providing server 70 to the information retrieval system 10. Further, the download of the manufacturing method manual data is performed by the manufacturing method manual acquisition means 32. For example, in the case of the illustrated example, the manufacturing method manual data created by Publisher A (which may include Chinese voice data) is transmitted from the data providing server 70 to the information retrieval system 10. Also, the download of the problem image data and the problem data (including the voice reading data) is performed by the problem image data / problem data acquisition means 35, and these data are transmitted from the data providing server 70 to the information retrieval system 10.

[0254] Subsequently, after the download is completed, click the "Search Start" button 107 in FIG. 3 to start the search for information related to the word (step S7 in FIG. 2).

[0255] On the other hand, in step S4 of FIG. 2, if it is determined that it is not the first use, since the download has already been completed, the operations in steps S5 and S6 of FIG. 2 are omitted, and the "Search Start" button 107 in FIG. 3 is clicked to start the search for information related to the word (step S7 in FIG. 2). If it is necessary to change the content downloaded and set at the first use, the settings can be changed by clicking each button 104 to 106 in FIG. 3.

[0256] When the "Search Start" button 107 in FIG. 3 is clicked (step S7 in FIG. 2), a search window 120 as shown in FIG. 4 is displayed on the screen of the display means 30. This search window 120 includes an image display section 121 for displaying an image of a space, an object, or its component articles, an explanation display section 122 for displaying an explanation of words indicating the space, the object, or its component articles on which the image is displayed in the image display section 121, a "Data Set" button 123 for performing a data set (setting change), a pointer position display section 124 for displaying the current position pointed by the pointer of a pointing device such as a mouse pointer in X, Y coordinates, a "Back" button 125 for returning to the previous screen, and an "End" button 126 for ending the program. These functions are also provided in the conventional information search system described in the above-mentioned Patent Document 1.

[0257] Furthermore, the search window 120 is provided with a "Warp" button 128A for the user to transition to a designated transition destination image (warp destination) registered by the user, a "Register" button 128B and a "Cancel" button 128C for a user-specified image (favorite) registered by the user judging that it has a utilization value in later processes, a "Start Image Change" button 128D for changing the start image of the search, a "Designated Transition Destination Image (Warp Destination) Registration" button 128E for the user to register a designated transition destination image (warp destination), a "Solve Problems and Learn" button 128F for solving problems and learning with the image reached by the search, and a "Manufacturing Method Manual Reference" button 128G for referring to manufacturing method manual data. These functions are functions added in the present invention.

[0258] In the image display section 121 of the search window 120 in FIG. 4, an image 130 of the "comprehensive space" is displayed. In this image 130 of the "comprehensive space", there is an image selection section 131 indicated by a picture of a city for selecting an image of the "urban space" which is a lower layer (the immediately lower layer), an image selection section 132 indicated by a picture of a house for selecting an image of the "space of a house" which is a lower layer (the immediately lower layer), and an image selection section 133 indicated by a picture of nature for selecting an image of the "natural space" which is a lower layer (the immediately lower layer). In this embodiment, the screen on which the image 130 of the "comprehensive space" is displayed is the topmost layer screen (step S8 in FIG. 2). Also, when the image ID of the start image selected by the user is stored in a designated start image storage means (not shown), instead of the image 130 of the "comprehensive space" at the topmost layer, a screen including the start image selected by the user is displayed. Note that, as shown in FIG. 4, no display is made on the explanation display section 122 for the "comprehensive space".

[0259] Then, the user determines whether to change the start image (step S9 in FIG. 2). If it is determined to change the start image, the "Change Start Image" button 128D is clicked to display the start image selection screen 300 (see FIG. 17) on the screen of the display means 60, and the start image is selected and registered (step S10 in FIG. 2).

[0260] Hereafter, the target information to be searched (information related to the word) will be described as being, for example, the spelling (Chinese characters), pronunciation, usage method, etc. of the Chinese word for an object such as "eraser". Also, it is assumed that the start image is not changed and the search starts from the image 130 of the "comprehensive space" at the topmost layer.

[0261] First, the user refers to the screen where the image 130 of the "Comprehensive Space" in FIG. 4 is displayed and determines whether the target information has been obtained (step S11 in FIG. 2). Here, if it is determined that the target information has not been obtained, the user refers to the image 130 of the "Comprehensive Space" and selects an image selection section that is likely to lead to an object called "eraser" (step S12 in FIG. 2). The reception of the selection operation of this image selection section is performed by the selection reception processing means 21. Here, it is assumed that the user clicks on the image selection section 132 shown by a picture of a house to select the image of the "space of the house".

[0262] Note that the user can also press the "Warp" button 128A to transition to a screen including the specified transition destination image (warp destination), or press the "Back" button 125 to return to the setting screen 100 in FIG. 3. Here, it is assumed that the user clicks on the image selection section 132 (step S12 in FIG. 2).

[0263] Then, as shown in FIG. 5, the image display section 121 of the search window 120 displays the image 140 of the "space of the house", which is the lower layer (the directly lower layer) of the image 130 of the "Comprehensive Space" in FIG. 4. Along with this screen switch, the pronunciation of the Chinese word corresponding to the house is output (step S13 in FIG. 2). This screen switch is performed by the screen switching processing means 22 according to the hierarchical information stored in the hierarchical information storage means 44 (see FIG. 10), and the output of the pronunciation is performed by the voice output processing means 24.

[0264] Note that in step S12 described above, if the "Warp" button 128A is pressed, the switching process to the screen including the specified transition destination image (warp destination) is performed by the screen switching processing means 22. If the "Back" button 125 is pressed, the switching process to the previous screen is performed by the screen switching processing means 22.

[0265] In FIG. 5, the image 140 of the "home space" is represented in the form of a floor plan of the house. Inside this image 140 of the "home space", there is an image selection section 141 indicated by a study pattern (e.g., a desk and a personal computer) for selecting an image of the "study space" which is a lower layer (the directly lower layer), an image selection section 142 indicated by a living room pattern (e.g., a table and a sofa) for selecting an image of the "living room space" which is a lower layer (the directly lower layer), an image selection section 143 indicated by an entrance pattern (e.g., shoes) for selecting an image of the "entrance space" which is a lower layer (the directly lower layer), an image selection section 144 indicated by a toilet pattern (e.g., a toilet bowl) for selecting an image of the "toilet space" which is a lower layer (the directly lower layer), an image selection section 145 indicated by a bathroom pattern (e.g., a bathtub) for selecting an image of the "bathroom space" which is a lower layer (the directly lower layer), an image selection section 146 indicated by a dining room pattern (e.g., a dining table and chairs) for selecting an image of the "dining room space" which is a lower layer (the directly lower layer), and an image selection section 147 indicated by a kitchen pattern (e.g., a sink) for selecting an image of the "kitchen space" which is a lower layer (the directly lower layer).

[0266] Also, in the search window 120 of FIG. 5, a "Return to TOP Screen" button 127 is additionally provided to return to the screen (the topmost layer screen) where the image 130 of the "comprehensive space" in FIG. 4 is displayed.

[0267] Furthermore, in the explanation display section 122 of the search window 120 in FIG. 5, an explanation 148 of the Chinese word corresponding to "home" is displayed. Although not shown in the figure, below the explanation 148 of the Chinese word, explanations of words in languages other than Chinese (e.g., German, etc.) corresponding to "home" are also displayed in parallel. These word explanations are performed by the explanation display processing means 23.

[0268] Then, the user refers to the screen where the image 140 of the "home space" in FIG. 5 is displayed, and again determines whether or not target information has been obtained (step S11 in FIG. 2). Here, if it is determined that the target information has been obtained, the user clicks the "End" button 126 to end the program. On the other hand, if it is determined that the target information has not been obtained, the processes of steps S12 and S13 are repeated.

[0269] That is, referring to the image 140 of the "home space" in FIG. 5, the user selects an image selection section that is likely to lead to an object called "eraser" (step S12 in FIG. 2). The reception of the selection operation of this image selection section is performed by the selection reception processing means 21. Here, it is assumed that the user clicks on the image selection section 141 shown by the picture of the study, in order to select the image of the "study space".

[0270] Note that on this screen of FIG. 5, the user can also press the "Warp" button 128A to transition to a screen including the designated transition destination image (warp destination), or press the "Back" button 125 to return to the image 130 of the "comprehensive space" in FIG. 4. However, here, it is assumed that the user clicks on the image selection section 141 (step S12 in FIG. 2).

[0271] Then, as shown in FIG. 6, in the image display section 121 of the search window 120, an image 150 of the "study space", which is the lower layer (the directly lower layer) of the image 140 of the "home space" in FIG. 5, is displayed. Along with this screen switch, the pronunciation of the Chinese word corresponding to the study is output (step S13 in FIG. 2). This screen switch is performed by the screen switching processing means 22 according to the hierarchical information stored in the hierarchical information storage means 44 (see FIG. 10), and the output of the pronunciation is performed by the voice output processing means 24.

[0272] In the image 150 of the "study space" in FIG. 6, there are an image selection section 151 indicated by a desk pattern for selecting an image of the "space on the desk" which is a lower layer (the directly lower layer), an image selection section 152 indicated by a chair pattern for selecting an image of an object called "chair" which is a lower layer (the directly lower layer), an image selection section 153 indicated by a personal computer pattern for selecting an image of an object called "personal computer" which is a lower layer (the directly lower layer), and an image selection section 154 indicated by a bookshelf pattern for selecting an image of the "space above the bookshelf" which is a lower layer (the directly lower layer).

[0273] Also, in the explanation display section 122 of the search window 120 in FIG. 6, an explanation 155 of a Chinese word corresponding to a study is displayed. Although not shown in the figure, below the explanation 155 of the Chinese word, explanations of words in languages other than Chinese (for example, German, etc.) corresponding to a study are also displayed in parallel. These word explanations are performed by the explanation display processing means 23.

[0274] Then, the user refers to the screen on which the image 150 of the "study space" in FIG. 6 is displayed, and again determines whether the target information has been obtained (step S11 in FIG. 2). Here, if it is determined that the target information has been obtained, the "End" button 126 is clicked to end the program. On the other hand, if it is determined that the target information has not been obtained, the processes of steps S12 and S13 are repeated.

[0275] That is, referring to the image 150 of the "study space" in FIG. 6, an image selection section that is likely to lead to an image of an object called "eraser" is selected (step S12 in FIG. 2). The reception of the selection operation of this image selection section is performed by the selection reception processing means 21. Here, it is assumed that the image selection section 151 indicated by the desk pattern for selecting an image of the "space on the desk" is clicked.

[0276] Note that the user can also transition to a screen including a specified transition destination image (warp destination) by pressing the "Warp" button 128A on this screen of FIG. 6, and can also return to the image 140 of the "Home Space" in FIG. 5 by pressing the "Back" button 125. Here, it is assumed that the image selection unit 151 is clicked (step S12 in FIG. 2).

[0277] Then, as shown in FIG. 7, in the image display unit 121 of the search window 120, an image 160 of the "Space on the Desk", which is the lower layer (the directly lower layer) of the image 150 of the "Study Space" in FIG. 6, is displayed. Along with this screen switch, the pronunciation of the Chinese word corresponding to the desk is output (step S13 in FIG. 2). This screen switch is performed by the screen switching processing means 22 according to the hierarchical information stored in the hierarchical information storage means 44 (see FIG. 10), and the output of the pronunciation is performed by the voice output processing means 24.

[0278] In the image 160 of the "Space on the Desk" in FIG. 7, there are an image selection unit 161 indicated by a picture of a pencil for selecting an image of an object "pencil" which is the lower layer (the directly lower layer), an image selection unit 162 indicated by a picture of an eraser for selecting an image of an object "eraser" which is the lower layer (the directly lower layer), an image selection unit 163 indicated by a picture of a phone for selecting an image of an object "phone" which is the lower layer (the directly lower layer), an image selection unit 164 indicated by a picture of a calculator for selecting an image of an object "calculator" which is the lower layer (the directly lower layer), an image selection unit 165 indicated by a picture of a dictionary for selecting an image of an object "dictionary" which is the lower layer (the directly lower layer), an image selection unit 166 indicated by a picture of a notebook for selecting an image of an object "notebook" which is the lower layer (the directly lower layer), an image selection unit 167 indicated by a picture of a photo frame for selecting an image of an object "photo frame" which is the lower layer (the directly lower layer), and an image selection unit 168 indicated by a picture of a reference book for selecting an image of an object "reference book" which is the lower layer (the directly lower layer).

[0279] In addition, in the explanation display section 122 of the search window 120 in Fig. 7, an explanation 169 of the Chinese word corresponding to "desk" is displayed. Although not shown in the figure, below the explanation 169 of the Chinese word, explanations of words in languages other than Chinese corresponding to "desk" (for example, German, etc.) are also displayed in parallel. These word explanations are performed by the explanation display processing means 23.

[0280] Then, the user refers to the screen on which the image 160 of "the space above the desk" in Fig. 7 is displayed, and again determines whether the target information has been obtained (step S11 in Fig. 2). Here, if it is determined that the target information has been obtained, click the "End" button 126 to end the program. On the other hand, if it is determined that the target information has not been obtained, the processes of steps S12 and S13 are repeated.

[0281] That is, referring to the image 160 of "the space above the desk" in Fig. 7, click and select the image selection section 162 indicated by the picture of an eraser to select the image of the object "eraser" (step S12 in Fig. 2). The reception of the selection operation of this image selection section is performed by the selection reception processing means 21.

[0282] Note that on this screen of Fig. 7, the user can also press the "Warp" button 128A to transition to a screen including the designated transition destination image (warp destination), and can also press the "Back" button 125 to return to the image 150 of "the study space" in Fig. 6. Here, it is assumed that the user clicks the image selection section 162 (step S12 in Fig. 2).

[0283] Then, as shown in Fig. 8, in the image display section 121 of the search window 120, an image 180 of the object "eraser", which is the lower layer (immediate lower layer) of the image 160 of "the space above the desk" in Fig. 7, is displayed. Along with this screen switch, the pronunciation of the Chinese word corresponding to the eraser is output (step S13 in Fig. 2). This screen switch is performed by the screen switch processing means 22 according to the hierarchical information stored in the hierarchical information storage means 44 (see Fig. 10), and the output of the pronunciation is performed by the voice output processing means 24.

[0284] In the image 180 of the object "eraser", there is no image selection part for selecting an image in the lower hierarchy (the immediately lower hierarchy). Therefore, the screen on which the image 180 of the object "eraser" is displayed is the screen at the lowest hierarchy in this search path.

[0285] Also, in the explanation display part 122 of the search window 120 in FIG. 8, an explanation 181 of the Chinese word corresponding to the eraser is displayed. Although not shown in the figure, below the explanation 181 of the Chinese word, explanations of words in languages other than Chinese (for example, German, etc.) corresponding to the eraser are also displayed in parallel. These word explanations are performed by the explanation display processing means 23.

[0286] Then, the user refers to the screen on which the image 180 of the object "eraser" in FIG. 8 is displayed, and again determines whether the target information has been obtained (step S11 in FIG. 2). Here, if it is determined that the target information has been obtained, the "End" button 126 is clicked to end the program. On the other hand, if it is determined that the target information has not been obtained, since the currently referenced screen is the screen at the lowest hierarchy of this search path, in order to perform a search in another search path, after clicking the "Return to TOP Screen" button 127, the processing of steps S12 and S13 can be repeated. Here, since the explanation of the Chinese word of the object "eraser" which is the target information has been obtained, the "End" button 126 is clicked to end the program.

[0287] Note that on this screen of FIG. 8, it is also possible to press the "Warp" button 128A to transition to a screen including the designated transition destination image (warp destination), and it is also possible to press the "Back" button 125 to return to the image 160 of "the space on the desk" in FIG. 7. Therefore, although the screen on which the image 180 of the object "eraser" in FIG. 8 is displayed is the screen at the lowest hierarchy, if the target information has not been reached there or if it is desired to search for related information, it is also possible to press the "Warp" button 128A to transition to a screen including the designated transition destination image (warp destination).

[0288] FIG. 9 shows a setting change screen 200 (window) for adding, deleting, overwriting, etc. of words to be searched. This setting change screen 200 is displayed when the "Dataset" button 123 (see FIGS. 4 to 8) of the search window 120 is clicked. For example, when installing a program (including hierarchical information and image coordinate information) recorded on a recording medium such as a CD-ROM downloaded from the data providing server 70 or purchased at a retail store, a screen used for making settings according to each user's preference, such as adding data that is not set at the time of installation. The reception process of the setting information by the user on this setting change screen 200 is performed by the setting means 27, and the received setting information is stored in the hierarchical information storage means 44 (see FIG. 10) and the image coordinate information storage means 45 (see FIG. 11).

[0289] The setting change screen 200 is provided with a name input section 201 for inputting the name of the word, an image file name input section 202 for inputting the image file name of the word, an audio file name input section 203 for inputting the audio file name of the word, an upper left X coordinate input section 204 for inputting the X coordinate of the upper left position of the image selection section (provided in the image of the immediately upper layer and for transitioning from the image of the immediately upper layer to one's own image (the image for which the description of the word is made)), an upper left Y coordinate input section 205 for inputting the Y coordinate of the upper left position of the image selection section, a lower right X coordinate input section 206 for inputting the X coordinate of the lower right position of the image selection section, and a lower right Y coordinate input section 207 for inputting the Y coordinate of the lower right position of the image selection section.

[0290] In addition, the setting change screen 200 is provided with a current layer position display section 208 for displaying the current layer position (the position within the layer of the currently displayed image, that is, the image displayed when the "Dataset" button 123 is pressed), a layer display section 209 for displaying the layers of each image in a tree form using the layer information stored in the layer information storage means 44 (see FIG. 10), and an explanation data input section 210 for inputting the explanation of the word in text.

[0291] Furthermore, on the setting change screen 200, there are provided a "Clear Text" button 211 for clearing the text data entered in the name input section 201, a "Reference" button 212 for input assistance to search for the storage location of the image file and call out the image file name when entering the image file name in the image file name input section 202, a "Reference" button 213 for input assistance to search for the storage location of the audio file and call out the audio file name when entering the audio file name in the audio file name input section 203, a "Add Word" button 214 for adding words, a "Delete Word" button 215 for deleting words, a "Edit Word" button 216 for overwriting and editing words, and an "End" button 217 for ending the program.

[0292] When adding a word, first, the user selects an image of the upper hierarchy (the immediately upper hierarchy) to which the space, object, or its component article that the word to be added refers belongs. For example, when adding and registering a word that refers to an object such as a "globe", an image 150 of a "study space" that includes the pattern of the globe is used as the image of the upper hierarchy (the immediately upper hierarchy) (see Fig. 6). Then, on the screen where the image 150 of this "study space" is displayed, move the pointer of a pointing device such as a mouse pointer, and stop the pointer of the pointing device such as a mouse pointer at the upper left position and the lower right position of the area where the pattern of the globe is drawn (the area that becomes the image selection section for selecting the image of the object "globe" which is the lower hierarchy (the immediately lower hierarchy)). At this time, the user refers to the pointer position display section 124 of the search window 120 and checks the numerical values of the X and Y coordinates of each stop position.

[0293] Subsequently, the user clicks the "Dataset" button 123 of the search window 120 to display the setting change screen 200 of Fig. 9 on the screen of the display means 60. Then, on this setting change screen 200, select the word hierarchy (level) in the tree view displayed in the hierarchy display section 209 of Fig. 9. Furthermore, click the "Clear Text" button 211 to leave the name input section 201 blank, and then enter the name of the word to be newly registered (for example, "globe").

[0294] Also, enter the names of the image file (a file containing image data of an object called "globe") and the audio file (a file containing audio data of the pronunciation of words in each language corresponding to the "globe") into the image file name input section 202 and the audio file name input section 203. Further, enter the coordinates of the upper left position and the lower right position of the area where the pattern of the globe is drawn, which was previously confirmed on the screen where the image 150 of the "study space" in FIG. 6 is displayed, into the respective coordinate input sections 204 to 207. Further, after entering the description of the word in text in the description data input section 210, click the "Add Word" button 214. As a result, an image selection section 156 for selecting an image of an object called "globe", which is its lower layer (the layer immediately below), is newly set in the image 150 of the "study space" in FIG. 6, and the word referring to the object called "globe" is additionally registered.

[0295] When deleting a word, the user selects the word to be deleted in the tree view displayed in the hierarchy display section 209 on the setting change screen 200 in FIG. 9. Then, after confirming the content (the content to be deleted) displayed in each input section 201 to 207, 210, click the "Delete Word" button 215. Note that it is preferable to perform word deletion from the lowest layer (level).

[0296] Overwriting and editing of a word is substantially the same as the operation procedure for adding a word. On the setting change screen 200 in FIG. 9, select the word to be overwritten (the word that will be deleted after editing) in the tree view displayed in the hierarchy display section 209, overwrite and enter the necessary content in each input section 201 to 207, 210, and then click the "Edit Word" button 216.

[0297] <Advantages of this Embodiment>

[0298] According to such an embodiment of the present invention, the following effects can be achieved. That is, in the information retrieval system 10, not only is the search for information related to a word advanced from a higher level to a lower level according to the hierarchical information stored in the hierarchical information storage means 44 (see FIG. 10), but also a transition from an arbitrary reached image reached by the search to a designated transition destination image (warp destination) registered by the user in association with this reached image can be performed. For this reason, the user can construct, on the system, associations between spaces, associations between a space and an object, or associations between an object and its constituent articles according to his or her own associations, and perform a search according to the associations. The registration of these associations by the user can be considered in parallel with the process of synapse formation in the user's brain. Therefore, efficient information retrieval according to the experience and preferences of individual users can be realized.

[0299] In addition, the information retrieval system 10 can also inherit the effects obtained by the conventional information retrieval system described in Patent Document 1 mentioned above. That is, in order to obtain target information (information related to a word), the user can perform a selection operation on the image selection part of the immediately lower level included in the image on the screen of the display means 60, and thus pass through the image of the space at least once, and sequentially switch to a screen for displaying the images of the lower levels. Then, such a selection operation by the user and the screen switching corresponding to the operation are repeated until the target information is obtained. At this time, the selection operation by the user is received by the selection reception processing means 21, and the screen switching is performed by the screen switching processing means 22 according to the hierarchical information stored in the hierarchical information storage means 44 (see FIG. 10).

[0300] Therefore, by performing a search that passes through the image of the space at least once, the user can obtain the target information through visual effects using the space as a clue. Accordingly, by simply performing a selection operation while referring to the image of the space, the user can reach the image to be the target of information acquisition. Thus, compared to the case of conventional searches using the number of strokes, pronunciation, radicals, triangular codes, square codes, etc., for example, the labor of specifying the number of strokes and the knowledge for counting the number of strokes can be eliminated, and the target information can be easily obtained. This effect is the same as the effect obtained by the conventional information search system described in Patent Document 1 mentioned above.

[0301] Furthermore, the designated transition destination image registration means 30 displays the image of the layer immediately above the reached image reached by the search as a candidate for the designated transition destination image (warp destination) (see the designated transition destination image selection section 410 of the designated transition destination image (warp destination) registration screen 400 in FIG. 20). Therefore, when there are a plurality of images of the immediately above layer for a certain reached image, not only can the user return to the image immediately before in the search path that has passed, but also can transition to other images of the immediately above layer (see FIG. 18).

[0302] Also, the designated transition destination image registration means 30 displays the user-specified image (favorite) as a candidate for the designated transition destination image (warp destination) (see the designated transition destination image selection section 420 of the designated transition destination image (warp destination) registration screen 400 in FIG. 20). Therefore, if the user registers an image that the user likes, an image that is often used, an image that the user feels will have high utility value in the future, etc. as the user-specified image (favorite) during the search, that user-specified image can be registered as the designated transition destination image.

[0303] Furthermore, the designated transition destination image registration means 30 displays the image obtained by keyword search or the image of the layer immediately above it as a candidate for the designated transition destination image (warp destination) (see the designated transition destination image selection section 432 of the designated transition destination image (warp destination) registration screen 400 in FIG. 20). Therefore, by relying on a word, the desired image can be found and registered as the designated transition destination image (warp destination).

[0304] In addition, since the designated transition destination image registration means 30 accepts the selection of the designated transition destination image (warp destination) by the user within the hierarchical display in the form of a tree (refer to the designated transition destination image selection section 440 of the designated transition destination image (warp destination) registration screen 400 in FIG. 20), the user can select the designated transition destination image (warp destination) to be registered while referring to the hierarchical information. Therefore, while confirming the association between the images predetermined as the hierarchical information (i.e., the standard association), the user can register the association between the personal images that match their own associations.

[0305] Furthermore, since the designated transition destination image registration means 30 has a function to invalidate the registration of the designated transition destination image (warp destination), it is possible to invalidate the associations between spaces that have not been used for a predetermined period, the associations between a space and an object, and the associations between an object and its constituent articles, even though the user has registered them. This invalidation of registration can be considered in combination with the fact that once-formed synapses are severed due to long-term non-use.

[0306] And since the designated transition destination image registration means 30 accepts the registration of the transition from the arrival image to the designated transition destination image (warp destination) and simultaneously accepts the registration of the reverse transition (transition from the designated transition destination image to the arrival image) (refer to the two-way transition selection section 450 of the designated transition destination image (warp destination) registration screen 400 in FIG. 20), when the user feels that the reverse association is also strong, that is, when the user feels that the two-way association is strong, it is possible to easily register the reverse transition without having to perform a separate search and the operation of reaching the image that is currently about to be registered as the designated transition destination image (warp destination).

[0307] In addition, since the information search system 10 is provided with the start image selection means 29, the user does not necessarily have to start the search from the image at the topmost hierarchy, and can change the start image of the search and start the search according to the progress of the user's learning, the growth of a child user, etc. Therefore, the convenience of the system for the user can be improved.

[0308] Furthermore, since the start image selection means 29 can execute (1) the start image selection process using keyword search, (2) the start image selection process using user-specified images, and (3) the start image selection process using the cumulative number of passes (see FIG. 17), the user can easily change the start image.

[0309] Also, since the information retrieval system 10 includes a problem processing means 33, a problem image data storage means 50 (see FIG. 15), and a problem storage means 51 (see FIG. 16), it can pose problems regarding the images displayed on the screen of the display means 60 (which is the main image for which the word is to be explained, not the symbol part of the image selection section for transitioning to the images in the immediately lower layer included in that image). Therefore, the user can learn a language in the space or object of that image, and thus can have the feeling of learning a language in the space or object (i.e., the image reached by the search) they visited. For this reason, a situation similar to language acquisition in the real world can be created, so the user can perform effective learning.

[0310] Furthermore, since the information retrieval system 10 includes an evaluation means 34, it can recommend a field of employment destination using the judgment result of the correctness of the answer to the problem (see FIG. 22). Therefore, since it can recommend a field of employment destination using the result of learning a language in the space or object (i.e., the image reached by the search) the user visited, it can make a recommendation in a situation close to a recommendation based on real-world experience.

[0311] Also, since the information retrieval system 10 includes an evaluation means 34, it can determine whether the user understands the image schema. For this reason, it can be determined whether the user is learning the image schema so that it can be applied to other scenarios or other fields, and if there is a possibility that the user has not acquired the image schema, the user can be informed of this and further learning can be promoted.

[0312] And since the problem processing means 33 can display problem instructions in multiple languages, the user can simultaneously learn multiple languages in the space or object of the image (which is the main image to be described by words, not the symbol part of the image selection section for transitioning to the immediately lower-level image included in that image) displayed on the screen of the display means 60. For this reason, the user can have the feeling of learning each language in the space or object (i.e., the image reached by the search) they visited. Therefore, a situation similar to the acquisition of each language in the real world can be created, so that the user can perform effective learning.

[0313] Furthermore, since the information retrieval system 10 includes the manufacturing method manual display processing means 31, it can display the manufacturing method manual for the object indicated by the image displayed on the screen of the display means 60. Therefore, while learning a language in the space or object (i.e., the image reached by the search) the user visited, the user can also learn the manufacturing method of the object related to that image, so that the fixation of the memory of the word indicated by that image can be easily realized.

[0314] Also, when the object related to the image displayed on the screen of the display means 60 is food, the manufacturing method manual display processing means 31 can display the recipe of that food. Therefore, while learning a language in the image of the object or its component articles (i.e., the image reached by the search) the user visited, the user can also learn the manufacturing method of the food related to that image, so that the fixation of the memory of the word indicated by that image can be easily realized.

[0315] <Modification form>

[0316] Note that the present invention is not limited to the above-described embodiment, and modifications and the like within the scope capable of achieving the object of the present invention are included in the present invention.

[0317] For example, in the above-described embodiment, the information retrieval system 10 was described as having a stand-alone system configuration as shown in FIG. 1. However, the information retrieval system of the present invention may also be a server-client system. In this case, the main body 20 of the above-described embodiment may be used as a server, and the display means 60 and the input means 61 may be provided on a client terminal.

[0318] Also, in the search window 120 (see FIGS. 4 to 8) of the above-described embodiment, new buttons 128A to 128G were provided. However, these buttons 128A to 128G are not functions for children, and the number of buttons in the search window 120 is large. Therefore, at least some of these buttons may be grouped into an "option" button, and at least some of these buttons may be provided on an option screen (window) displayed by pressing the "option" button.

[0319] (Version update)

[0320] Furthermore, in the data providing server 70 of the above-described embodiment, it was described that one version of a program (including hierarchical information and image coordinate information), image data, explanatory data, voice data, manufacturing method manual data (including recipe data), problem image data, and problem data (including voice reading data) was stored respectively. However, when there are version updates for these, the updated program (including hierarchical information and image coordinate information), image data, explanatory data, voice data, manufacturing method manual data (including recipe data), problem image data, and problem data (including voice reading data) may be stored in the data providing server 70 in association with a version number, so that the user can obtain the latest version or an arbitrarily selected version of the program and each data.

[0321] In addition, the data providing server 70 may store change information indicating what has changed due to version updates, and transmit this change information to the main body 20 via the network 1 in response to a display request from the main body 20. This display request from the main body 20 can be made, for example, when the program of the information search system 10 is launched on the main body 20 or when the program is terminated. The change information associated with this version update includes information on added spaces, deleted spaces, added objects, deleted objects, added constituent articles, and deleted constituent articles. For added spaces, objects, and constituent articles, corresponding image data, explanatory data, audio data, manufacturing manual data (including recipe data), problem image data, and problem data (including voice reading data) will also be added. For example, when new objects or constituent articles appear in the world, the manufacturing manual data (including recipe data) related to the new objects or constituent articles will increase, so the change information includes that fact. Therefore, for example, as time passes by 5 years or 10 years, new objects and constituent articles will appear, and conversely, objects and constituent articles that are no longer used will also occur. By providing the change information to the user, the user can determine whether a version update is necessary for their own learning. For this reason, the user can perform learning that keeps up with the changes in the world.

[0322] (Concept of time axis)

[0323] The information retrieval system 10 of the above-described embodiment was configured to perform information retrieval using hierarchical information that defines the space, objects, constituent articles, and their subordinate relationships at a single point in time, which is the present. And even when configured to be able to perform version updates as described above, depending on whether to select the latest version or an old version, a system that uses different spaces, different objects, different constituent articles, or a system that uses different hierarchical information can be realized. However, the constructed information retrieval system uses the space, objects, and constituent articles at a single point in time (in the case of the latest version, the most recent single point in time; in the case of an old version, a past single point in time), or uses the hierarchical information at a single point in time, without change.

[0324] On the other hand, it may be an information retrieval system that accepts changing hierarchical information that defines the space, objects, constituent articles, or their subordinate relationships displayed on the screen over the time axis. In the following, an information retrieval system having this concept of the time axis will be described using the same reference numerals as those in the above-described embodiments. For example, on the screen (the top-level screen) where the image 130 of the "comprehensive space" in FIG. 4 is displayed, or on the setting screen 100 in FIG. 3 or a guidance screen (a screen that displays how to use this system, version information, provider information of this system, expiration information if there is an expiration date, etc.) displayed before that, a "time period selection section" for selecting an arbitrary point in time on the time axis may be provided so that the user can select the time period (the time period to be learned, the time period for which problems are to be solved) for which information retrieval is desired. Here, the reason for calling an arbitrary point in time selected here a "time period" is that the hierarchical information that defines the space, objects, constituent articles, or their subordinate relationships displayed on the screen does not change finely and continuously over time, but changes after a certain period of time. That is, the presence or absence status of each of the space, objects, and constituent articles does not change significantly over a few days or weeks. For example, it takes about 10 years for the situation to change for the first time. For example, around 1990, the only popular telephone was a landline telephone, and mobile phones were not yet popular, but around 2000, mobile phones became popular, etc., and these are classified into "time periods" such as the 1990s and 2000s.

[0325] Also, the original eras studied in history (Japanese history or world history) correspond to the "eras" selected by the above-mentioned "Era Selection Section". Usually, in terms of system settings, the longer the time period that divides the "eras" in the "Era Selection Section" is set shorter as it approaches modern times, and longer as it goes back to ancient times. The setting of "eras" (the way of dividing the time axis, for example, the length of the dividing time period, the number of divisions) is arbitrary. For example, it can be set as divisions such as the Stone Age, the Jomon and Yayoi periods, the Edo period, the Meiji period, pre-war, post-war, the 1970s, the 1990s, the 2000s, the 2010s, and the 2020s. Also, future eras can be set. In that case, as spaces, objects, and constituent items, imaginary things will be mixed in. Or, the hierarchical information that defines their subordinate relationships will also be partially imaginary. However, if they are things whose names and concepts are fixed in movies, comics, novels, etc. to a certain extent in modern society, data related to those imaginary things, that is, image data, explanatory data, audio data, manufacturing method manual data (including recipe data), problem-related image data, problem data (including voice reading data) can be prepared. For example, a time machine that appears in a movie may exist as an object, or a devastated urban space after a nuclear war or outer space may exist as a space for survival and living.

[0326] The "Era Selection Section" displayed on the screen of the display means 60 is, for example, on a time axis where time progresses horizontally from the left end to the right end (however, time may progress in the reverse direction), or on a time axis where time progresses vertically from the upper end to the lower end (however, time may progress in the reverse direction). It may be a selection section of a slider input method for selecting an era by moving a slider. However, even if the slider itself can continuously change its position, the "era" selected by the position movement of the slider is not continuously changing as described above, but is an era delimited by an appropriate time length. Also, the "Era Selection Section" may be a selection section of a selection button method in which selection buttons are provided for each era. For example, a selection button for the Stone Age, a selection button for the Edo period, a selection button for the 2020s, etc. may be provided. The arrangement position of this "Era Selection Section" on the screen is arbitrary. For example, when provided within the search window 120 which is a search screen, it can be provided in the upper part of the image display section 121, the left or right part of the image display section 121, the part between the image display section 121 and the explanation display section 122, the lower part of the explanation display section 122, etc.

[0327] Also, when the era changes, the hierarchical information may or may not change. For example, in the Stone Age, there is no urban space in the comprehensive space (see Fig. 4), and the space of a house is not the space inside a building but the space inside a cave. Therefore, for example, when a selection operation is performed on the "Era Selection Section" displayed on the screen (the screen of the highest hierarchical level) where the image 130 of the "Comprehensive Space" in Fig. 4 is displayed and the Stone Age is selected, the image selection section 133 indicated by a natural pattern for selecting the image of the "Natural Space", which is the lower hierarchical level (the directly lower level), remains to move to a space such as a hunting scene, but the image selection section 131 indicated by an urban pattern for selecting the image of the "Urban Space", which is the lower hierarchical level (the directly lower level), disappears, and further, the image selection section 132 indicated by a house pattern for selecting the image of the "Space of a House", which is the lower hierarchical level (the directly lower level), changes to a pattern of a "cave" for selecting the image of the "cave space".

[0328] Also, when the Edo period is selected, the image selection section 133 indicated by a natural pattern for selecting an image of the "natural space" which is the lower layer (the immediately lower layer) remains for moving to a space such as a battlefield, etc., but the image selection section 131 indicated by an urban pattern for selecting an image of the "urban space" which is the lower layer (the immediately lower layer) changes to the pattern of "castle town" for selecting an image of the "castle town space", and the image selection section 132 indicated by a house pattern for selecting an image of the "space of a house" which is the lower layer (the immediately lower layer) changes to the pattern of "castle" for selecting an image of the "space inside the castle".

[0329] Note that the "era selection section" may be provided on a screen of a layer lower than the screen (the screen of the topmost layer) on which the image 130 of the "comprehensive space" in FIG. 4 is displayed (see FIGS. 5 to 8). However, since there may not necessarily be corresponding spaces, corresponding objects, and corresponding constituent articles in other eras selected by the "era selection section" displayed on the screen of the lower layer, in that case, it may be configured to move to the screen (the screen of the topmost layer) on which the image of the "comprehensive space" in the selected other era is displayed. Also, the system may be pre-set so that the space, objects, and constituent articles in the era before movement and the space, objects, and constituent articles in the era after movement are in a similar situation. Therefore, the correspondence relationship between each of the space, objects, and constituent articles in a certain era and any of the space, objects, and constituent articles in all other eras is stored in advance in the "era-to-era hierarchical information correspondence relationship storage means". For example, the image ID of the living room space in the 2020s is associated with the image ID of the tea room space in the Edo period and stored in the "era-to-era hierarchical information correspondence relationship storage means", etc.

[0330] When it comes to an information retrieval system that incorporates the concept of a time axis in this way and enables information retrieval across multiple eras, the program (the algorithm for displaying the search window 120, which is the search screen) is the same as the information retrieval system 10 of the above-described embodiment and may be common to all eras. However, hierarchical information, image coordinate information, image data, explanatory data, voice data, manufacturing method manual data (including recipe data), problem-related image data, and problem data (including voice read-out data) should be prepared separately for each era and stored in the storage means 40. That is, the storage means 40 (all storage means 41 to 51 included in the storage means 40) are provided for each era. When allowing movement not only starting from the movement between the topmost hierarchies but also between intermediate hierarchies during the movement between eras, the above-described "inter-era hierarchical information correspondence relationship storage means" should be provided.

[0331] In addition, a "time switching means" is added to the processing means 20A. By this "time switching means", the selection of "time" by the user in the "time selection section" displayed on the screen of the display means 60 is received, and a process of switching the time is executed so that various screen display processes are performed using the storage means 40 of the selected time. That is, this "time switching means" stores the identification information of the selected "time" in the "selected time storage means" (which may be the main memory). Then, the processing means 20A (selection reception processing means 21, screen switching processing means 22, explanation display processing means 23, voice output processing means 24, user-specified image registration means 28, start image selection means 29, designated transition destination image registration means 30, manufacturing method manual display processing means 31, problem processing means 33, evaluation means 34) refers to the identification information of the time stored in this "selected time storage means" and executes various screen display processes using the storage means 40 of the selected time. Note that for the programs (including hierarchical information and image coordinate information) and image data that are essential for the operation of this system, the user needs to first download and acquire them from the data providing server 70 (refer to step S1 in FIG. 2). At this time, since the programs are common to all times, the common programs are acquired, and in addition, the hierarchical information, image coordinate information, and image data of all times are acquired. Also, the explanation data acquisition means 25, voice data acquisition means 26, manufacturing method manual acquisition means 32, and problem image data - problem data acquisition means 35 also acquire the data of all times.

[0332] When an information search system that incorporates the concept of the time axis in this way and can perform information searches in multiple times is used, the user can perform various information searches across times, that is, across multiple times, and can learn historical terms, manufacturing methods of objects that existed in history, etc.

Industrial Applicability

[0333] As described above, the information retrieval system and program of the present invention are suitable for use, for example, when retrieving the spelling or semantic content description of a word indicated by an image, the pronunciation of a word, the manufacturing method of an object corresponding to a word, or when solving a problem including an image schema learning problem in an image of a space or an object and learning a language other than the mother tongue such as Chinese, French, English, etc.

Explanation of Signs

[0334] 10 Information retrieval system 21 Selection reception processing means 22 Screen switching processing means 23 Explanation display processing means 28 User-specified image registration means 29 Start image selection means 30 Designated transition destination image registration means 31 Manufacturing method manual display processing means 33 Problem processing means 34 Evaluation means 41 Image data storage means 42 Explanation data storage means 44 Hierarchical information storage means 45 Image coordinate information storage means 46 Manufacturing method manual storage means 47 Cumulative passage count storage means 48 User-specified image storage means 49 Designated transition destination image storage means 50 Problem image data storage means 51 Problem storage means 60 Display means

Claims

1. An information retrieval system configured by a computer to retrieve information related to words on the screen of a display means, image data storage means for storing a plurality of image data for respectively displaying images of a plurality of spaces and a plurality of image data for respectively displaying images of a plurality of objects belonging to at least one of these spaces or constituent articles thereof; explanation data storage means for storing explanation data of words indicating the space and explanation data of words indicating the object or its constituent article for one or more languages selected by the user on a setting screen displayed on the screen of the display means; hierarchical information storage means for storing hierarchical information for hierarchically arranging the respective image data in a state of passing through the image of the space at least once from the highest layer to the lowest layer according to any one of the subordinate relationships between the spaces, the subordinate relationship between the space and the object, or the subordinate relationship between the object and its constituent article; selection reception processing means for receiving a selection operation of the image selection unit by the user on the screen of the display means on which an image of a space including an image selection unit for selecting an image of a space or object belonging to the immediately lower layer is displayed, or an image of an object including an image selection unit for selecting an image of a constituent article is displayed; screen switching processing means for switching the screen displayed on the display means to a screen for displaying an image of the space or the object or its constituent article corresponding to the image selection unit selected by the user's selection operation, using the image data stored in the image data storage means according to the hierarchical information stored in the hierarchical information storage means; explanation display processing means for displaying an explanation of a word indicating the space or the object or its constituent article corresponding to the image selection unit selected by the user's selection operation on the same screen as the switched screen on which the image of the space or the object or its constituent article corresponding to the image selection unit selected by the user's selection operation is displayed, using the explanation data stored in the explanation data storage means in accordance with the screen switching processing by this screen switching processing means. The processing by the selection reception processing means, the processing by the screen switching processing means, and the processing by the explanation display processing means are repeated by the selection operation of the image selection unit by the user, and according to the hierarchical information stored in the hierarchical information storage means, from the upper layer to the lower layer, the image of the space, the object, or its component articles, and the explanation of the word indicating the space, the object, or its component articles are repeatedly displayed on the screen of the display means in a state of being displayed, Furthermore, in order to realize the transition to a designated transition destination image that is not limited to the transition to the image of the layer immediately below the reached image by the selection operation of the image selection unit included in the reached image, and the transition to the image one before the reached image in the search path reaching the reached image, while transitioning from the upper layer to the lower layer, the registration of the designated transition destination image by the user is received, and the identification information of the designated transition destination image is stored in the designated transition destination image storage means in association with the identification information of the reached image, and a designated transition destination image registration means is provided, The selection reception processing means is also configured to receive the selection operation of the user for the transition to the designated transition destination image, The screen switching processing means also executes a process of switching the screen for displaying the reached image to the screen for displaying the designated transition destination image according to the selection operation of the user for the transition to the designated transition destination image received by the selection reception processing means, using the designated transition destination image in which the identification information is stored in the designated transition destination image storage means in association with the identification information of the reached image. An information retrieval system characterized by the above.

2. The designated transition destination image registration means is configured to display, on the screen of the display means, as a candidate for the designated transition destination image, the image of the layer immediately above the reached image stored in the hierarchical information storage means to the user, and receive the selection from the displayed candidate images. The information retrieval system according to claim 1, characterized by the above.

3. A user-specified image registration means is provided that receives an input specification by the user regarding setting an image that is the object of word explanation as a user-specified image on the screen displayed on the display means, and stores the identification information of the user-specified image in the user-specified image storage means. The designated transition destination image registration means configured to display, on the screen of the display means, the user-specified image in which the identification information is stored in the user-specified image storage means to the user as a candidate for the specified transition destination image, and to accept a selection from the displayed candidate images The information retrieval system according to claim 1, characterized in that.

4. The specified transition destination image registration means attaches and accepts the input of a search keyword for the specified transition destination image by the user, and uses the accepted search keyword to search for a word that matches or is similar to the search keyword among the words indicating the images included in the hierarchical information stored in the hierarchical information storage means, and displays, on the screen of the display means, the image corresponding to the searched word and the image of the layer immediately above this image as candidates for the specified transition destination image, and is configured to accept a selection from the displayed candidate images The information retrieval system according to claim 1, characterized in that.

5. The specified transition destination image registration means is configured to display, in a tree form, the layers of each image for the user using the hierarchical information stored in the hierarchical information storage means, and to accept the selection of the specified transition destination image by the user within this tree-form layer display The information retrieval system according to claim 1, characterized in that.

6. In the specified transition destination image storage means the last use date and time using the transition from the arrival image to the specified transition destination image, or the registration date and time of the specified transition destination image when the use of the transition from the arrival image to the specified transition destination image has never occurred, is stored in association with the identification information of the arrival image, together with the identification information of the registered specified transition destination image The specified transition destination image registration means is configured to determine whether or not a predetermined time has elapsed from the last use date and time or the registration date and time when the use has never occurred, and if it has elapsed, to invalidate the registration of the specified transition destination image The information retrieval system according to claim 1, characterized in that.

7. The specified transition destination image registration means When registering the specified transition destination image to enable transition from the arrival image to the specified transition destination image, accept the user's selection to also register the transition from the specified transition destination image in the reverse direction to the arrival image. When the selection of this reverse transition is accepted, assume that the image currently being registered as the specified transition destination image becomes a temporary arrival image in another search, and the image currently serving as the arrival image becomes the temporary specified transition destination image in the other search. The identification information of the temporary specified transition destination image in this assumed state is stored in the specified transition destination image storage means in association with the identification information of the temporary arrival image. The information retrieval system according to claim 1, characterized in that.

8. To set a start image for a search in place of the top-level image, (1) Prompt the user to input a search keyword for the start image, and using the received search keyword, search for words that match or are similar to the search keyword among the words indicating images included in the hierarchical information stored in the hierarchical information storage means. Display the image corresponding to the retrieved word and the image of the level immediately above this image on the screen of the display means as candidates for the start image, and accept a selection from the displayed candidate images, which is a keyword search-based start image selection process. (2) Accept the user's input specification regarding designating the image being the object of word explanation on the screen displayed on the display means as the user-specified image, store the identification information of the user-specified image in the user-specified image storage means, and display the user-specified image whose identification information is stored in the user-specified image storage means on the screen of the display means as a candidate for the start image for the user. Accept a selection from the displayed candidate images, which is a user-specified image-based start image selection process. (3) Store the cumulative number of passes of each image passed through in past searches in the cumulative number of passes storage means. Excluding the images previously excluded from the target of the start image, display the top predetermined number of images with a large cumulative number of passes on the screen of the display means as candidates for the start image, and accept a selection from the displayed candidate images, which is a cumulative number of passes-based start image selection process The information retrieval system according to claim 1, characterized in that it comprises start image selection means for executing at least one of the processes.

9. The start image selection means executes the keyword search start image selection process of (1) above, and is configured to execute the user-specified image start image selection process of (2) above and / or the cumulative passage number start image selection process of (3) above The information retrieval system according to claim 8, characterized in that.

10. Problem data including an image schema learning problem using the object included in the image of the space or the constituent article included in the image of the object as a theme, and a problem object to be moved indicating the correct answer and its movement destination area, are stored in association with the identification information of the image data of the question source by question storage means, and Using the problem data stored in the problem storage means in association with the identification information of the image data of the question source that is the target of word explanation on the screen displayed on the display means, a problem including the image schema learning problem is displayed on the screen of the display means for the user, the user's answer is received, the correctness of the answer is judged using the problem object to be moved and its movement destination area stored in the problem storage means, and the judgment result is displayed on the screen of the display means and stored in the problem storage means in association with the identification information of the image data of the question source. Problem processing means The information retrieval system according to claim 1, characterized in that it is provided with.

11. Evaluation means for displaying, on the screen of the display means, information on the field of the employment destination recommended to the user, using the judgment result of the correctness of the answer stored in the problem storage means and the employment recommendation field stored in the employment recommendation field image data storage means in association with the identification information of the image data. The information retrieval system according to claim 10, characterized in that it is provided with.

12. Evaluation means for judging whether the user understands the image schema of the problem type using the judgment result of the correctness of the answer to the image schema learning problem of the same problem type stored in the problem storage means for each of the plurality of pieces of image data, and displaying the judgment result on the screen of the display means. The information retrieval system according to claim 10, characterized in that it is provided with.

13. In the problem storage means, as the instruction texts of the problems constituting the problem data, instruction texts of problems in a plurality of languages are stored. The problem processing means is configured to display, on the screen of the display means, the instruction texts of problems in a plurality of languages stored in the problem storage means. The information retrieval system according to claim 10, characterized in that.

14. Manufacturing method manual data storage means for storing, in association with the identification information of the image data, the object forming the space indicated by the image data, or the manufacturing method of the object indicated by the image data or its component articles. Manufacturing method manual display processing means for displaying, on the screen of the display means, the manufacturing method manual data stored in the manufacturing method manual data storage means in association with the identification information of the image data that is the target of word explanation on the screen displayed on the display means. The information retrieval system according to claim 1, characterized in that it comprises.

15. The information retrieval system according to claim 14, characterized in that the manufacturing method manual data is recipe data when the object or its component articles are food.

16. It comprises era switching means for receiving the selection of an era by the user at the era selection section displayed on the screen of the display means. The image data storage means, the explanation data storage means, the hierarchical information storage means, and the designated transition destination image storage means are provided for each era. The selection reception processing means, the screen switching processing means, the explanation display processing means, and the designated transition destination image registration means are configured to execute respective processes using the data stored in the image data storage means, the explanation data storage means, the hierarchical information storage means, and the designated transition destination image storage means for the era selected by the era switching means. The information retrieval system according to claim 1, characterized in that.

17. A program for causing a computer to function as the information retrieval system according to any one of claims 1 to 16.

Images