Information communication terminal device adapted for information search service, display control method in said device, and computer program for executing said method

Abstract

The present invention is an information communication terminal device that is connected to a search server in a manner enabling communication. The information communication terminal device comprises: a user interface unit that includes a touch panel; a search query acquisition unit that acquires a search query; a search result acquisition unit that acquires search results including a plurality of search result images transmitted from the search server in response to the search query; and a display control unit that carries out control for displaying each of the plurality of search result images included in the search results on a screen on which a virtual space is represented on the touch panel. The display control unit performs control so as to carry out animated display of each of the plurality of search result images in order at visually perceptible time intervals.

Description

[Technical Field]

[0001] The present invention relates to an information communication terminal device adapted for an information search service, a display control method in the device, and a computer program for executing the method. [Background technology]

[0002] So-called Internet search is an information search service on a website for searching through vast amounts of information (resources) distributed across the Internet. In such Internet searches, a server program called a search engine on the service provider's website typically performs an index search based on a search query provided by a user and provides the user with the search results. For example, when a user accesses a search site via a web browser and enters a character or string related to the resource they want to search for as a search query, the search engine analyzes the search query and searches an index, scores the extracted resources based on a predetermined ranking algorithm, sorts them in order of score, and provides the list of search results to the user.

[0003] In such Internet searches, users can search not only for text data but also for multimedia data such as image data. In image searches, search results are generally displayed on a browser as a list of reduced-size images, but other display formats have been proposed, such as in Patent Document 1 below.

[0004] That is, Patent Document 1 below discloses a technique for displaying response results to a search query as well as response results to related queries. Patent Document 1 also discloses a technique for displaying these results in an ordered, aligned, or unaligned order along multiple display axes, including at least one axis corresponding to the ordering of various search queries. Patent Document 1 further discloses a technique for translating results along one or more of the display axes so that a user can browse various results. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Special Publication No. 2013-544406 Summary of the Invention [Problem to be solved by the invention]

[0006] Many people often record ideas that come to mind while working, either on paper, such as memo pads or notebooks, or, more recently, on electronic media, such as tablet computers. Designers and other creative professionals in particular often search the internet, hoping for inspiration, and record the ideas they find in the search results on various media.

[0007] In creative activities like this, switching from a pen (stylus) to a mouse or to typing on a keyboard to search the internet interrupts the thought process and hinders a seamless thought process. Additionally, tablet computers are equipped with software keyboards, which allow users to input directly using the pen, but from a usability perspective, this is not sufficient to realize a seamless thought process.

[0008] Furthermore, current image searches simply display extracted images as a list on a browser, without any consideration being given to stimulating and activating people's thinking. The technology disclosed in Patent Document 1 can display images for related queries in addition to images for a search query, either aligned or not, but does not consider realizing a seamless thought process or stimulating and activating thinking.

[0009] The pleasant, tingly sensations that people experience when stimulated by auditory or visual stimuli are known as ASMR (Autonomous Sensory Meridian Response). Therefore, it is believed that pleasant auditory or visual stimuli are effective in stimulating and activating people's thinking.

[0010] Therefore, the present invention aims to provide an information and communication terminal device and a display control method using the same that enable users to embody a seamless thought process and encourage the stimulation and activation of thought.

[0011] More specifically, one of the objects of the present invention is to provide an information and communication terminal device and a display control method using the same that, in an information search service, enables the user to perform the entire process from entering a search query to displaying search results in a smooth series of actions, thereby avoiding interruptions to the thought process and enabling a seamless realization of the thought process.

[0012] Another object of the present invention is to provide an information and communication terminal device and a display control method therefor that can stimulate and activate thinking by displaying search results for a search query in a manner that is comfortable for the user. [Means for solving the problem]

[0013] The present invention, which aims to solve the above problems, comprises the following invention-specific matters or technical features.

[0014] According to one aspect, the present invention provides an information communication terminal device communicatively connected to a search server. The information communication terminal device may include a user interface unit including a touch panel, a search query acquisition unit that acquires a search query, a search result acquisition unit that acquires search results including a plurality of search result images transmitted from the search server in response to the search query, and a display control unit that controls display of each of the plurality of search result images included in the search results on a screen that represents a virtual space on the touch panel. The display control unit may control display of each of the plurality of search result images in sequence with a visually noticeable time difference, as an animation.

[0015] The information communication terminal device may further include a recognition unit that performs a first recognition process based on line drawing data handwritten on the screen of the touch panel. The search query acquisition unit may acquire text (keywords) recognized by the recognition unit as the search query.

[0016] The recognition unit can start execution of the first recognition process in response to a first operation action by a user on the line drawing data on the screen of the touch panel.

[0017] Here, the first operation action may be drawing a line that surrounds at least a part of an area indicated by the line drawing data on the screen of the touch panel.

[0018] The recognition unit may be configured to perform a second recognition process on the image data to generate object information, and the search query acquisition unit may acquire the search query based on the generated object information.

[0019] The display control unit may also control the animation display so that each of the plurality of search result images gradually appears from a first position determined for the search result image within the screen on the touch panel.

[0020] The animation display may be at least one of a manner in which the search result image is visually gradually completed, a manner in which the search result image is visually represented in perspective, and a manner in which the search result image is visually emphasized.

[0021] The display control unit may also determine the first position within the screen for each of the plurality of search result images, and control each of the search result images to be displayed at the determined first position.

[0022] The first positions for the search result images are not geometrically aligned with one another, that is, the display controller may randomly determine the first positions for the search result images.

[0023] The display control unit may also control the animation display so that the displayed search result image gradually disappears after a predetermined time has elapsed.

[0024] In addition, the display control unit may control the at least one featured search result image to be moved to a second position on the screen in response to a second operation action by the user on at least one featured search result image among the plurality of search result images on the screen on the touch panel.

[0025] In addition, the search query acquisition unit may acquire a new search query based on metadata associated with at least one featured search result image in response to a second operation action by the user on at least one featured search result image among the plurality of search result images on the screen on the touch panel.

[0026] In addition, the display control unit may control the display of at least one featured search result image to be highlighted in response to a second operation action by the user on at least one featured search result image among the plurality of search result images on the screen on the touch panel, in order to prompt the user to confirm whether or not to perform a search on the at least one featured search result image.

[0027] Additionally, the search query acquisition unit may acquire a new search query based on metadata associated with the at least one featured search result image when the user selects to perform a search.

[0028] Furthermore, the display control unit may perform control such that the animation display is performed in response to the second operation action, by gradually increasing the speed at which each of the plurality of search result images appears.

[0029] In addition, the display control unit may control the display of at least one non-focused search result image from the screen in response to a third operation action by the user on at least one non-focused search result image among the plurality of search result images on the screen on the touch panel.

[0030] In addition, in response to the third operation action, the search query acquisition unit may acquire a new search query based on metadata associated with some of the search result images, excluding the at least one non-interest search result image, from the plurality of search result images.

[0031] Another aspect of the present invention is a method for displaying search results by an information communication terminal device communicatively connected to a search server. The method may include: acquiring a search query from a user's input to a touch panel of the information communication terminal device; acquiring search results including a plurality of search result images transmitted from the search server in response to the search query; and performing display control to display each of the plurality of search result images included in the search results on a screen representing a virtual space on the touch panel of the information communication terminal device. The performing of the display control may include controlling the plurality of search result images to be displayed in animation in sequence with a visually noticeable time difference.

[0032] Furthermore, according to another aspect, the present invention may be a computer program for realizing a method of causing an information communication terminal device to display search results transmitted from a search server in response to a search query, or a computer-readable recording medium on which the same is non-temporarily recorded.

[0033] In this disclosure, "means" or "unit" does not simply mean physical means, but also includes cases where all or part of the functions of the means or unit are realized by software. Furthermore, the functions of one means or unit may be realized by two or more physical means, and the functions of two or more means or units may be realized by one physical means.

[0034] Furthermore, in this disclosure, "system" includes a logical collection of multiple devices (or functional modules that realize specific functions), regardless of whether each device or functional module is physically configured as a single entity or as separate entities. [Effects of the Invention]

[0035] According to the present invention, it becomes possible for the user to embody a seamless thought process and to encourage the stimulation and activation of thought.

[0036] Furthermore, according to the present invention, in an information search service, a smooth series of operations can be realized from inputting a search query to displaying search results, thereby enabling the user to avoid interruptions in their thought process and embody a seamless thought process.

[0037] Furthermore, according to the present invention, search results for a search query can be displayed in a manner that is comfortable for the user, thereby stimulating and activating the user's thinking and enabling more creative activities.

[0038] Other technical features, objects, effects, or advantages of the present invention will become apparent from the following embodiments described with reference to the accompanying drawings. [Brief explanation of the drawings]

[0039] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of a creative support system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information communication terminal device according to an embodiment of the present invention. [Figure 3] FIG. 3 is a diagram showing an example of the contents of a memory module of an information communication terminal device according to one embodiment of the present invention. [Figure 4] FIG. 4 is a block diagram showing an example of a functional configuration model of an information communication terminal device according to an embodiment of the present invention. [Figure 5] FIG. 5 is a flowchart showing an example of processing by an information communication terminal device according to an embodiment of the present invention. [Figure 6] FIG. 6 is a flowchart showing an example of handwriting input processing by an information communication terminal device according to an embodiment of the present invention. [Figure 7] FIG. 7 is a flowchart showing an example of search processing by an information communication terminal device according to an embodiment of the present invention. [Figure 8]FIG. 8 is a flowchart showing an example of a search result display process performed by an information communication terminal device according to an embodiment of the present invention. [Figure 9] FIG. 9 is a flowchart showing an example of animation display processing by an information communication terminal device according to an embodiment of the present invention. [Figure 10] FIG. 10 is a flowchart showing an example of an object selection process performed by an information communication terminal device according to an embodiment of the present invention. [Figure 11] FIG. 11 is a flowchart showing an example of an object movement process performed by an information communication terminal device according to an embodiment of the present invention. [Figure 12] FIG. 12 is a flowchart showing an example of an object deletion process performed by an information communication terminal device according to an embodiment of the present invention. [Figure 13] FIG. 13 is a diagram showing an example of a screen on a touch panel of an information communication terminal device according to one embodiment of the present invention. [Figure 14] FIG. 14 is a diagram showing an example of a screen on a touch panel of an information communication terminal device according to one embodiment of the present invention. [Figure 15] FIG. 15 is a diagram showing an example of a screen on a touch panel of an information communication terminal device according to one embodiment of the present invention. [Figure 16] FIG. 16 is a diagram showing an example of a screen on a touch panel of an information communication terminal device according to one embodiment of the present invention. [Figure 17] FIG. 17 is a diagram showing an example of a screen on a touch panel of an information communication terminal device according to one embodiment of the present invention. [Figure 18] FIG. 18 is a diagram showing an example of a screen on a touch panel of an information communication terminal device according to one embodiment of the present invention. [Figure 19] FIG. 19 is a flowchart showing another example of the object selection process by the information communication terminal device according to one embodiment of the present invention. [Figure 20] FIG. 20 is a diagram showing another example of the screen on the touch panel of the information communication terminal device according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0040] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and are not intended to exclude various modifications or applications of techniques not explicitly described below. The present invention can be implemented in various modifications (e.g., combinations of the embodiments) without departing from the spirit of the present invention. Furthermore, in the following description of the drawings, identical or similar parts are denoted by identical or similar reference numerals. The drawings are schematic and do not necessarily correspond to actual dimensions, ratios, etc. Parts in the drawings may have different dimensional relationships or ratios.

[0041] (whole system) FIG. 1 is a diagram illustrating an example of a schematic configuration of a creation support system according to an embodiment of the present invention. As shown in the diagram, the creation support system 1 of this embodiment includes, for example, a search server 20 and an information communication terminal device 30 that are communicably connected to each other via a communication network 10. The search server 20 and the information communication terminal device 30 form a server / client model that realizes the information search system. The creation support system 1 may also include a creation support server 40.

[0042] The communication network 10 may include, for example, an IP-based computer network (hereinafter referred to as an "IP network"). In this disclosure, the communication network 10 is used in a broad sense to include the Internet constructed by an IP network, but is not limited to an IP network and is not intended to exclude networks of other protocols that enable node-to-node communication. The communication network 10 may also include a wireless network (e.g., Wi-Fi (registered trademark)) constructed by wireless base stations or wireless access points (not shown). The communication network 10 may also include a mobile communication network that complies with mobile communication system standards.

[0043] The search server 20 is a computing device that provides information search services to users. The search server 20 includes, for example, a search engine 22 and a database 24. Although not shown, the search server 20 may also include a crawler that functions as a robot agent. The crawler visits web pages on the Internet, collects and analyzes information from the visited web pages, and creates an index for referencing and searching the database 24. The search server 20 performs an index search using the search engine 22 based on a search query received from a user's information communication terminal device 30, extracts relevant resources from the database 24, and transmits them to the information communication terminal device 30. The search engine 22 may be configured to perform a search using machine inference based on the search query and / or search history, for example. In the present disclosure, the resources extracted in response to the search query are assumed to be images. Images may include still images and moving images. The websites (search sites) provided by the search server 20 may be any known search sites on the Internet as long as they are accessible to users via the information communication terminal device 30.

[0044] The information and communication terminal device 30 is a computing device operated by a user, typically a tablet computer, but is not limited to this and may be any device configured to realize the present invention. In this example, the information and communication terminal device 30 is a tablet computer that allows interactive operation using a touch panel or touch screen. As will be described later, the information and communication terminal device 30 is equipped with a creative support program that realizes a user interface environment that allows the user to effectively demonstrate their creative abilities.

[0045] The creation support server 40 is, for example, a computing device that functions as, for example, a cloud server for the information communication terminal device 30. The creation support server 40 manages, for example, the accounts of users who operate the information communication terminal device 30. The creation support server 40 can also store, as an operation history, predetermined operation actions performed by users on the information communication terminal device 30 and a series of processing steps corresponding to these actions.

[0046] (Hardware configuration of information communication terminal device) 2 is a diagram showing an example of the hardware configuration of an information communication terminal device adapted to a creative support system according to an embodiment of the present invention. In this diagram, of the various hardware resources constituting the information communication terminal device 30, those particularly related to the present invention are shown.

[0047] That is, as shown in the figure, the information communication terminal device 30 can typically be configured to include one or more processor modules 31, a chipset 32, a memory module 33, an I / O controller 34, various peripheral interfaces 35, and various input / output devices 36 (such as a touch panel 36a).

[0048] The processor module 31 includes, for example, but is not limited to, a processor (processor core), a microcontroller, a digital signal processor, and / or a combination thereof. The chipset 32 ​​is made up of a circuit that integrates a bridge for a bus that connects the processor module 31, the memory module 33, the I / O controller 34, etc., and other components necessary to configure the information communication terminal device 30. The chipset 32 ​​is controlled, for example, by the processor module 31. In this disclosure, the processor module 31 may be simply referred to as a "processor."

[0049] The memory module 33 is a primary storage device made up of volatile memory (e.g., RAM), non-volatile memory (e.g., ROM, flash memory, etc.), and / or a combination thereof, and is available for use by the processor module 31. The memory module 33 may also include video memory (VRAM) for the touch panel 36a. In this disclosure, the memory module 33 may be simply referred to as "memory."

[0050] As shown in FIG. 3 , the memory module 33 stores various software resources, such as device drivers, an operating system (OS) program, one or more application programs, and various data. In the present disclosure, the application programs include a creative support program that runs on the OS of the information communication terminal device 30 under the control of the processor module 31. The creative support program, under the control of the processor module 31, realizes a user interface environment in the information communication terminal device 30 that allows the user to effectively demonstrate their creative abilities. Some processes in the creative support program may be executed as different threads or processes on the processor module 31. The various data include, for example, image data (still image data and moving image data). The various data are stored as image files in, for example, image folders logically configured on the memory module 33.

[0051] The I / O controller 34 is a circuit that controls data transfer between various peripheral interfaces 35 (e.g., an I / O interface 35a and a communication interface 35b). The I / O interface 35a controls the operation of input / output devices 36 that constitute a user interface. The communication interface 35b is a circuit that enables computer communication via the IP network 12.

[0052] The input / output device 36 includes, for example, a touch panel 36a, a speaker 36b, a camera 36c, etc. Although not shown, the input / output device 36 may include a microphone for audio input.

[0053] The touch panel 36a includes a display for displaying multimedia data such as text data, image data (still image data and moving image data), and graphic data, and a transparent touch sensor formed to approximately match the size of the display. The touch panel 36a is an example of a device that realizes a user interface environment. The input / output device 36 may include a display and a touchpad configured separately from the display, instead of the touch panel 36a. Under the control of the processor module 31, the touch panel 36a displays various screens on the display and accepts interactive operations from the user. The user can provide various inputs to the touch panel 36a by operating actions using, for example, a pen (stylus) or a finger.

[0054] The speaker 36b outputs audio or sound based on an audio signal generated by a sound driver (not shown). Under the control of the processor module 31, the speaker 36b, for example, reads out text data or outputs music, sound effects, etc.

[0055] The camera 36c generates image data using an imaging element under the control of the processor module 31. The image data generated by the camera 36c is stored as an image file in the memory module 33. In the present disclosure, such an image file is subjected to image recognition processing to generate object information (e.g., name, etc.), and is converted into a search query based on the object information.

[0056] (Functional configuration model of information communication terminal equipment) 4 is a block diagram showing an example of a functional configuration model of an information communication terminal device according to an embodiment of the present invention. In the figure, the information communication terminal device 30 is shown as a functional configuration model including, for example, a control unit 310, a storage unit 320, a user interface unit 330, and a communication interface unit 340. Such a functional configuration model is realized by executing the creation support program under the control of the processor module 31, in cooperation with the various software and / or hardware resources described above.

[0057] The control unit 310 comprehensively controls and executes various processes (tasks) in the information communication terminal device 30. In this example, the control unit 310 is configured to include a search query acquisition unit 311, a recognition unit 312, a search request transmission unit 313, a search result acquisition unit 314, and a display control unit 315. Details of these functional configurations in the control unit 310 will be described later.

[0058] The storage unit 320 may include, for example, a search result storage unit 321, a screen data storage unit 322, and an operation history storage unit 323.

[0059] The search result storage unit 321 stores search results obtained from the search server 20 under the control of the control unit 310. In the present disclosure, the search results include one or more images. The images are image files in a predetermined image format (e.g., JPEG, PNG, WebP, etc.) and are associated with metadata. The image files may include metadata as part of the image files. The metadata may be, for example, text data indicating the URL of the image or its name, but is not limited to these. In the present disclosure, images obtained as search results may be referred to as "search result images." The search result storage unit 321 may also include a storage area for storing search result images selected by the user. For example, the search result storage unit 321 stores the search result image files selected by the user in an asset folder under the control of the control unit 310.

[0060] The screen data storage unit 322, under the control of the control unit 310, stores data (hereinafter referred to as "screen data") for representing the screen provided by the user interface unit 330 as a virtual space. The screen data may include objects and their coordinate information, etc. In the present disclosure, an object may be a symbol or element displayed on the screen, including, for example, a search result image, an icon, a line drawing, etc. The screen data stored in the screen data storage unit 322 is rendered by the control unit 310 (e.g., a renderer) and passed to the user interface unit 330, whereby the screen including the object is displayed on the touch panel 36a.

[0061] The operation history storage unit 323, under the control of the control unit 310, stores, as an operation history, a predetermined operation action received via the touch panel 36a and the process and results of a series of processes performed by the control unit 310 in response to the operation action. Note that such an operation history may be held on the creation support server 40 under the control of the control unit 310.

[0062] The user interface unit 330 provides an interactive operating environment to the user via the touch panel 36a or the like. That is, under the control of the control unit 310, the user interface unit 330 displays a screen representing a virtual space on the touch panel 36a and accepts various operation actions by the user in response to the screen. The user inputs operation actions using, for example, a pen.

[0063] As one example, the user interface unit 330 accepts handwritten input by the user and provides a screen on which a line drawing is displayed in response to the input. The handwritten input information may be, for example, a line drawing showing a character or a string of characters (text), a closed curve, or the like. As another example, the user interface unit 330 provides a screen on which an object is animated under the control of the control unit 310. As yet another example, the user interface unit 330 accepts a user's operation action on an object. Examples of operation actions on an object include, but are not limited to, tapping (single tapping, double tapping, etc.), dragging, flicking, swiping, etc.

[0064] The communication interface unit 340 controls communication so that the information communication terminal device 30 can access the search server 20 via the communication network 10. For example, under the control of the control unit 310, the communication interface unit 340 controls the transmission of a search request based on a search query to the search server 20. Also, for example, the communication interface unit 340 receives search results transmitted from the search server 20 in response to the search query of the search request, and passes them to the control unit 310.

[0065] Next, the functional configuration of the control unit 310 will be described in detail.

[0066] The search query acquisition unit 311 acquires search queries for the search server 20 from various channels. As an example, the search query acquisition unit 311 identifies a line drawing in the screen data in response to a predetermined operation action by the user on the touch panel 36a, and acquires, as a search query, a result of character recognition by the recognition unit 312 based on data related to the line drawing (line drawing data). More specifically, when the user, as a predetermined operation action, moves the pen in contact with the touch panel 36a so as to surround at least a part of an area showing the line drawing data on the screen displayed on the touch panel 36a, the search query acquisition unit 311 identifies the line drawing data surrounded by the trajectory of the movement and transfers it to the recognition unit 312. In response to this, the search query acquisition unit 311 acquires text data output from the recognition unit 312 and sets it as the search query. The operation action of the user moving the pen in such a way as to surround at least a part of an area showing the line drawing is one aspect of a first operation action. In this case, the first operation action may be a series of operation actions in which the user moves the touch panel 36a to surround at least a part of the area showing the line drawing and then performs an additional gesture (e.g., a check motion). Alternatively, the first operation action may be an operation action in which the user taps (e.g., double taps) or presses and holds near the line drawing on the screen displayed on the touch panel 36a.

[0067] As another example, the search query acquisition unit 311 identifies an object in the screen data in response to a predetermined operation action of the user on the touch panel 36a, and acquires metadata associated with the object as a search query. For example, an operation action such as a tap by the user on an object (search result image) of interest on the screen displayed on the touch panel 36a is one aspect of the second operation action. Alternatively, an operation action such as a flick by the user on a search result image that the user wants to exclude (not focus on) from among multiple search result images on the screen displayed on the touch panel 36a is one aspect of the third operation action. In response to the third operation action, the search query acquisition unit 311 acquires a new search query based on metadata associated with the search result images excluding the excluded search result images from among the multiple search result images.

[0068] As yet another example, the search query acquisition unit 311 selects an arbitrary object in the screen data at an arbitrary timing, and acquires metadata associated with the object as a search query.

[0069] The recognition unit 312 performs a predetermined recognition process on various objects on the screen displayed on the touch panel 36a to generate and output text data. In the present disclosure, the recognition unit 312 is configured to include a character recognition unit 3121 and an image recognition unit 3122. The recognition unit 312 may also include a voice recognition unit (not shown) that performs voice recognition processing to generate text data. The search query acquisition unit 311 may acquire a search query based on the text data obtained by the voice recognition processing.

[0070] The character recognition unit 3121 uses a predetermined character recognition algorithm to analyze the line drawing data, and generates and outputs text data that the line drawing data likely represents (first recognition process). The character recognition unit 3121 may also be configured to have a function of collating the generated text data with a dictionary database (not shown) and converting it into existing words, for example. This allows what the user originally intended to be obtained as a character recognition result, even for scribbled line drawing data.

[0071] The image recognition unit 3122 uses a predetermined image recognition algorithm to analyze image data such as a photograph, extract or recognize objects that the image data may represent, and output object information (such as names) as text data (second recognition process). For example, if the image data is a landscape photograph of a bridge with cars passing by, the image recognition unit 3122 recognizes "car" and "bridge" from the image data, and generates and outputs text data indicating their names. For example, the image recognition unit 3122 may perform image recognition processing on image data such as a photograph that the user reads from an image folder, thereby generating object information.

[0072] In this example, the recognition unit 312 is configured within the control unit 310, but this is not limited to this, and part or all of it may be configured on an external cloud server (e.g., the creative support server 40), and the recognition results may be obtained via the communication interface unit 340.

[0073] Furthermore, instead of outputting a unique recognition result, the recognition unit 312 may output multiple recognition result candidates, allowing the user to select a desired one from the multiple recognition result candidates displayed on the screen of the touch panel 36a.

[0074] The search request sending unit 313 generates and outputs a search request based on the search query acquired by the search query acquisition unit 311. The search request output from the search request sending unit 313 is transmitted to the search server 20 via the communication interface unit 340.

[0075] The search result acquisition unit 314 receives and acquires search results transmitted from the search server 20 in response to a search request via the communication interface unit 340. For example, the search result acquisition unit 314 divides a plurality of search result images searched and extracted by the search server 20 into a predetermined number of groups and acquires them as search results. The search result acquisition unit 314 passes the search results including the acquired plurality of search result images to the display control unit 315. The search result acquisition unit 314 also saves the search results in the search result storage unit 321.

[0076] The display control unit 315 controls the display of a screen that represents a virtual space on the touch panel 36a. For example, the display control unit 315 controls the display of a trajectory of a user's handwritten input on the screen displayed on the touch panel 36a in response to the user's handwritten input.

[0077] Furthermore, for example, the display control unit 315 controls the display of multiple search result images acquired by the search result acquisition unit 314 on the screen of the touch panel 36a in an animated manner. More specifically, the display control unit 315 animates the multiple search result images so that they appear at any position on the screen in order with a visually noticeable time difference. The display control unit 315 randomly determines the appearance position (first position) of each search result image on the screen. Therefore, the appearance positions of the multiple search result images are not geometrically aligned with each other. One example of an animation display in which search result images appear on the screen is a mode in which the search result images gradually complete from a visually empty or blurred state. Another example is a mode in which the search result images appear to be moving in a virtual space. An example of a mode in which the search result images appear to be moving in a virtual space includes, for example, a mode in which the search result images appear to be visually approaching using perspective. Yet another example is a mode in which the search result images are visually emphasized.

[0078] The display control unit 315 may also include a physics calculation engine 3151. The physics calculation engine 3151 simulates the behavior of objects (e.g., search result images) in virtual space by performing physical calculations according to predetermined physical parameters. Examples of predetermined physical parameters include mass, velocity, external force, and fluid velocity. As one example, the physics calculation engine 3151 simulates the behavior of search result images in virtual space by likening them to the behavior of sparks floating in the air from a bonfire. As another example, the physics calculation engine 3151 simulates the behavior of search result images in virtual space by likening them to the behavior of bubbles rising from water. As a result, instead of or in addition to the above-described animated display, the display control unit 315 enables the search result images on the screen to be animated, as if they are floating in virtual space. Such animated display of search result images provides a sense of comfort to the user, stimulates and stimulates thinking, and enables more creative activities.

[0079] Furthermore, the physics calculation engine 3151 can simulate the fluctuation of search result images floating in virtual space so that they exhibit 1 / f fluctuation. 1 / f fluctuation is a fluctuation in which the power spectral density is inversely proportional to the frequency f (where f>0). By displaying an animation that expresses the behavior of search result images with such 1 / f fluctuation, the user can resonate with their own biological rhythm, thereby stabilizing their mind and enabling them to engage in more creative activities.

[0080] Furthermore, when a predetermined time (e.g., 10 seconds) has elapsed since the animated display of the search result images, the display control unit 315 controls the animation display so that the search result images gradually disappear or fade out naturally, thereby erasing the search result images from the screen. This prevents the screen from being overflowed with search result images, and an appropriate number of search result images are animatedly displayed, preventing the user's thought process from being hindered. Note that even search result images that have been erased from the screen may remain stored without being erased from the search result storage unit 321.

[0081] (Processing of information communication terminal device) 5 is a flowchart showing an example of processing by an information communication terminal device according to one embodiment of the present invention. This processing is realized by the information communication terminal device 30 executing a creative support program under the control of the processor module 31 and cooperating with various software and / or hardware resources. Note that the figure explains the flow of processing in response to a user's operation action on the information communication terminal device 30 executing the creative support program.

[0082] That is, when the control unit 310 of the information communication terminal device 30 starts executing the creative support program, it displays a screen on the touch panel 36a as a user interface environment for providing creative support to the user, and monitors whether or not the user has made any input (operation action) on the screen (S501).

[0083] When the control unit 310 determines that the user has performed an operation action on the screen (Yes in S501), the control unit 310 then determines whether or not there is a selectable object at the position on the screen corresponding to the operation action (S502). That is, the control unit 310 determines whether or not the user has performed an input on an empty area (an area where nothing is drawn) on the screen of the touch panel 36a.

[0084] If the control unit 310 determines that there is no selectable object at the position on the screen corresponding to the user's operation action (No in S502), the control unit 310 then determines whether the user's operation action is handwriting input (S503). Handwriting input is an action consisting of a combination of one or more strokes in which the user moves the pen while keeping it in physical contact with the touch panel 36a.

[0085] When the control unit 310 determines that the user's operation action is handwriting input (Yes in S503), it performs handwriting input processing according to the input strokes (S504). Details of the handwriting input processing will be described with reference to Fig. 6. After the handwriting input processing is completed, the control unit 310 returns to the processing of monitoring the user's operation action (S501).

[0086] On the other hand, if the control unit 310 determines that the user's operation action is not handwriting input (No in S503), in this example, it is considered to be invalid input, and the control unit 310 returns to the process of monitoring the user's operation action (S501). For example, a very short single stroke may be considered to be invalid input.

[0087] On the other hand, if the control unit 310 determines that a selectable object exists at a position on the screen corresponding to the user's operation action (Yes in S502), it identifies the operation action (S505) and performs processing accordingly. Examples of operation actions include, but are not limited to, a "tap" that lightly taps an object on the screen with a pen, a "drag" that grabs an object on the screen with a pen and drags it, a "flick" that quickly sweeps an object on the screen with a pen, and a "slide" or "swipe" that touches an empty area on the screen with a pen and moves it in a predetermined direction. This disclosure describes processing based on tapping, dragging, and flicking among the operation actions. In the following example, the operation action on a selectable object on the screen corresponds to the second operation action or the third operation action.

[0088] That is, when the user's operation action is a tap, the control unit 310 performs a process of selecting an object (S506). When the user's operation action is a drag, the control unit 310 performs a process of moving the object in response to the drag (S507). When the user's operation action is a flick, the control unit 310 performs a process of flicking the object out of the screen in response to the flick, thereby erasing it from the screen (S508). Details of these processes will be described with reference to FIGS. 6 to 11, respectively. After each process is completed, the control unit 310 returns to the process of monitoring the user's operation action (S501).

[0089] 6 is a flowchart showing an example of handwriting input processing by an information communication terminal device according to an embodiment of the present invention, which shows details of the handwriting input processing (S504) shown in FIG.

[0090] That is, as shown in the figure, the control unit 310 controls the touch panel 36a to draw a line on the screen displayed on the touch panel 36a in accordance with an input signal output by the touch panel 36a in response to the trajectory of a stroke made by the user's contact movement of the pen (S601). Next, the control unit 310 determines whether the user's handwriting input has ended (S602). For example, if the control unit 310 does not detect a next stroke within a predetermined time after the pen is released from the touch panel 36a after a stroke, the control unit 310 determines that the handwriting input has ended. If the control unit 310 determines that the user's handwriting input has not ended (No in S602), the control unit 310 continues drawing the line (S601).

[0091] On the other hand, when the control unit 310 determines that the user's handwriting input has ended (Yes in S602), it defines line drawing data based on the user's handwriting input (S603). The line drawing data may be vector data of a set of multiple lines (regardless of whether they are straight or curved).

[0092] Next, the control unit 310 determines whether the defined line drawing data represents a predetermined graphic shape (S604). In this example, the predetermined graphic shape is a closed graphic shape. A closed graphic shape is not limited to a graphic shape in which lines are completely closed, but also includes a shape that can be considered substantially closed even if some of the lines are separated. The predetermined graphic shape may also be limited to a closed graphic shape consisting of simple curves, such as a circle or an ellipse. Alternatively, the control unit 310 may determine whether the line drawing data represents a predetermined geometric line drawing (e.g., an underline or a tick mark) instead of a closed graphic.

[0093] If the control unit 310 determines that the defined line drawing data does not represent a predetermined graphic shape (No in S604), the control unit 310 temporarily stores the line drawing data in the screen data storage unit 322 (S605). That is, the line drawing data temporarily stored in the screen data storage unit 322 can become the above-mentioned other line drawing data (object) to be selected by a subsequent operation action of the user.

[0094] On the other hand, if the control unit 310 determines that the defined line art data indicates a predetermined graphic shape (Yes in S604), the control unit 310 subsequently determines whether other line art data (line art data to be selected) exists within the area (range) surrounded by the predetermined graphic shape (S606). In other words, the control unit 310 determines whether the user intentionally selected other line art data present on the screen. The other line art data here may be line art data previously handwritten by the user. The area surrounded by a predetermined graphic shape includes an area that can be considered to be substantially surrounded. Furthermore, if at least a portion of the other line art data is included within the area, the control unit 310 may determine that other line art data exists. In other words, if the user draws a closed figure to enclose at least a portion of the area indicated by the line art data previously handwritten and displayed on the screen in order to select the other line art data, the control unit 310 may determine that other line art data exists. In this way, the operation action of drawing line art data indicating a predetermined graphic shape in order to select the other line art data corresponds to the first operation action.

[0095] If the control unit 310 determines that no line drawing data to be selected exists within the area surrounded by the predetermined graphic shape (No in S606), the control unit 310 considers that the defined line drawing data is not handwritten and input for selection by the user, and temporarily stores the line drawing data in the screen data storage unit 322 (S605). This allows the user to store line drawing data such as memos written on the touch panel 36a. Such line drawing data can become part of the screen data.

[0096] On the other hand, if the control unit 310 determines that other line drawing data exists within the area surrounded by the predetermined graphic shape (Yes in S606), the character recognition unit 3121 performs character recognition on the other line drawing data within the area (S607). That is, the character recognition unit 3121 performs character recognition processing on the line drawing data to generate text data. Subsequently, the search result acquisition unit 314 acquires the generated text data as a search query (S608). As will be described later, the search request transmission unit 313 transmits a search request based on the acquired search query to the search server 20, and the search result acquisition unit 314 acquires the search results (search result images) from the search server 20.

[0097] Next, the display control unit 315 converts the generated text data into, for example, an icon that represents the character-recognized text (S609). The display control unit 315 controls so that the converted icon is displayed on the screen instead of the handwritten line drawing data. Alternatively, the generated text data may be treated as text data as is without being converted into an icon.

[0098] As described above, when a user inputs handwriting in an empty area on the screen of the touch panel 36a, the information communication terminal device 30 determines whether the input is text or a selection of previously input text. If the information communication terminal device 30 determines that the handwriting input is text, it temporarily stores the line drawing data representing the input. If the information communication terminal device 30 determines that the input is a selection of line drawing data representing previously input text, it performs character recognition processing on the line drawing data, converts it into text data, and acquires it as a search query.

[0099] 7 is a flowchart showing an example of search processing by an information communication terminal device according to an embodiment of the present invention. Execution of this processing begins when a search query is acquired by the search query acquisition unit 311. For example, the control unit 310 may execute the search processing in response to the acquisition of the search query in a separate thread or process. As a result, as will be described later, a new search query is acquired from the search results in response to the search request resulting from the acquisition of the initial search query, and further search processing is performed in a chain or derivative manner.

[0100] That is, as shown in the figure, the control unit 310 monitors whether a search query has been acquired by the search query acquisition unit 311 (S701). If the control unit 310 determines that a search query has been acquired (Yes in S701), the search request transmission unit 313 generates a search request based on the search query and transmits it to the search server 20 (S702). When the search server 20 receives a search request from the information communication terminal device 30, it extracts relevant resources by performing a search with reference to the database 24 based on the search query included in the search request, and transmits this as search results to the information communication terminal device 30. The search results include one or more search result images. When the search server 20 extracts the search results, it transmits them to the information communication terminal device 30 in predetermined numbers, for example, in response to a fetch request from the search result acquisition unit 314 of the information communication terminal device 30.

[0101] As a result, the search result acquisition unit 314 receives the search results sent by the search server 20 in response to the search request (S703). For example, the search result acquisition unit 314 sends a fetch request for multiple search result images searched and extracted by the search server 20, divides them into a predetermined number of parts, and acquires them as search results.

[0102] Next, the display control unit 315 performs processing to display the acquired search results (search result images) on the screen (S704). Details of the search result display processing will be described with reference to FIG. 8. In general, the control unit 310 controls the display of the search results so that each of the multiple search result images is displayed in an animated manner at any position on the screen in order with a visually noticeable time difference. Examples of animation display include, but are not limited to, a manner in which the search result images are visually gradually completed, a manner in which the search result images visually express a sense of perspective on the screen, and a manner in which the search result images are visually emphasized.

[0103] After the display control unit 315 has finished displaying the search results, the control unit 310 determines whether to end the search process (S705). For example, if the user selects to pause the search, the control unit 310 ends the search process. On the other hand, if the control unit 310 determines to continue the search process (No in S705), the search query acquisition unit 311 acquires a new search query based on the search results (S706). The search query acquisition unit 311 may acquire the search query based on, for example, metadata associated with the search result image. When the search query acquisition unit 311 acquires a new search query, the control unit 310 returns to the process of S702 and continues the search process. That is, the search request transmission unit 313 transmits a search request based on the acquired new search query to the search server 20, acquires search results in response to this, and displays them on the screen. In this way, a new search query is acquired from the search results in response to the search request obtained by the initial search query, and further search processes are performed in a chain or derivative manner.

[0104] 8 is a flowchart showing an example of a search result display process performed by an information communication terminal device according to an embodiment of the present invention, which shows details of the handwriting input process (S704) shown in FIG.

[0105] As shown in the figure, when the search result acquisition unit 314 acquires search result images from the search server 20 in bulk or in parts, the display control unit 315 selects one of the search result images (S801). Next, the display control unit 315 starts animation display processing for the selected search result image (S802). The display control unit 315 may execute the animation display processing, for example, in a separate thread. As a result, the selected search result image is displayed as an animation, gradually appearing sequentially as an object in a virtual space with a visually recognizable time difference. Details of the animation display processing will be described with reference to FIG. 9. At this time, the display control unit 315 may control the display of at least a portion of the metadata associated with the search result image in association with the animated object.

[0106] Next, the display control unit 315 determines whether all of the acquired search result images have been selected (S803). If the display control unit 315 determines that all of the search result images have not been selected (No in S803), the process returns to S801. On the other hand, if the display control unit 315 determines that all of the search result images have not yet been selected (Yes in S803), the process of displaying the search results ends.

[0107] 9 is a flowchart showing an example of animation display processing by an information communication terminal device according to an embodiment of the present invention, which shows details of the animation display processing (S802) shown in FIG.

[0108] As shown in the figure, the display control unit 315 simulates the behavior of each search result image in the virtual space by performing physical calculations based on predetermined physical parameters (e.g., mass, speed, external force, air current, etc.) using the physics calculation engine 3151 (S901). For example, the display control unit 315 simulates each search result image as if it were floating in the virtual space. In this case, the display control unit 315 may also perform physical calculations so that the fluctuation of the search result images floating in the virtual space is 1 / f fluctuation.

[0109] Next, the display control unit 315 performs control so that the search result images are displayed as animations on a screen representing a virtual space according to the results of the physics calculations (S902). That is, the display control unit 315 generates screen data according to the results of the physics calculations and outputs this to the user interface unit 330. As a result, the search result images are displayed, for example, as if they are floating on the screen representing the virtual space. At this time, the display control unit 315 may perform control so that at least a portion of the metadata associated with the search result images is displayed in association with the animated object.

[0110] Next, the display control unit 315 determines whether the animation display of the search result image satisfies a predetermined end condition (S903). The predetermined end condition is, for example, whether a predetermined time (e.g., 10 seconds) has elapsed since the start of the animation display of the search result image, but is not limited to this.

[0111] If the display control unit 315 determines that the animation display of the search result image does not satisfy the predetermined end condition (No in S903), it continues to display the animation by the above simulation. On the other hand, if the display control unit 315 determines that the animation display of the search result image satisfies the predetermined end condition (Yes in S903), the display control unit 315 controls the animation display so that the search result image disappears (fade out) naturally, and erases the search result image from the screen (S904).

[0112] As described above, the information communication terminal device 30 acquires a search query based on handwritten input by the user, makes a search request to the search server 20, acquires search results (search result images) in response to this, and performs control so that the search result images are displayed as animation on a screen representing a virtual space. The information communication terminal device 30 also acquires a new query based on the search results, and further acquires search results from the search server 20. As a result, various search queries are developed starting from the initial handwritten input, allowing searches that are not limited to a single perspective to be performed, thereby stimulating and activating the user's thinking.

[0113] 10 is a flowchart showing an example of the object selection process by the information communication terminal device according to one embodiment of the present invention, which shows details of the object selection process (S506) shown in FIG.

[0114] 5, when the user taps on a selectable object on the screen as an operation action, the control unit 310 selects the object corresponding to the tapped position as the object of interest (S1001). Here, the object is assumed to be a search result image or an icon.

[0115] Next, the control unit 310 reads metadata associated with the selected object of interest (S1002), and the search query acquisition unit 311 sets the read metadata as a search query (S1003). This enables the information communication terminal device 30 to transmit a search request based on the set search query to the search server 20, and to obtain search results transmitted from the search server 20 in response to the request.

[0116] Next, the display control unit 315 changes the value of a predetermined physical parameter for animation display (S1003). For example, the display control unit 315 changes the value of the predetermined physical parameter so that the movement speed of the object in the virtual space increases. The change in the value of the predetermined physical parameter may affect all objects in the virtual space. This causes the object to move faster in the virtual space than before the selection, giving the user the feeling that the entire screen has been activated. Note that after changing the value of the predetermined physical parameter so that the movement speed of the object increases, the display control unit 315 may return the value to its original value after a predetermined time has elapsed.

[0117] When a search result image is selected as an object of interest by a user's tap, the control unit 310 may control the search result image to be saved in an asset folder so that the search result image can be distinguished from other search result images. Alternatively, when the user's operation action is a double tap on the search result image, the control unit 310 may control the search result image to be saved in the asset folder.

[0118] As described above, when a user selects an object on the screen by tapping, the information communication terminal device 30 can acquire a new search query based on the selected object and obtain further search results. Furthermore, the information communication terminal device 30 can use the user's tap as a trigger to control the animation display so that the object on the screen moves faster, thereby creating a transition from a calm state to an active state and encouraging the user to think more actively.

[0119] The object selection process may include a processing step for user confirmation, as will be described later.

[0120] 11 is a flowchart showing an example of object movement processing by an information communication terminal device according to an embodiment of the present invention, which shows details of the object movement processing (S507) shown in FIG.

[0121] 5, when the user drags a selectable object on the screen as an operation action, the display control unit 315 selects the object corresponding to the drag start position as the object of interest and controls the animation display so that the object of interest moves in accordance with the drag movement (S1101). The control unit 310 also monitors whether the drag has ended (S1102). That is, the user can move the object from its current position to a second position by dragging.

[0122] When the control unit 310 determines that the dragging has ended (Yes in S1102), as described above, the control unit 310 reads metadata associated with the dragged object of interest (S1103), and the search query acquisition unit 311 sets the read metadata as the search query (S1104). As a result, the information communication terminal device 30 transmits a search request based on the set search query to the search server 20, and acquires search results transmitted from the search server 20 in response to the request. Next, the display control unit 315 changes the value of a predetermined physical parameter for animation display (S1105). This causes the object to move faster in the virtual space than before the selection, giving the user the feeling that the entire screen has been activated. Note that the display control unit 315 may change the value of the predetermined physical parameter to increase the object's movement speed, and then restore the value to its original value after a predetermined time has elapsed.

[0123] For example, if the user drags a search result image onto another icon (for example, an icon indicating an asset folder), the control unit 310 may perform control so that the search result image is saved in the asset folder.

[0124] As described above, the user can drag an object of interest to any location on the screen. The information and communication terminal device 30 can acquire a search query based on the object on which the user has performed an operation action, and can acquire further search results. Furthermore, the information and communication terminal device 30 can use the user's drag as a trigger to increase the speed at which the object moves on the screen, thereby creating a transition from a calm state to an active state, thereby stimulating and activating the user's thinking.

[0125] 12 is a flowchart showing an example of an object deletion process by an information communication terminal device according to an embodiment of the present invention, which shows details of the object deletion process (S508) shown in FIG.

[0126] That is, as shown in FIG. 5, when the user flicks on a selectable object on the screen as an operation action, the control unit 310 selects the object corresponding to the flicked position as a non-focused object (excluded object), and the display control unit 315 then controls the animation display so that the non-focused object is ejected off the screen (S1201).

[0127] Next, the control unit 310 selects at least one of the remaining objects on the screen that have not been selected as a non-focus object (S1202). For example, the control unit 310 randomly selects one object from the objects on the screen. In this case, the control unit 310 may select the one object from among objects that have not been set as a search query in the past.

[0128] Next, the control unit 310 reads metadata associated with the selected object (S1203), and the search query acquisition unit 311 sets the read metadata as the search query (S1204). As a result, the information communication terminal device 30 transmits a search request based on the set search query to the search server 20, and acquires search results transmitted from the search server 20 in response to the request. Next, the display control unit 315 changes the value of a predetermined physical parameter for animation display (S1205). This causes the object to move faster in the virtual space than before the selection, giving the user the feeling that the entire screen has been activated. Note that the display control unit 315 may change the value of the predetermined physical parameter to increase the object's movement speed, and then restore the original value after a predetermined time has elapsed.

[0129] As described above, the user can erase an object that the user determines to be unnecessary by flicking it off the screen. The information communication terminal device 30 can acquire a search query based on the object that the user did not erase, and can acquire further search results. Furthermore, the information communication terminal device 30 can change from a calm state to an active state by increasing the speed at which the object moves on the screen as the user drags it, thereby stimulating and activating the user's thinking.

[0130] (Example) Next, a specific example of an operation performed by a user on the information communication terminal device 30 configured as above will be described.

[0131] That is, suppose that the user handwrites a keyword or text (word) T, for example, "mountain," on a screen 1300 displayed on the touch panel 36a as shown in Fig. 13. As described above, at this point, the information communication terminal device 30 internally handles the handwritten "entity" as line drawing data (e.g., vector data). Next, the user draws a circle C on a screen 1400 as shown in Fig. 14, surrounding the handwritten text T, "mountain."

[0132] In response to this, the information communication terminal device 30 performs character recognition processing on the line drawing data corresponding to "mountain" circled C, generates text data corresponding to "mountain," and acquires this as a search query. At this time, to notify the user that character recognition for the line drawing data was successful, the information communication terminal device 30 iconizes or tiles the recognized text, as shown in screen 1500 in FIG. 15, for example. Alternatively, although not shown, the recognized words may be displayed as text data (characters in a predetermined font). Next, the information communication terminal device 30 generates a search request based on the acquired search query and transmits it to the search server 20. In response to this, the search server 20 refers to the database 24, extracts one or more search result images, and transmits them to the information communication terminal device 30 as search results.

[0133] When the information communication terminal device 30 receives the search results transmitted from the search server 20, the information communication terminal device 30 controls the display so that multiple search result images RESULT(1) included in the search results appear in sequence at arbitrary positions (e.g., random positions) on a screen 1600, floating in virtual space in an animated manner, as shown in FIG. 16. FIG. 17 shows a screen 1700 a short time after the state of the screen 1600 shown in FIG. 16. FIG. 17 shows a state in which a search result for "mountain" also includes an image of a mountain with a "buffalo" in it. This is because such an image may typically include metadata such as "buffalo." As a result, the keyword "mountain" evokes the unexpected keyword "buffalo" in the user's mind.

[0134] 16 and 17, the user can select a search result image by tapping, move it by dragging, or flick it off the screen. Meanwhile, while the search result image is being displayed as an animation, the information communication terminal device 30 can acquire a new search query and continue the search process.

[0135] For example, suppose that the user taps with a pen on the search result image showing "buffalo" in the lower left corner of the screen 1700 shown in FIG. 17. In response to this, the information communication terminal device 30 refers to the metadata associated with the search result image and acquires it as a search query. In this example, it is assumed that "buffalo" is associated with the metadata. Therefore, the information communication terminal device 30 generates a search request with "buffalo" as the search query and transmits it to the search server 20. As a result, the information communication terminal device 30 acquires the search result image for "buffalo" transmitted from the search server 20, and displays, for example, a search result image RESULT(2) as shown in FIG. 18 as an animation on the screen.

[0136] As described above, according to this embodiment, when a user handwrites some text on the screen of the touch panel 36a and draws a path that surrounds it, the information communication terminal device 30 performs character recognition processing on the line drawing data within the area surrounded by the path, converts it into text data, and acquires it as a search query. Search results in response to a search request based on the acquired search query are then acquired from the search server 20. This eliminates the need for the user to switch from pen operation to keyboard or mouse operation for an Internet search, allowing for a smooth flow of operations and uninterrupted thought process. This allows the user to embody a seamless thought process, encouraging and stimulating thought.

[0137] Furthermore, according to this embodiment, multiple search result images contained in search results in response to a search request are not displayed in an ordered list, but are displayed in animation in any position on the screen in order with a visually noticeable time difference, which gives the user a comfortable feeling, stimulates and activates their thinking, and enables more creative activities.

[0138] (Variation) Fig. 19 is a flowchart showing another example of object selection processing by an information communication terminal device according to an embodiment of the present invention. This figure shows a modification of the object selection processing shown in Fig. 10. Specifically, the object selection processing shown in Fig. 19 differs from the object selection processing shown in Fig. 10 in that a processing step is added in which the user confirms whether or not to execute a search for the selected object.

[0139] As described above, assume that the user taps with a pen on the search result image showing "Buffalo" at the bottom left of the screen shown in Fig. 17. In response to this, control unit 310 selects the object corresponding to the tapped position as the object of interest (S1901).

[0140] Next, under the control of the control unit 310, the display control unit 315 highlights the selected object (S1902), controls the animation display to temporarily stop, and, under the control of the control unit 31, prompts the user to select whether or not to perform a search according to the object (S1903). FIG. 20 shows an example of a screen on which the selected object is highlighted. In the same figure, an example is shown in which the selected object 2001 (i.e., "buffalo") is enlarged and displayed in an area approximately at the bottom center of the screen 2000. If the user wishes to perform a search for the selected object 2001, the user touches, for example, the object 2001 itself or the execute button 2002. Alternatively, if the user wishes to cancel, the user touches an area on the screen 2000 other than the object 2001 or the cancel button 2003.

[0141] If the user selects to perform a search according to the selected object (Yes in S1902), the control unit 310 reads metadata associated with the selected object of interest (S1904). Subsequently, the search query acquisition unit 311 sets the read metadata as a search query (S1905). This enables the information communication terminal device 30 to transmit a search request based on the set search query to the search server 20 and to obtain search results transmitted from the search server 20 in response to the request.

[0142] Next, the display control unit 315 changes the value of a predetermined physical parameter for animation display (S1903). For example, the display control unit 315 changes the value of the predetermined physical parameter so that the movement speed of the object in the virtual space increases. The change in the value of the predetermined physical parameter may affect all objects in the virtual space. This causes the object to move faster in the virtual space than before the selection, giving the user the feeling that the entire screen has been activated. Note that after changing the value of the predetermined physical parameter so that the object's movement speed increases, the display control unit 315 may return the value to its original value after a predetermined time has elapsed.

[0143] On the other hand, if the user selects not to execute a search according to the selected object (No in S1903), control unit 310 ends the object selection process (S506 shown in FIG. 5).

[0144] As described above, when the user selects an object on the screen by tapping it, the information communication terminal device 30 enlarges the object and prompts the user to confirm whether or not to perform a search. This allows the screen display to behave more like what the user expects, thereby improving usability.

[0145] The above-described embodiments are merely examples for explaining the present invention, and are not intended to limit the present invention to these embodiments. The present invention can be embodied in various forms without departing from the spirit of the present invention.

[0146] For example, in the methods disclosed herein, steps, operations, or functions may be performed in parallel or in a different order, provided that the results do not conflict. The steps, operations, and functions described are provided merely as examples, and some of the steps, operations, and functions may be omitted or combined into one, or other steps, operations, or functions may be added, without departing from the spirit of the invention.

[0147] Furthermore, although various embodiments are disclosed in this specification, specific features (technical matters) in one embodiment can be added to or substituted for specific features in other embodiments, with appropriate modifications, and such forms are also included in the spirit of the present invention. [Explanation of symbols]

[0148] 1. Creative support system 10. Communication Network 20...Search server 22...Search engine 24...Database 30...Information and communication terminal equipment 31...Processor module 32...Chipset 33...Memory module 34...I / O controller 35...Peripheral interface 35a...I / O interface 35b...Communication interface 36...Input / output devices 36a...Touch panel 36b...Speaker 36c camera 310...Control unit 311...Search query acquisition section 312...Recognition section 313...Search request sending unit 314...Search result acquisition section 315...Display control unit 320...Storage section 321...Search result storage section 322...Screen data storage unit 323...Operation history memory unit 330...User interface section 340...Communication interface unit 40...Creation support server

Claims

1. An information communication terminal device communicably connected to a search server, a user interface unit including a touch panel; a search query acquisition unit that acquires a search query; a search result acquisition unit that acquires search results including a plurality of search result images transmitted from the search server in response to the search query; a display control unit that performs control to display each of the plurality of search result images included in the search result on a screen that represents a virtual space on the touch panel, The display control unit causing each of the plurality of search result images to appear in turn with a visually recognizable time difference from a first position determined for the search result image on the screen of the touch panel, and controlling the search result images to be displayed as animations as if they were floating in the virtual space; Information and communication terminal device.

2. a recognition unit that performs a first recognition process based on line drawing data handwritten on the screen of the touch panel; the search query acquisition unit acquires the text recognized by the recognition unit as the search query; 2. The information communication terminal device according to claim 1.

3. the recognition unit starts execution of the first recognition process in response to a first operation action by a user on the line drawing data on the screen of the touch panel.

3. The information communication terminal device according to claim 2.

4. the first operation action is drawing a line that surrounds at least a part of an area indicated by the line drawing data on the screen of the touch panel; 4. The information communication terminal device according to claim 3.

5. the recognition unit is configured to perform a second recognition process on the image data to generate object information; the search query acquisition unit acquires the search query based on the generated object information.

3. The information communication terminal device according to claim 2.

6. The animation display is at least one of a manner in which the search result image is visually gradually completed, a manner in which the search result image is visually represented in perspective, and a manner in which the search result image is visually emphasized.

2. The information communication terminal device according to claim 1.

7. the display control unit determines the first position within the screen for each of the plurality of search result images, and controls the display of each of the search result images at the determined first position.

2. The information communication terminal device according to claim 1.

8. the first locations for each of the search result images are not geometrically aligned with one another; 8. The information communication terminal device according to claim 7.

9. the display control unit randomly determines the first position for each of the search result images; 8. The information communication terminal device according to claim 7.

10. the display control unit controls the animation display so that the displayed search result image gradually disappears after a predetermined time has elapsed.

2. The information communication terminal device according to claim 1.

11. the display control unit controls, in response to a second operation action by a user on at least one featured search result image among the plurality of search result images on the screen of the touch panel, to move the at least one featured search result image to a second position on the screen.

2. The information communication terminal device according to claim 1.

12. the search query acquisition unit acquires a new search query based on metadata associated with the at least one featured search result image in response to a second operation action by the user on the at least one featured search result image among the plurality of search result images on the screen of the touch panel; 2. The information communication terminal device according to claim 1.

13. the display control unit controls, in response to a second operation action by a user on at least one featured search result image among the plurality of search result images on the screen on the touch panel, to highlight the at least one featured search result image in order to prompt the user to confirm whether or not to perform a search on the at least one featured search result image.

2. The information communication terminal device according to claim 1.

14. the search query acquisition unit acquires a new search query based on metadata associated with the at least one featured search result image when the user selects to perform a search; 14. The information communication terminal device according to claim 13.

15. the display control unit controls the animation display to gradually increase the speed at which each of the plurality of search result images appears in response to the second operation action; The information communication terminal device according to claim 11.

16. the display control unit controls, in response to a third operation action by a user on at least one non-interest search result image among the plurality of search result images on the screen of the touch panel, to erase the at least one non-interest search result image from the screen.

2. The information communication terminal device according to claim 1.

17. the search query acquisition unit acquires, in response to the third operation action, a new search query based on metadata associated with some of the search result images excluding the at least one non-interest search result image among the plurality of search result images; 17. The information communication terminal device according to claim 16.

18. A method for displaying search results by an information communication terminal device communicably connected to a search server, comprising: acquiring a search query from a user's input on a touch panel of the information communication terminal device; obtaining a search result including a plurality of search result images transmitted from the search server in response to the search query; performing control to display each of the plurality of search result images included in the search result on a screen representing a virtual space on a touch panel of the information communication terminal device; The controlling includes controlling the display of each of the plurality of search result images in turn with a visually recognizable time difference from a first position determined for the search result image on the screen of the touch panel, and displaying the search result image in an animated manner as if it were floating in the virtual space. How search results are displayed.

19. A computer program for causing an information communication terminal device to realize a method for displaying search results transmitted from a search server in response to a search query, the computer program comprising: The method comprises: acquiring the search query from a user's input to a touch panel of the information communication terminal device; obtaining a search result including a plurality of search result images transmitted from the search server in response to the search query; performing display control to display each of the plurality of search result images included in the search result on a screen that represents a virtual space on the touch panel, The display control includes controlling the display so that each of the plurality of search result images appears in turn with a visually recognizable time difference from a first position determined for the search result image on the screen of the touch panel, and the search result image is displayed as an animation as if floating in the virtual space. Computer program.

Images