PROGRAM, INFORMATION PROCESSING APPARATUS, AND INFORMATION PROCESSING METHOD

Abstract

Provided is technology relating to enabling easier recognition or memory of presented information. A program according to the present invention causes a computer to function as a control unit that controls a device to present a sensation based on the five senses.

Description

[Technical field]

[0001] The present invention relates to a program, an information processing device, and an information processing method. [Background technology]

[0002] 2. Description of the Related Art There are known systems or devices that present some information to a user. Patent Document 1 discloses a navigation system that provides route guidance by presenting route information to a user. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-133366 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the prior art, a user may have difficulty perceiving or remembering information presented by a system or device.

[0005] One object of the present invention is to provide techniques relating to enabling easier perception or memorization of presented information. [Means for solving the problem]

[0006] A program according to one aspect of the present invention causes a computer to function as a control unit that controls a device to present sensations based on the five senses.

[0007] An information processing device according to one aspect of the present invention includes a control unit that controls a device to present sensations based on the five senses.

[0008] An information processing method according to one aspect of the present invention includes a control unit controlling a device to present sensations based on the five senses. Effect of the Invention

[0009] According to the present invention, it is possible to provide a technique related to enabling easier recognition or memorization of presented information. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration according to an embodiment. [Diagram 2] 1 is a diagram illustrating an example of a system configuration for locating the position of a terminal device in one embodiment. [Diagram 3] 1 is a diagram illustrating an example of a system configuration for locating the position of a terminal device in one embodiment. [Figure 4] FIG. 2 is a diagram for illustrating an example of a processing flow of the system in one embodiment. [Diagram 5] FIG. 2 is a conceptual diagram for explaining a five-sense device according to an embodiment. [Figure 6] FIG. 1 is a conceptual diagram for explaining the sensation provided by the system in one embodiment. [Figure 7] A conceptual diagram illustrating the situation detected by the system in one embodiment and the characteristics of the character expression presented. [Figure 8] FIG. 2 is a diagram for explaining the relationship between a system and multiple users in one embodiment. [Figure 9] FIG. 2 is a diagram for explaining the relationship between a system and multiple users in one embodiment. [Figure 10] FIG. 2 is a conceptual diagram illustrating a hardware configuration of a computer used by the system in one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An embodiment of the present invention will be described below. Note that the following embodiment is an example for explaining the present invention, and is not intended to limit the present invention to the embodiment. In addition, the present invention can be modified in various ways without departing from the gist of the invention. Furthermore, a person skilled in the art can adopt an embodiment in which each element described below is replaced with an equivalent, and such an embodiment is also included in the scope of the present invention.

[0012] An exemplary configuration of a system 1 according to an embodiment will be described with reference to Fig. 1. In this embodiment, the system 1 provides various experiences to a user by presenting various sensations based on the five senses (i.e., sight, smell, taste, hearing, and touch). In addition to the five senses, the system 1 may present sensations based on brain sensation (for example, transmitting information via a BMI (Brain Machine Interface)).

[0013] The system 1 mainly comprises a cloud system 10, a terminal device 20, a five-sense device 30, an in-facility positioning system 40, and an other system / map DB 50. Each of the above components of the system 1 is configured to be able to communicate (or access) with each other. In this embodiment, each of the above components (or functions or processes of the system 1 described below) is realized by the cooperation of one or more information processing devices (or computers; the same applies below) and software (programs). Furthermore, among the above components of the system 1, the functions of multiple components may be realized on one device. For example, the functions of at least two of the components of the cloud system 10, the terminal device 20, and the five-sense device 30 may be realized on one device.

[0014] Although each of the components included in the system 1 shown in Fig. 1 is shown one by one, at least some of the components may be included in multiples. For example, the system 1 may include multiple terminal devices 20 or five-sense devices 30. In addition, the system 1 may not include some of the components shown in Fig. 1, and may include components not shown in Fig. 1.

[0015] The cloud system 10 executes processes for implementing various functions and providing various services to the user. For example, the cloud system 10 transmits a control command to the terminal device 20 by a control unit (not shown) for controlling the five-sense device 30 so as to present sensations based on the five senses to the user. That is, according to the present embodiment, the system 1 does not present only visual information or audio information to the user, but presents sensations based on the five senses, so that the user can more easily recognize or remember the information presented by the system 1. In the following description, the term "sensations based on the five senses" does not necessarily mean sensations based on all of the five senses, but means sensations based on all or a part of the five senses.

[0016] Furthermore, the cloud system 10 may transmit, to the terminal device 20, a control command for controlling the five-sense device 30 to present a character representation that represents a predetermined character by at least a partial combination of the senses based on the five senses, by the control unit. For example, the five-sense device 30 presents at least a partial combination of the image, smell, taste, sound, and touch (skin sensation) of the character A in order to present the character representation of the character A in response to the control command. For example, if the character A is a person, the five-sense device 30 can present a human-like appearance by presenting a human image, a human smell, a human taste, a human voice, and the sensation of a person touching the user's body. Note that, since taste is often not confirmed when recognizing a person, the taste of a person does not need to be presented. Similarly, in the following description, the cloud system 10 does not need to present taste to represent a character.

[0017] Thus, according to this embodiment, the system 1 presents a character representation that represents a predetermined character by a combination of at least some of the senses based on the five senses. Therefore, since the system 1 presents a character representation rather than simply presenting only visual information or audio information to the user, for example, the information presented by the system 1 is more likely to be memorable and the user can more easily recognize or remember the information.

[0018] The above-mentioned predetermined characters may include real characters or fictional characters. Real characters include living or deceased human or animal characters. Real characters may be, for example, the user's family members (e.g., father, mother, siblings, children, grandfather, or grandmother) or the user's pets (e.g., dogs or cats). Fictional characters include, for example, characters created by humans or artificial intelligence technology.

[0019] The real-life character may be a character created based on a real person or animal. For example, a character may be created using artificial intelligence technology based on the characteristics of a real person or animal. Also, a character may be created based on information read by a system that reads information related to the five senses of a person using a sound recording system, a camera system, an odor detection system, or other sensors.

[0020] Furthermore, in this embodiment, the cloud system 10 may transmit, to the terminal device 20, a control command for controlling the five-sense device 30 to present a guiding expression associated with a direction by a predetermined character, by the control unit. For example, the five-sense device 30 presents a guiding expression associated with a direction, such as forward, backward, right, or left, by the character expression of character A in response to the control command. The guiding expression associated with a direction is presented using sensations based on the five senses. Below, the guiding expression associated with a direction will be described for each of the five senses.

[0021] Taste: Guided by taste A sense of direction (or orientation, the same applies below) is associated with each flavor in advance. For example, directional meanings are assigned to flavors such as ethnic, Eskimo, coconut, milk, Asian, European, American, and African. The five-sense device 30 can associate the direction of the flavor with its meaning by presenting the flavor to the user, and guide the user.

[0022] Olfactory: Guidance by smell The five-sense device 30 associates a sense of direction with an odor in advance, and by presenting an odor to the user, the five-sense device 30 can associate a direction with a meaning assigned to the odor and guide the user.

[0023] Touch: Guide by touch The user is guided by the sense of direction presented by touch. The five-sense device 30 may guide the user by, for example, presenting vibration or tactile sensation to the user, thereby pushing or pulling the user. The five-sense device 30 may also present vibration or tactile sensation such as tapping to a part of the user's body that corresponds to the direction to be guided. The user is guided in a direction corresponding to the part of the body where the vibration or tactile sensation is felt (for example, right for the right shoulder, right arm, or right foot; left for the left shoulder, left arm, or left foot; front for the stomach; back for the back or waist; a direction corresponding to the position of the fingers or hands).

[0024] Vision: Visual guidance The five-sense device 30 can guide the user in a direction that is associated with the meaning of the presented image. The five-sense device 30 can guide the user by presenting directions such as right, left, forward, backward, and the like and things that are visually associated with the directions (for example, visual expressions such as characters or images that evoke the directions).

[0025] Hearing: Guided by sound The five-sense device 30 guides the user in a direction that is given meaning to the sound presented. For example, a sound heard by the ears is given meaning (e.g., words), and the five-sense device 30 can guide the user by the sound. The five-sense device 30 can also guide the user in a direction that can be associated with a voice or a type of sound that is perceived, or in a direction that can be directly grasped.

[0026] The guiding expression includes an expression that guides the user to perform an action including moving forward or turning in a predetermined direction, or stopping. The five-sense device 30 can change or adjust the speed of the guiding action by changing the force pulling the user, for example.

[0027] The five-sense device 30 can adjust the angle of turning and guide the user, for example, by adjusting the angle of pulling. The five-sense device 30 can present the user with a sensation to warn the user of the turning before guiding the user to turn. The five-sense device 30 can present the user with a sensation of pulling backwards, for example, to guide the user who is moving (for example, walking) to stop or to proceed slowly. In this case, the pulling force in guiding the user to stop may be stronger than the pulling force in guiding the user to proceed slowly. The five-sense device 30 can present the user with a sensation of pulling diagonally upwards to guide the user to climb (for example, stairs). The five-sense device 30 can present the user with a sensation of pulling diagonally downwards to guide the user to descend (for example, stairs).

[0028] In addition to the above, the five-sense device 30 can present voices, sounds, smells, tastes, rhythmic changes to the body, letters, and the like, to guide the user according to the meaning that the user grasps.

[0029] If there is a dangerous situation, the five-sense device 30 may notify the user of the danger (guiding the user with light, sound, voice, body, smell, taste, etc.). Also, if the situation is safe, the five-sense device 30 may notify the user of safety (guiding the user with light, sound, voice, body, smell, taste, etc.).

[0030] Returning to the explanation of Figure 1, the cloud system 10 has the following functional configurations: "user authentication," "scenario," "voice UI" (voice user interface), "back-end system," "database," "AI" (artificial intelligence), and "front-end system (app)."

[0031] "User authentication" identifies (authenticates) a user to a system or application.

[0032] A "scenario" has a process (algorithm) that specifies the output for a series of input data.

[0033] The "voice UI" understands what a user says when speaking to a device (for example, the five-sense device 30) and operates according to the user's commands.

[0034] The "back-end system" installs software or interfaces with a database.

[0035] A "database" manages data on a computer.

[0036] "AI" uses artificial intelligence techniques such as machine learning or deep learning to generate output based on information (or features) detected from video, context, or user behavior. In other words, it learns from experience and adapts to new inputs to flexibly perform tasks like a human would. By recognizing patterns from large amounts of data, computers can be trained to handle a variety of difficult tasks in business and life.

[0037] A "front-end system (application)" processes the parts of a web service or web application that are directly visible to the user (user interface). A "front-end system (application)" accepts characters entered by users on websites and web applications, processes the display of buttons that users click, and also processes to link with back-end systems.

[0038] The terminal device 20 performs processing (for example, control of the five-sense device 30) according to a control command or data received from the outside (for example, a control command received from the cloud system 10) or a user operation. The terminal device 20 has, for example, a "five-sense app" as an application for executing the above processing.

[0039] The five-sense device 30 presents sensations based on the five senses. For example, the five-sense device 30 presents, for example, vision, smell, taste, hearing, and touch based on a control signal received from the terminal device 20. Presentation of vision includes, for example, presenting a stimulus that the user recognizes as an image. Presentation of smell includes, for example, presenting a stimulus that the user recognizes as an odor. Presentation of taste includes, for example, presenting a stimulus that the user recognizes as a taste. Presentation of hearing includes, for example, presenting a stimulus that the user recognizes as a sound. Presentation of touch includes, for example, presenting a stimulus that the user recognizes as a touch.

[0040] The five-sense device 30 is composed of one or more devices. In the five-sense device 30 of this embodiment, a component that presents at least some of the senses of sight, smell, taste, hearing, and touch may be referred to as an actuator. The five-sense device 30 may be installed, for example, in a position where it is possible to present sensations based on the five senses to the user. The five-sense device 30 is configured, for example, so that the user can wear it on or hold it on their body. As shown in FIG. 1, the five-sense device 30 includes an "actuator," a "sensor," and a "communication unit."

[0041] The five-sensory device 30 may include, for example, but is not limited to, at least a part of a display, an aroma shooter, a food texture presentation device, a speaker, or a vibration device as an "actuator."

[0042] In addition, the five senses device 30 may be equipped with at least some of the following "sensors" that detect the surrounding environment or the state of the device itself, including, but not limited to, an ultrasonic sensor, an infrared sensor, a laser sensor, a radar, a light sensor, a sound sensor (microphone), an image sensor (camera), a pressure sensor, an air pressure sensor, a temperature sensor, a humidity sensor, an acceleration sensor, an angular velocity sensor, an altitude sensor, a geomagnetic sensor, or a human presence sensor.

[0043] The in-facility positioning system 40 is a system for locating the position of the terminal device 20 when the terminal device 20 is located inside a facility (indoors). The in-facility positioning system 40 includes a "sensor" and a "communication unit". When the terminal device 20 is located outdoors, the position of the terminal device 20 is located by a Global Navigation Satellite System (GNSS) such as a Global Positioning System (GPS) or any other method.

[0044] The other system / map DB 50 includes, for example, an external system, an external sensor, and a DB (database server) of an outdoor map and an indoor map. The external sensor may have a similar configuration to the sensor of the five-sense device 30.

[0045] 2 and 3, the environmental configuration (particularly the configuration for locating the position of the terminal device 20) when the terminal device 20 is located outdoors and indoors will be described. FIG. 2 shows an exemplary environmental configuration when the terminal device 20 is located outdoors. The terminal device 20 performs a positioning process for identifying its own position, for example, based on a GPS signal received from a GPS satellite. The terminal device 20 may transmit, for example, data based on the identified own position and data received from the other system / map DB 50 via the Internet to the five-sense device 30.

[0046] 3 shows an exemplary environmental configuration when the terminal device 20 is located indoors. The position of the terminal device 20 is identified by a positioning process based on, for example, a signal (including, for example, address information of the access point) received from an access point (for example, a WiFi access point) for communication with the terminal device 20 and other access point position information stored in the system / map DB 50 or the like. Even when the position of the terminal device 20 is indoors, the position may be identified by a positioning process based on a GPS signal received from a GPS satellite. The identified position information of the terminal device 20 may be transmitted to the five-sense device 30.

[0047] An example of a process flow from the occurrence of a trigger to the presentation of sensations based on the five senses to the user in the system 1 will be described with reference to Fig. 4. Fig. 4 exemplarily shows a first flow indicated by a solid arrow and a second flow indicated by a dotted arrow. The first flow is a flow triggered by a detection result by an external sensor of the other system / map DB 50. The second flow is a flow triggered by a detection result by a sensor of the five sense device 30.

[0048] In either flow, the control unit (not shown) of the cloud system 10 or the terminal device 20 controls the five-sense device 30 to present sensations based on the five senses, for example, according to a user setting or a detection result by a sensor. The control process is realized, for example, by the control unit of the cloud system 10 or the terminal device 20 executing a program stored in a storage unit of the cloud system 10 or the terminal device 20, etc., and controlling the operation of the hardware of the cloud system 10 or the terminal device 20 (i.e., by cooperation between software and hardware).

[0049] As a prerequisite for starting the first flow and the second flow, various settings (user settings) are registered in the terminal device 20 in response to user operations, etc., and information on the settings is further transmitted to the cloud system 10. The settings include, for example, settings related to trigger conditions and settings related to sensations based on the five senses presented by the five-sense device 30. Each flow will be described below.

[0050] In the first flow, first, when a detection result by an external sensor of the other system / map DB 50 satisfies a predetermined trigger condition, a notification is sent to the cloud system 10 (step S101). Next, the cloud system 10 sends an instruction according to the satisfied trigger condition to the terminal device 20 (step S102).

[0051] Next, the terminal device 20 transmits a control signal corresponding to the instruction received from the cloud system 10 to the five-sense device 30 (step S103). Next, the five-sense device 30 controls the actuator based on the received control signal (step S104) to present sensations based on the five senses to the user (step S105).

[0052] In the second flow, first, when the detection result by the sensor of the five-sense device 30 satisfies a predetermined trigger condition, a notification is transmitted to the actuator of the five-sense device 30 (step S201). After that, the five-sense device 30 transmits an instruction according to the satisfied trigger condition to the terminal device 20 (step S202).

[0053] Next, the terminal device 20 transmits a control signal to the cloud system 10 in response to the instruction received from the five-sense device 30 (step S203). Next, the cloud system 10 transmits a control signal to the terminal device 20 for controlling the five-sense device 30 in response to the trigger condition being satisfied based on the received control signal (step S204). Next, the terminal device 20 transmits a control signal to the five-sense device 30 in response to the received control signal (step S205). Next, the five-sense device 30 presents sensations based on the five senses to the user (step S207) by controlling the actuator based on the control signal received from the terminal device 20 (step S206).

[0054] With reference to FIG. 5, the configuration (device or system; the same applies below) included in the five-sense device 30 for presenting each of the sensations based on the five senses will be described below.

[0055] Visual: A system using a device that reproduces what can be seen. The sensory device 30 may, depending on the configuration, include a system that communicates using direct electrical signals into the user's brain, or a device that presents visuals as seen by the eye (e.g., makes a picture or physical object visible), which may change the field of view or the way the eye sees by sending signals electronically, either remotely or directly, or may otherwise present or change the visual experience in a picture or physical representation in response to receiving external signals.

[0056] Olfaction: A system using devices that can reproduce the sense of smell The sensory device 30 may, depending on the configuration, include a system that communicates using direct electrical signals into the brain, or a device that generates odors to be sensed as odors in the nose, which may present the user with an olfactory sensation by sending signals electronically, either remotely or directly, or may otherwise generate or change odors in response to receiving signals.

[0057] Gustatory: A system using a device that can reproduce the sensation of taste. The sensory device 30 may include devices that communicate using direct electrical signals into the brain, or devices that electronically transmit signals remotely or directly into the mouth so that specific tastes can be extracted in the mouth by placing the device in the mouth, altering the amount and movement of saliva and chewing in the mouth, or otherwise extracting and altering tastes by receiving signals according to the configuration.

[0058] Hearing: A system using devices that can reproduce sounds so that they can be felt. The sensory device 30 may include a system that communicates using direct electrical signals into the brain, or a device that generates sounds to be felt as sounds, depending on the configuration, that can send signals electronically, remotely or directly, to change the person's perception of how they hear, or that can otherwise generate and change sounds by receiving signals depending on the configuration.

[0059] Haptic: A system that uses devices that can reproduce the sensation of touch. The sensory device 30 may include a device that transmits electrical signals directly into the brain, or a device that transmits signals electronically to the user's body in a remote or direct manner, so that a particular touch sensation is generated by a device that touches the user's body, depending on the settings. The device may change the area or movement of the body's touch, or otherwise generate or change a physical sensation in response to the signal received, depending on the settings.

[0060] As described above, the five-sensory device 30 is capable of transmitting sensations using direct electrical signals into the brain, that is, presenting sensations based on cerebral sensation (for example, transmitting information by BMI).

[0061] FIG. 6 conceptually illustrates sensations presented to the user by the system 1. When a trigger condition is satisfied, the system 1 presents a sensation based on the five senses to the user through the five-sense device 30. The sensation may be virtual or real. When a trigger condition is satisfied, a sensation based on the five senses may be presented to, for example, a plurality of users (the user and other people) who hold the five-sense device 30. The trigger condition may be based on detection (sensing) by a sensor, or may be a condition according to a system setting (for example, a condition based on the passage of time or a user operation).

[0062] 7, a relationship between the surrounding environment or the user's situation detected by a sensor in the system 1 and the characteristics (amount of expression) of the character representation presented by the system 1 will be described. In this embodiment, the characteristics of the character representation presented by the five-sense device 30 may be changed according to the surrounding environment or the user's situation (i.e., the detection result by the sensor) detected by the sensor of the five-sense device 30 (or an external sensor of another system / map DB 50) through processing by the control unit of the cloud system 10 or the terminal device 20.

[0063] The characteristics of a character's expression (amount of expression) are the nature of the expression that the character presents to the user. For example, when the characteristics of the character's expression are sensory, strong, and fast, it is assumed that the sensor detects that the character is not gaining the user's sympathy (the detection method will be described later). In this case, the characteristics of the character's expression may be changed to logical, weak, or slow in order to gain the user's sympathy.

[0064] As shown in FIG. 7, the amount of expression of a character expression includes sensory or logical, strong or weak, fast (early) or slow, regular or rhythmic or irregular, emotional expression, and short or long (prolonged).

[0065] Sensory or logical means, for example, the presentation of an expression that stimulates the right brain of the user more (right brain approach) or the presentation of an expression that stimulates the left brain more (left brain approach). The right brain approach is an approach where the user feels and communicates to the user, and the left brain approach is an approach where the user thinks. For example, in the right brain approach, communication is from the left side of the user, or communication is first (emotions such as touch and vision are also communicated through the five-sense device 30). In the left brain approach, communication is from the right side of the user, for example. As another example of the left brain approach, suppose an alert has occurred. When an emergency signal sound is sounded, a touch sensation explaining the situation is given to the right hand (intensity of stimulus 3). Or, a cold stimulus is given to the left hand (intensity of stimulus 2 = zone where there is a risk of drowning in the sea or river).

[0066] As shown in FIG. 7, the surrounding environment or user's situation detected by the sensor includes smell, balance, somatic sensation, sound, presence, visceral sensation, light, voice, empathy, signals, letters, sensitivity / intelligence, air pressure, sensory vocabulary, memory, materials, emotions, and 1 / f fluctuation.

[0067] The presence status is detected, for example, by a human sensor, a temperature sensor, or an ultrasonic sensor. The empathy status means, for example, the empathy of the user with respect to the presented character representation, and the presence or absence of empathy is detected, for example, according to the facial expression, voice, or behavior of the user detected by an image sensor or a sound sensor (for example, when happiness, fun, etc. progresses and a sense of rapport is formed with each other, it becomes a barometer of empathy). The emotion and sensibility status is detected, for example, according to the facial expression, voice, or behavior of the user detected by an image sensor or a sound sensor. The memory status includes whether the user has memorized something, and is detected by analyzing the user's brain waves (for example, analyzing the movement of the signal as memory for the hippocampus). The 1 / f fluctuation status includes, for example, the 1 / f fluctuation of the surrounding environment or the user detected by an image sensor. The 1 / f fluctuation status also includes the detection result of a sensor such as ultrasonic waves, whether it is within the range of an algorithm as a natural reaction in a system that detects fluctuations.

[0068] Furthermore, a signal means a digital signal of 0 or 1. It is assumed to be some kind of communication access. Materials include rough, smooth, etc. Materials have the same meaning as sensory vocabulary, and are recognized as materials and converted, being a higher-level expression of sensory vocabulary. For example, if something is rough, it is detected as asphalt, and if it is smooth, it is detected as glass. Characters are recognized by OCR (Optical Character Recognition) or an image sensor.

[0069] In this embodiment, the five-sense device 30 may be controlled to change the emotion (emotional expression) expressed by the character expression according to the detection result by the sensor through processing by the control unit of the cloud system 10 or the terminal device 20. The emotional expressions include, for example, joy, like, sadness, fear, anxiety, surprise, dislike, anger, boredom, regret, hurt, moved, excited, tension, excited, upset, and relief. The change in emotion includes a change in the character expression due to a change in the visual expression, auditory expression, tactile expression, and the like of the character. For example, in response to the detection of the user's emotion of joy in the system 1, the character expression presented by the five-sense device 30 may change from anxiety to joy.

[0070] The relationship between multiple users (e.g., oneself and others) of the system 1 in this embodiment will be described with reference to Fig. 8 and Fig. 9. As shown in Fig. 8, a certain user (oneself) performs operations related to other users (others) via the system 1 (terminal device 20) and is operated by other users via the system 1 (terminal device 20).

[0071] FIG. 9 shows a pattern of destinations when the system 1 guides a user to a destination. In this embodiment, the cloud system 10 or the control unit of the terminal device 20 can control the five-sense device 30 to present a guidance expression to a user (the user) in response to an operation (for example, a destination setting operation) on the terminal device 20 by a certain user (the user). The guidance expression is as described above. That is, the system 1 functions, for example, like a car navigation system. The system 1 also accepts and registers the setting of a destination for sensory guidance via the terminal device 20, and functions as an automatic guidance system, an automatic detour system, a return direction guidance system, a turn point notification system, or a point passing system (linked with the communication environment and nearby beacons).

[0072] Furthermore, the cloud system 10 or the control unit of the terminal device 20 can control the five-sense device 30 to present a guiding expression to another user (other person) in response to an operation (for example, a destination setting operation or a guiding operation) performed by a certain user (yourself) on the terminal device 20. The guiding expression can guide the other person to the destination. In this case, for example, the user holds the terminal device 20 and the other person holds the five-sense device 30.

[0073] For example, in the example shown in FIG. 9, the user can set a place he or she wants to go to (e.g., Tokyo Station) as destination 1 via his or her terminal device 20 (referred to as terminal device 20a). In addition, the user can set a place he or she wants another person (e.g., his or her child) to go to (e.g., a school) as destination 2 via terminal device 20a. In addition, the user can set a meeting place between the user and the other person as destination 3 via terminal device 20a. In addition, the other person can set the other person's destination (destination 2 or destination 3) via the other person's terminal device 20 (referred to as terminal device 20b).

[0074] The system 1 automatically sets a route to the set destination using the cloud system 10 or the terminal device 20, and controls the user's own five-sense device 30 (referred to as five-sense device 30a) or another person's five-sense device 30 (referred to as five-sense device 30b) to present sensations based on the five senses, thereby guiding the user or another person to the destination using the above-mentioned character expressions and guiding expressions.

[0075] Furthermore, the user can perform an operation on the terminal device 20a to guide the other person, and can use the character representation and the guidance representation to guide the other person to the destination 2 or the destination 3 via the five-sense device 30b. In this case, the character representation and the guidance representation may correspond to the user's own character.

[0076] In this embodiment, the system 1 may operate to build rapport with the user. In order to build rapport, the system 1 adjusts the character expression or the guidance expression presented by the five-sensory device 30b so that the expression is compatible with the user.

[0077] In order to match the compatibility, first, it is determined whether the character expression or the guiding expression is compatible with the user. For example, the control unit of the cloud system 10 or the terminal device 20 may determine the degree of match between the user's behavior and the guiding expression based on the detection result of the user's behavior by the sensor. For example, if the guiding expression presented by the five-sense device 30 indicates the right, while the user's behavior detected by the sensor is moving to the left, the degree of match may be determined to be low (low score). On the other hand, if the guiding expression presented by the five-sense device 30 indicates the right, and the user's behavior detected by the sensor is moving to the right, the degree of match may be determined to be high (high score).

[0078] The control unit of the cloud system 10 or the terminal device 20 may control the five-sense device 30 to change the characteristics of the guiding expression according to the degree of matching determined as described above. The characteristics of the guiding expression are the same as the characteristics (amount of expression) of the character expression described with reference to FIG. 7. That is, the characteristics of the guiding expression may include, for example, sensory or logical, strong or weak, fast (early) or slow, regular, rhythmic or irregular, emotional expression, and short or long (long). For example, when the characteristics of the guiding expression are sensory, strong, and fast, the sensor determines the degree of matching as a low score. In this case, adjustments are made such as changing the characteristics of the guiding expression to logical, weak, or slow so that the score of the degree of matching becomes higher.

[0079] Furthermore, the control unit of the cloud system 10 or the terminal device 20 may control the five-sense device 30 to perform an action that matches the user's action based on the theory of paging so as to increase the score of the degree of coincidence. For example, the five-sense device 30 may be controlled to perform fast guidance when the user's action is fast, slow guidance when the user's action is slow, and fast guidance when there is no time (for example, when there is no time until the time when the user wants to arrive at the destination set by the user). Furthermore, when considering whether to hurry or not, the five-sense device 30 may be controlled to wait quietly.

[0080] Furthermore, based on the theory of mirroring for rapport building, for example, if the user turns right despite the system 1 guiding the user to turn left, the system 1 may also guide the user to turn right. In this case, if there is a dangerous situation or the like caused by the user turning right, the system 1 may immediately notify the user of the situation via the five-sense device 30. As a result, rapport can be built.

[0081] Furthermore, in order to build rapport, the system 1 may perform matching. For example, the system 1 may set the tone of the voice presented by the five-sense device 30 in accordance with the tone (high / low) of the user's voice detected by the sensor. The system 1 may set the tempo of the voice presented by the five-sense device 30 in accordance with the tempo (fast / slow) of the user's voice detected by the sensor. The system 1 may set the volume of the voice presented by the five-sense device 30 in accordance with the volume (loud / soft) of the user's voice detected by the sensor. The system 1 may set the rhythm of the voice presented by the five-sense device 30 in accordance with the rhythm (e.g., rhythmic / monotonous) of the user's voice detected by the sensor.

[0082] The system 1 may control the five-sense device 30 to present an expression of an emotion corresponding to the destination to which the user is guided. Specifically, the control unit of the cloud system 10 or the terminal device 20 may control the five-sense device 30 to present a character expression corresponding to the destination based on the detection that the user has reached the destination in response to the guiding expression by the five-sense device 30. For example, if the destination is a happy place for the user, a happy expression may be presented, and if it is a sad place, a sad expression may be presented. In addition, if an input corresponding to an emotion expressing gratitude is made by the user to the terminal device 20, or if an emotion of gratitude is detected by a sensor, the five-sense device 30 may be controlled to present an expression of an emotion (e.g., joy) corresponding to the gratitude by the user.

[0083] The system 1 may associate character information corresponding to the character representation presented by the five-sense device 30 (e.g., after arriving at the destination) with the user's emotions (e.g., fun or not fun) detected by the sensor, and store this in a memory unit as the user's experience.

[0084] In the system 1, the control unit of the cloud system 10 or the terminal device 20 may store emotion information corresponding to a predetermined action, in response to a sensor detecting a predetermined action by a user, in association with a position where the predetermined action was detected. For example, if an emotion of joy is associated with an action of raising both hands, emotion information indicating joy may be stored in a storage unit in the system 1 in association with a position where the action of raising both hands was detected, in response to a sensor detecting the action of a user raising both hands.

[0085] That is, according to this embodiment, for a location to which the user is guided by the system 1 (e.g., a destination or a location passed through), emotional information corresponding to the user's body movements at that location can be stored (i.e., the user's emotional information can be tagged with the space). The emotional information stored may be information using all or some of the five senses. The user can check the tagged emotional information via the cloud system 10 or the terminal device 20. By checking the tagged emotional information via the cloud system 10 or the terminal device 20, the user can more easily recognize and recall memories.

[0086] As described above, according to this embodiment, the relationship with the device in the three-dimensional space is replaced by a direct relationship with a specific person or character, and the person can build a rapport with the device. As a result, a story that is more recognizable and memorable is created, and the guidance to a destination using physical sensations (for example, guidance from the current location to another location, from another location to another location, or from another location to the current location) is given emotional meaning and a human relationship is created, making it easier to reproduce recognition and memories.

[0087] <Computer hardware configuration> Next, an exemplary hardware configuration of a computer (information processing device) for implementing the cloud system 10 and the terminal device 20 in this embodiment will be described with reference to FIG.

[0088] 10, the computer 700 mainly includes a processor 701, a memory 703, a storage device 705, an operation unit 707, an input unit 709, a communication unit 711, and an output unit 713. The computer 700 does not need to include at least some of these components. Furthermore, the computer 700 may include other components that are generally included in a general-purpose computer or a dedicated computer, in addition to these components.

[0089] The processor 701 is a control unit that executes a program stored in the memory 703 to control various processes in the computer 700. Therefore, the processor 701 realizes the functions of each component described in the above embodiment in cooperation with other components of the computer 700 and the program, and controls the execution of the above processes.

[0090] The memory 703 is a storage medium such as a RAM (Random Access Memory), etc. In the memory 703, the program code of the program executed by the processor 701 and data required for executing the program are temporarily read from the storage device 705 or the like, or are stored in advance.

[0091] The storage device 705 is a non-volatile storage medium such as a hard disk drive (HDD), etc. The storage device 705 stores an operating system, various programs for implementing the above-mentioned configurations, data of the above-mentioned processing results, and the like.

[0092] The operation unit 707 is a device for receiving input from a user. Specific examples of the operation unit 707 include a keyboard, a mouse, a touch panel, a joystick, a button key, various sensors, a wearable device, etc. The operation unit 707 may be removably connected to the computer 700 via an interface such as a USB (Universal Serial Bus).

[0093] The input unit 709 is a device for inputting data from outside the computer 700. Specific examples of the input unit 709 include a drive device for reading data stored in various storage media. The input unit 709 also includes a microphone for collecting surrounding sounds, converting the sounds into audio data, and inputting the audio data. The input unit 709 may be removably connected to the computer 700. In this case, the input unit 709 is connected to the computer 700 via an interface such as a USB.

[0094] The communication unit 711 is a device for performing data communication via a network, either wired or wirelessly, with a device external to the computer 700. The communication unit 711 may be removably connected to the computer 700. In that case, the communication unit 711 is connected to the computer 700 via an interface such as a USB.

[0095] The output unit 713 is a device that outputs various data. The output unit 713 is, for example, a display device for displaying data, or a speaker for outputting sound. Specific examples of the display device include a liquid crystal display, an organic EL display, and a display of a wearable device. The output unit 713 may be removably connected to the outside of the computer 700. In that case, the display device that is the output unit 713 is connected to the computer 700 via, for example, a display cable. In addition, when a touch panel is adopted as the operation unit 707, the output unit 713 can be configured integrally with the operation unit 707.

[0096] <Modification>

[0097] The program for implementing the system 1 (or the cloud system 10, or the terminal device 20) in this embodiment can be recorded on various recording media such as an optical disk such as a CD-ROM, a magnetic disk, a semiconductor memory, etc. Also, the program can be installed or loaded on a computer by downloading it through a recording medium or via a communication network, etc.

[0098] The present invention is not limited to the above-described embodiment, and can be embodied in various other forms without departing from the scope of the present invention. The above-described embodiment is merely illustrative in all respects and should not be construed as being restrictive. [Explanation of symbols]

[0099] 1 System 10. Cloud Systems 20 Terminal Equipment 30 Five Senses Devices 40 In-facility positioning system 50 Other systems / map DB

Claims

1. Computer, A program for functioning as a control unit for controlling a device to present an auditory sensation, The control unit is presenting the auditory sense in accordance with a user setting set in a terminal device for controlling the device in response to a user operation or a detection result by a sensor that detects a user's situation, and controlling the device so as to present a character expression that expresses a predetermined character based on the auditory sense as a human voice; Controlling the device to present a guiding expression associated with a direction by the predetermined character; determining a degree of agreement between the user's behavior and the prompting expression based on a detection result of the user's behavior by the sensor; According to the determined degree of coincidence, an amount of expression of the character expression is changed as a characteristic of the prompting expression, and the device is controlled to perform an action in accordance with the user's action so as to increase the degree of coincidence, thereby presenting the prompting expression to the auditory sense; The character representation presented by the device is presented to the user so as to build a rapport relationship between the character representation of the human voice presented by the device and the user, which is a direct relationship between the user and a specific person, and the user's cognition and memory are reproduced by guiding the user to a destination using a physical sensation based on a memorable story and giving meaning to the user's emotions and creating a relationship as a person, and In response to a predetermined action by the user being detected by the sensor, the program causes the device to store emotion information of the user corresponding to the predetermined action in association with a position where the predetermined action was detected.

2. The program according to claim 1 , wherein the predetermined character includes a real character or a fictional character.

3. The program according to claim 1 , wherein the control unit controls the device to change a characteristic of the character representation in response to a detection result by the sensor.

4. The program according to claim 1 , wherein the control unit controls the device to change the emotion expressed by the character representation in accordance with a result of detection by the sensor.

5. The computer-readable medium according to claim 1 , wherein the guidance expression includes an expression that guides the user to perform an action including moving forward, turning, or stopping.

6. The program according to claim 1 , wherein the control unit controls the device to present the guiding expression to a second user, who is a different person from the first user, in response to an operation of the terminal device by a first user, who is the user himself, and presents the guiding expression audibly.

7. The program according to claim 1 , wherein the control unit controls the device to present the character representation corresponding to the destination based on detection that the user has reached the destination in response to the guiding representation, and presents the character representation aurally.

8. An information processing device including a control unit that controls a device to present a sensation based on hearing, The control unit is presenting the auditory sense in accordance with a user setting set in a terminal device for controlling the device in response to a user operation or a detection result by a sensor that detects a user's situation, and controlling the device so as to present a character expression that expresses a predetermined character based on the auditory sense as a human voice; Controlling the device to present, by the predetermined character, a guidance expression that is an expression amount of the character expression associated with a direction; determining a degree of agreement between the user's behavior and the prompting expression based on a detection result of the user's behavior by the sensor; According to the determined degree of coincidence, an amount of expression of the character expression is changed as a characteristic of the prompting expression, and the device is controlled to perform an action in accordance with the user's action so as to increase the degree of coincidence, thereby presenting the prompting expression to the auditory sense; The character representation presented by the device is presented to the user so as to build a rapport relationship between the character representation of the human voice presented by the device and the user, which is a direct relationship between the user and a specific person, and the user's cognition and memory are reproduced by guiding the user to a destination using a physical sensation based on a memorable story and giving meaning to the user's emotions and creating a relationship as a person, and In response to a predetermined action by the user being detected by a sensor, the device stores emotion information of the user corresponding to the predetermined action, in association with a position where the predetermined action was detected.

9. An information processing method in an information processing device including a control unit that controls a device to present a sensation based on hearing, The control unit is presenting the auditory sense in accordance with a user setting set in a terminal device for controlling the device in response to a user operation or a detection result by a sensor that detects a user's situation, and controlling the device so as to present a character expression that expresses a predetermined character based on the auditory sense as a human voice; Controlling the device to present, by the predetermined character, a guidance expression that is an expression amount of the character expression associated with a direction; determining a degree of agreement between the user's behavior and the prompting expression based on a detection result of the user's behavior by the sensor; According to the determined degree of coincidence, an amount of expression of the character expression is changed as a characteristic of the prompting expression, and the device is controlled to perform an action in accordance with the user's action so as to increase the degree of coincidence, thereby presenting the prompting expression to the auditory sense; The character representation presented by the device is presented to the user so as to build a rapport relationship between the character representation of the human voice presented by the device and the user, which is a direct relationship between the user and a specific person, and the user's cognition and memory are reproduced by guiding the user to a destination using a physical sensation based on a memorable story and giving meaning to the user's emotions and creating a relationship as a person, and An information processing method, in which, in response to a predetermined action by the user being detected by a sensor, the device stores emotional information of the user corresponding to the predetermined action, associating the predetermined action with a position where the predetermined action was detected.

Images