Management system, management computer, user terminal, and application

Abstract

This system provides a management system for controlling audio information output from an audio generator when multiple target objects are used separately in multiple locations, a management computer for managing multiple target objects used by the user, a user terminal, and an application. [Solution] A management system 10 comprising a first target object 13 and a second target object 14, a user terminal 12, a speaker that outputs sound, a sound output device 43, and a management computer 11 that is communicatively connected to the user terminal and performs processing, wherein the management computer performs the following processes: storing the first and second target objects and the first and second usage locations in an auxiliary memory 49, determining which of the first and second usage locations is being used, determining the usage location of the used target object, and controlling audio information, including sound output from the speaker and sound output device, based on the determination results of the target object and usage location being used.

Description

Technical Field

[0001] The present disclosure relates to a management system having a plurality of objects used by a user, a management computer for managing a plurality of objects used by the user, a user terminal, and an application.

Background Art

[0002] An example of an audio generation program for controlling audio output from a plurality of robots is described in Patent Document 1. Patent Document 1 describes a robot including an audio generation unit that is used by a user and generates audio, and an audio output unit that outputs the generated audio. Further, when there are a plurality of robots in the home, the audio generation unit described in Patent Document 1 generates the audio of each robot so that the voices of the respective robots do not become the same.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] According to the audio generation program described in Patent Document 1, the present inventors have recognized that there is room for improvement in how to control the audio information output from the audio generation device when a plurality of target objects are separately used at a plurality of usage locations.

[0005] An object of the present disclosure is to provide a management system, a management computer, a user terminal, and an application that can appropriately control the audio information output from an audio generation device when a plurality of target objects are separately used at a plurality of usage locations.

Means for Solving the Problems

[0006] This embodiment discloses a management system comprising: an object having a configuration that allows interaction with a user; a user terminal used by the user and connected to the object in a communicative manner; an audio output device that outputs audio corresponding to the user's interaction status with the object; and a computer connected to the user terminal in a communicative manner, which stores information and performs processing, wherein the object includes a plurality of object objects configured to be used in a plurality of different locations, and the computer is configured to perform the following processes: storing the configuration of each of the plurality of object objects and the respective locations suitable for each of the plurality of object objects in memory; determining which of the plurality of object objects is being used; determining which location the used object is located in; and controlling audio information, including audio output from the audio output device, based on the determination result of the object being used and the determination result of the location where the object being used is located. [Effects of the Invention]

[0007] In the management system disclosed in this embodiment, the audio information, including the sound output from the audio output device, can be tailored to the object being used and the location where the object is located, making it easier to match the usage status of the object with the audio information, including the sound output from the audio output device. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing an example configuration of the management computer and user terminals included in the management system. [Figure 2] This is a block diagram showing an example of the configuration of target objects included in the management system. [Figure 3] This diagram illustrates an example of how a management computer determines the usage status of a target object. [Figure 4] This flowchart shows an example of how the management system works. [Figure 5]This flowchart shows other examples of the management system in operation. [Modes for carrying out the invention]

[0009] (Description of the management system) Figures 1 and 2 show the overall configuration of the management system 10. The management system 10 includes devices such as a management computer 11, a user terminal 12, a first target object 13, and a second target object 14 as its components. The user terminal 12, the first target object 13, and the second target object 14 are used by the user 15. The first target object 13 and the second target object 14 may have been purchased from a store where the user 15 is a customer, or may have been given as a gift from a store where the user 15 is a customer. The user terminal 12 and the management computer 11 are configured to communicate bidirectionally via a network 16. The user terminal 12 and the first target object 13 are configured to communicate bidirectionally via a network 17. The user terminal 12 and the second target object 14 are configured to communicate bidirectionally via a network 18.

[0010] Information is transmitted and received mutually between the management computer 11 and the user terminal 12. Information is also transmitted and received mutually between the user terminal 12 and the first target object 13. Furthermore, information is transmitted and received mutually between the user terminal 12 and the second target object 14. The format of information handled by one or more of the devices includes information itself, data, signals, commands, etc. The types of information handled by one or more of the devices include voice, conversation, music, still images, videos, text, photographs, movies, illustrations, characters, numbers, symbols, sentences, maps, language models, evaluation data, learning models, etc. The management computer 11 is configured to connect to an external computer 52 other than the user terminal 12 via the network 16. The external computer 52 is configured to provide various types of information to the network 16. Therefore, the management computer 11 can access the external computer 52 to acquire various types of external information and process the acquired external information.

[0011] Network 16 consists of one or more communication systems, which are either wireless communication systems or wired communication systems. Wireless communication systems include radio wave communication, optical communication, infrared communication, radio wave communication, satellite communication, etc. Wired communication systems include communication circuits and communication cables. Network 16 includes one or more networks, which are the Internet, intranet, wide area network, and intranet. Networks 17 and 18 both include short-range wireless communication systems. Short-range wireless communication systems are wireless communication systems that transmit and receive information using radio waves or light, etc., with a communication distance of 5m to 15m. Short-range wireless communication systems include, for example, wireless LAN (Wi-Fi®) and Bluetooth®. A short-range wireless communication system consists of a communication device installed on the user terminal 12, an access point installed at the location where the user terminal 12 is used, etc.

[0012] (Description of the first and second target objects) Comparing the first target object 13 and the second target object 14, their usage locations and configurations differ. The first target object 13 is an object used in the first usage location A1. The first usage location A1 is an environment in which user 15 is less restricted in using the target object. The first usage location A1 could be, for example, user 15's home, inside user 15's car, or user 15's relative's house. The second target object 14 is used in the second usage location A2, which has a different environment from the first usage location A1. The second usage location A2 is an environment in which user 15 is restricted when using the target object. The second usage location A2 could be, for example, a park with third parties other than user 15 present, inside a train with third parties other than user 15 present, or inside user 15's workplace building.

[0013] Furthermore, the configuration of the first object 13 and the second object 14 includes size, mass, and appearance. For example, the size and mass of the first object 13 may be larger than those of the second object 14. The first object 13 has the appearance of, for example, a life-size body pillow, robot, or stuffed animal for the user 15. The second object 14 has the appearance of, for example, a cushion, doll, or pad. The surface material of both the first object 13 and the second object 14 is cloth, leather, urethane resin, sponge, etc. When the user 15 comes into contact with the surface of the first object 13 and the second object 14 with a part of their body, they are deformable by the contact pressure. The size of the first object 13 is larger than that of the second object 14. For this reason, the first usage location A1, where the first object 13 can be prevented from coming into contact with surrounding objects, is suitable for using the first object 13. Furthermore, the second target object 14 is relatively small in size, relatively lightweight, and easy to transport, making it suitable for use at the second usage location A2.

[0014] User 15 can heal their mind by interacting with the first object 13 and the second object 14. This interaction includes bringing a part of User 15's body into contact with the first object 13 and the second object 14, engaging in a simulated dialogue between User 15 and the first object 13 and the second object 14, and User 15 taking the second object 14 to the second usage location A2.

[0015] The first target object 13 has a main body, which, as shown in Figure 2, is equipped with a processor 19, main memory 20, auxiliary memory 21, pressure sensor 22, speaker 23, operation unit 24, communication device 25, power supply 26, and identification information display unit 71. The identification information is unique information that identifies the first target object 13, and includes, for example, a serial number, a two-dimensional code, etc. The electrical and electronic components provided in the main body function with power supplied from the power supply 26. The processor 19 is communicably connected to the main memory 20, auxiliary memory 21, pressure sensor 22, speaker 23, operation unit 24, and communication device 25 via a bus 27. The processor 19 may be provided inside the main body and may be configured as a central processing unit (CPU) in which an arithmetic unit (arithmetic circuit) and a control unit (control circuit) are integrated.

[0016] Furthermore, the processor 19 may be implemented by a digital signal processor, an application-specific integrated circuit (ASIC), a graphics processing unit (GPU), or a microprocessing unit (MPU) in addition to the central processing unit, instead of the central processing unit. Image data and audio data may be processed by the GPU.

[0017] The main memory 20 is a volatile storage device, and when the processor 19 executes processing, the main memory 20 functions as a work area and a buffer area. The auxiliary memory 21 is a non-volatile storage device and a non-temporary storage medium. Non-temporary applications are stored in the auxiliary memory 21. Also, information processed by the processor 19 is stored in the auxiliary memory 21. The processor 19 operates non-temporary applications to execute various processes. The processor 19 has a configuration for controlling the voice output from the speaker 23, a configuration for processing the signal detected by the pressure sensor 22, a configuration for determining the operation of the operation unit 24, a configuration for processing the information sent to the user terminal 12 via the communication device 25, and a configuration for processing the information acquired from the user terminal through the communication device 25.

[0018] A plurality of pressure sensors (pressure-sensitive devices) 22 are provided on the main body of the first target object 13. The plurality of pressure sensors 22 are arranged at different positions in the direction along the surface of the main body of the first target object 13. The pressure sensor 22 is, for example, a capacitive type, a resistive film type, an optical type, etc., and outputs a signal corresponding to the pressure received when a part of the user 15's body comes into contact.

[0019] The speaker 23 is an electronic device that converts an electrical signal into voice and outputs it. The speaker 23 outputs the voice included in the voice information generated by the management computer 11 and acquired via the user terminal 12. The operation unit 24 is composed of various switches, buttons, levers, touch panels, etc. provided on the main body of the first target object 13. The user 15 can manually switch between starting and ending the use of the first target object 13 by operating the main switch of the operation unit 24. Also, the user 15 can cause the voice information to be output from the speaker 23 by operating the operation unit 24. The communication device 25 is composed of a communication circuit, a cable, an antenna, etc. The communication device 25 is connected to the user terminal 12 via the network 17. The power supply 26 may be either a DC power supply or an AC power supply. The DC power supply may be either a primary battery or a secondary battery.

[0020] The second target object 14 has a main body, and as shown in FIG. 2, a processor 28, a main memory 29, an auxiliary memory 30, a pressure sensor 31, a speaker 32, an operation unit 33, a communication device 34, a power supply 35, and an identification information display unit 72 are provided in the main body. The identification information is unique information for identifying the second target object 14 and includes, for example, a serial number, a two-dimensional code, etc. The electrical components and electronic components provided in the main body function with the electric power supplied from the power supply 35. The processor 128 is communicably connected to the main memory 29, the auxiliary memory 30, the pressure sensor 31, the speaker 32, the operation unit 33, and the communication device 34 via a bus 36. The processor 28 may be provided inside the main body and may be constituted by a central processing unit (CPU (Central Processing Unit)) in which an arithmetic unit (arithmetic circuit) and a control unit (control circuit) are integrated.

[0021] Also, the processor 28 may be realized by an arithmetic processing circuit such as a digital signal processor, an application specific integrated circuit (ASIC), a GPU (Graphics Processing Unit), an MPU (Micro Processing Unit), etc., instead of or in addition to the central processing unit. The GPU may perform processing of image data and audio data.

[0022] The main memory 29 is a volatile storage device and functions as a work area and buffer area when the processor 28 performs processing. The auxiliary memory 30 is a non-volatile storage device and a non-temporary storage medium. Non-temporary applications are stored in the auxiliary memory 30. The auxiliary memory 30 also stores information and various data processed by the processor 28. The processor 28 runs the non-temporary applications and performs various processes. The processor 28 has a configuration for controlling the sound output from the speaker 32, a configuration for processing signals detected by the pressure sensor 31, a configuration for determining the operation of the operation unit 33, a configuration for processing information to be sent to the user terminal 12 via the communication device 34, and a configuration for processing information obtained from the user terminal 12 via the communication device 34.

[0023] Multiple pressure sensors (pressure-sensitive devices) 31 are provided on the main body of the second target object 14. The multiple pressure sensors 31 are arranged at different positions along the surface of the main body of the second target object 14. The pressure sensors 31 are, for example, capacitive, resistive, or optical, and output a signal corresponding to the pressure received when a part of the user's body 15 comes into contact with them.

[0024] The speaker 32 is an electronic device that converts electrical signals into sound and outputs it. The speaker 32 outputs sound contained in the voice information generated by the management computer 11 and acquired via the user terminal 12. The operation unit 33 consists of various switches, buttons, levers, touch panels, etc., provided on the main body of the second target object 14. The user 15 can manually switch the start and end of use of the second target object 14 by operating the main switch on the operation unit 33. The user 15 can also output voice information from the speaker 32 by operating the operation unit 33. The communication device 34 consists of a communication circuit, cable, antenna, etc. The communication device 34 is connected to the user terminal 12 via the network 18. The power supply 35 consists of a DC power supply. The DC power supply may be either a primary battery or a secondary battery.

[0025] (User terminal description) User terminal 12 is a computer used and operated by user 15. User terminal 12 is configured to be usable by user 15 at both the first usage location A1 and the second usage location A2. In other words, user terminal 12 is a portable computer that user 15 can carry and move between locations. Portable computers include smartphones, tablet computers, notebook computers, etc.

[0026] The user terminal 12 comprises a main unit (casing), a processor 37, main memory 38, auxiliary memory 39, an operation unit 40, a display unit 41, a microphone 42, an audio output device 43, a position determination unit 44, a communication device 45, and the like. The processor 37 is located inside the main unit and is composed of a central processing unit (CPU) that integrates an arithmetic unit (arithmetic circuit) and a control unit (control circuit). The processor 37 is connected to the main memory 38, auxiliary memory 39, operation unit 40, display unit 41, microphone 42, audio output device 43, image acquisition unit 70, position determination unit 44, communication device 45, and the like via a bus 46.

[0027] The main memory 38 is a volatile storage device and functions as a work area and buffer area when the processor 37 performs processing. The auxiliary memory 39 is a non-volatile storage device and a non-temporary storage medium. Non-temporary applications are stored in the auxiliary memory 39. The applications stored in the auxiliary memory 39 include applications downloaded from and installed from the management computer 11.

[0028] The auxiliary memory 39 has a larger capacity than the main memory 38, and operates according to input and output commands from the processor 37. The auxiliary memory 39, which is a non-temporary storage medium, can be implemented by, for example, a magnetic disk, an optical disk, flash memory, etc. An example of a magnetic disk is a hard disk drive. An example of an optical disk is a compact disc, a digital video disc, a Blu-ray disc, etc. Flash memory is a type of semiconductor memory, and an example of a flash memory is an SD memory card, a USB flash drive, a solid-state drive, etc. One or more elements included in the auxiliary memory 39 can be defined as a storage medium 39A that can be attached to and removed from the main unit.

[0029] The operation unit 40 includes at least one element from, for example, a liquid crystal display, an organic electroluminescent display, a touchpad, etc. This element has a display screen on which various buttons, tabs, etc., are displayed. The operation unit 40 may also consist of operation buttons and operation switches provided on the main body of the user terminal 12. By operating the operation unit 40, the user 15 can input various information into the user terminal 12, execute various processes on the user terminal 12, execute various processes on the user terminal 12, exchange various information between the user terminal 12 and the management computer 11, etc.

[0030] Furthermore, by operating the control unit 40, user 15 can exchange various types of information between user terminal 12 and first target object 13, and exchange various types of information between user terminal 12 and second target object 14, etc. In addition, by operating the toggle buttons on the control unit 40, user 15 can switch the start and end of use of the first target object 13 and the start and end of use of the second target object 14, respectively.

[0031] The display unit 41 is a display that is viewed by the user 15, and the display includes structures such as a liquid crystal display or an organic electroluminescent display. The display unit 41 can display various screens, information from websites on the network 16, etc. The display unit 41 can also display various buttons, tabs, the current time, etc. on the screen. In other words, the display unit 41 can also perform the functions of the operation unit 40.

[0032] The microphone 42 is a device that acquires the voice emitted by the user 15 and converts it into an electrical signal for output. The audio output device 43 is an electronic device that converts the electrical signal into sound for output, and the audio output device 43 can be implemented as a speaker, headphones, earphones, etc. The audio output device 43 is a device that outputs the sound contained in the audio information acquired from the management computer 11. The audio output device 43 may be connected to the bus 46 by wired communication or wireless communication.

[0033] The image acquisition unit 70 captures an object, generates an image, and converts the generated image into an electrical signal for output. The image acquisition unit 70 includes, for example, a camera, scanner, barcode reader, etc. The objects captured by the image acquisition unit 70 include scenery, people, documents, code information, etc. Code information includes one-dimensional codes and two-dimensional codes. One-dimensional codes include barcodes. Two-dimensional codes include QR codes (registered trademark). Images include videos, still images, photographs, etc. Images acquired by the image acquisition unit 70 are stored in the auxiliary memory 39.

[0034] The position determination unit 44 is composed of, for example, a detection circuit and a determination circuit. The position determination unit 44 processes signals received from the satellite via the communication device 45, and signals from the gyro sensor and magnetic sensor installed in the user terminal 12. Based on these processing results and map information stored in the auxiliary memory 39, the position determination unit 44 determines the current location of the user terminal 12 and the movement path of the user terminal 12. The current location and movement path of the user terminal 12 are the current location and movement path in the map information. When the user 15 operates the operation unit 40, the map information is displayed on the display unit 41 and the location of the first usage location A1 can be identified. Therefore, the position determination unit 44 can determine the location information of the user terminal 12, that is, whether the user terminal 12 is being used at the first usage location A1 or the second usage location A2.

[0035] Furthermore, the location determination unit 44 can process the communication status between the communication device 45 and the multiple access points constituting the short-range communication system, and based on the processing results, it can determine whether the user terminal 12 is located at the first usage location A1 or the second usage location A2. In addition, the location determination unit 44 can indirectly determine whether the user terminal 12 is located at the first usage location A1 or the second usage location A2 based on the operation of the operation unit 40. For example, if the operation unit 40 is operated and the use of the first target object 13 is started, the location determination unit 44 can determine that the user terminal 12 is located at the first usage location A1. If the use of the second target object 14 is started, the location determination unit 44 can determine that the user terminal 12 is located at the second usage location A2. Note that the location determination unit 44 may be integrated into the processor 37.

[0036] The communication device 45 includes a communication circuit, communication cable, antenna, etc. The communication device 45 is connected to the management computer 11 via the network 16, the communication device 45 is connected to the first target object 13 via the network 17, and the communication device 45 is connected to the second target object 14 via the network 18.

[0037] The processor 37 performs various processes by running applications stored in the auxiliary memory 39. The processes performed by the processor 37 include calculations, decisions, comparisons, control, and storage in the auxiliary memory 39. The processes performed by the processor 37 also include reading information from the auxiliary memory 39, processing information obtained from the management computer 11, processing information obtained from the first target object 13, and processing information obtained from the second target object 14.

[0038] The information acquired by the processor 37 from the first target object 13 includes first contact information determined from the signal of the pressure sensor 22. The information acquired by the processor 37 from the second target object 14 includes second contact information determined from the signal of the pressure sensor 31. The processing performed by the processor 37 includes processing information to send to the management computer 11, processing information to send to the first target object 13, processing information to send to the second target object 14, processing audio input to the microphone 42, processing audio output from the audio output device 43, determining operations performed by the operation unit 40, processing screens and information to be displayed on the display unit 41, and so on.

[0039] (Description of the management computer) The management computer 11 may be managed and operated by the manager or employee of the store (including companies, organizations, corporations, etc.) where the user 15 is a customer. The management computer 11 comprises a main unit (casing), a processor 47, main memory 48, auxiliary memory 49, a communication device 50, an operation unit 73, etc. The processor 47 is located inside the main unit and may consist of a central processing unit (CPU) in which an arithmetic unit (arithmetic circuit) and a control device (control circuit) are integrated. The processor 47 is communicated via a bus 51 to the main memory 48, auxiliary memory 49, communication device 50, operation unit 73, etc. The processor 47 comprehensively controls other devices and circuits located inside the main unit and devices and circuits located outside the main unit. The processor 47 operates non-temporary applications and performs various processes based on instructions input by operating the operation unit 73, information and data acquired from the user terminal 12, etc. The processing performed by the processor 47 includes calculations, decisions, comparisons, control, etc. The processing performed by the processor 47 also includes reading information from the auxiliary memory 49, processing information obtained via the network 16, and adding, changing, and updating information stored in the auxiliary memory 49.

[0040] Furthermore, the processor 47 may be implemented by a digital signal processor, an application-specific integrated circuit (ASIC), a graphics processing unit (GPU), or a microprocessing unit (MPU) in addition to the central processing unit, instead of the central processing unit. Image data and audio data may be processed by the GPU.

[0041] The main memory 48 is a volatile storage device and functions as a work area and buffer area when the processor 47 executes processing. The auxiliary memory 49 is a non-volatile storage device and a non-temporary storage medium. Non-temporary applications are stored in the auxiliary memory 49. Applications include programs, configuration files, data storage files, various libraries, etc. The auxiliary memory 49 also stores information used by the processor 47 to execute processing, information before processing by the processor 47, and information after processing by the processor 47. Furthermore, the auxiliary memory 49 stores language models for the artificial intelligence unit of the processor 47 to execute processing, information and data used by the artificial intelligence unit to perform machine learning, learning models and evaluation data used by the artificial intelligence unit, etc. The configuration and functions of the artificial intelligence unit will be described later.

[0042] The auxiliary memory 49 has a larger capacity than the main memory 48, and operates according to input and output commands from the processor 47. The auxiliary memory 49 includes, for example, magnetic disks, optical disks, flash memory, etc. Examples of magnetic disks include hard disk drives, and examples of optical disks include compact discs, digital video discs, Blu-ray discs, etc. Flash memory is a type of semiconductor memory, and examples of flash memory include SD memory cards, USB flash drives, solid-state drives, etc. One or more elements included in the auxiliary memory 49 can be defined as a storage medium 49A that can be attached to and removed from the main unit.

[0043] The communication device 50 includes devices, equipment, and standards for connecting the management computer 11 to the network 16 by at least one of either a wireless communication system or a wired communication system. The devices and equipment for connecting the management computer 11 to the network 16 include communication circuits, communication cables, antennas, routers, etc. The operation unit 73 may be operated by the store manager or employees. The operation unit 73 can be implemented by, for example, a mouse, keyboard, stylus, touch panel, etc. The store manager or employees can input information, data, and instructions to the management computer 11 by operating the operation unit 73. The information, data, and instructions input by operating the operation unit 73 are stored in the auxiliary memory 49.

[0044] The processor 47 is configured to implement functional units such as the website management unit 54, user information management unit 55, target object information management unit 56, communication information management unit 57, external information processing unit 58, artificial intelligence unit 59, voice information management unit 60, and usage status determination unit 61 by running non-temporary applications.

[0045] The website management unit 54 is configured to manage the website that the management computer 11 provides to the network 16, manage the screens provided to the website, provide applications on the website, and so on.

[0046] The user information management unit 55 processes user information obtained from the user terminal 12 and stores the user information in the auxiliary memory 49. The user information includes the user 15's name, gender, date of birth (birthday) and age, address, telephone number, contract details, identification information of the first target object 13, identification information of the second target object 14, email address, password, and the user 15's interests. The contract details include items such as the "information provision timing" for providing voice information from the management computer 11 to the user terminal 12, the usage fee (cost) for the user terminal 12 to acquire voice information, and the payment account for the fee. The information provision timing refers to the timing at which the management computer 11 provides voice information to the user terminal 12, regardless of whether the management computer 11 acquires communication information from the user terminal 12 or not. The technical meaning of communication information will be described later.

[0047] The timing of information provision may include each time the management computer 11 receives a request from the user terminal 12 to acquire voice information, the timing when the management computer 11 periodically provides voice information to the user terminal 12 in accordance with the contract, etc. Usage fees are incurred each time the user terminal 12 acquires voice information from the management computer 11, or at the timing when the user terminal 12 periodically acquires voice information from the management computer 11. The timing of periodic provision of voice information is managed by a calendar function.

[0048] The email address and password are accounts used to identify different users. User 15's interests include User 15's hobbies, User 15's favorite foods, sports User 15 is interested in, music User 15 is interested in, movies User 15 is interested in, tourist destinations User 15 is interested in, events User 15 is interested in, and products that the store operating the management computer 11 sells or gives to the customer (User 15), etc.

[0049] The target object information management unit 56 processes information about the first target object 13 and the second target object 14 obtained from the user terminal 12, and has a configuration that stores the target object information about the first target object 13 and the second target object 14 in the auxiliary memory 49.

[0050] The target object information includes identification information for the first target object 13, a first usage location A1 where the first target object 13 is expected to be used, identification information for the second target object 14, and a second usage location A2 where the second target object 14 is expected to be used. The identification information for the first target object 13 and the identification information for the second target object 14 are unique and assigned to each object to distinguish them from one another. The usage locations for the first target object 13 and the second target object 14 refer to locations where user 15 can use the first target object 13 and the second target object 14, and which are suitable for interaction with the first target object 13 and the second target object 14. The first usage location A1 for the first target object 13 and the second usage location A2 for the second target object 14 are different.

[0051] User 15 can use the first target object 13 at the first usage location A1 and the second target object 14 at the second usage location A2 by operating the user terminal 12, but does not input specific examples of the first usage location A12 or the second usage location A2. The target object information management unit 56 processes the target object information in association with the user 15's account and stores it in the auxiliary memory 49.

[0052] The communication information management unit 57 is configured to acquire communication information between user 15 and the first target object 13, and communication information between user 15 and the second target object 14, respectively, from the user terminal 12. The communication information management unit 57 also processes the communication information acquired from the user terminal 12 and stores the processed communication information in the auxiliary memory 49, associating it with the user 15's account, the identification information of the first target object 13, and the identification information of the second target object 14, respectively.

[0053] The communication information between user 15 and the first target object 13 includes the date, time, and duration of use of the first target object 13 at the first usage location A1, the date, time, and duration of use of the second target object 14 at the second usage location A2, the audio input by user 15 to the voice output device 43 of the user terminal 12, the first contact information obtained from the first target object 13, and the second contact information obtained from the second target object 14.

[0054] The AC information management unit 57 can determine the user's contact state with the second target object 14 by processing the signal from the pressure sensor 31 included in the first contact information. For example, the AC information management unit 57 can determine that the user 15 "stroked" the first target object 13 if it detects that a plurality of pressure sensors 22 arranged in a predetermined direction sequentially increase and decrease in pressure. The AC information management unit 57 can also determine that the user 15 "pinched" the first target object 13 if it detects that the pressure received by a predetermined number of pressure sensors 22 increases and is maintained. Furthermore, the AC information management unit 57 can determine that the user 15 "kneaded" the first target object 13 if it detects that the pressure received by a predetermined number of pressure sensors 22 repeatedly increases and decreases. The user's contact state with the first target object 13 also corresponds to the AC information between the user 15 and the first target object 13.

[0055] Furthermore, the AC information management unit 57 can determine the user's contact state with the second target object 14 by processing the signals from the pressure sensors 31 included in the second contact information. For example, if the AC information management unit 57 detects that a plurality of pressure sensors 31 arranged in a predetermined direction sequentially increase and decrease in pressure, it can determine that the user 15 "stroked" the second target object 14. Also, if the AC information management unit 57 detects that the pressure received by a predetermined number of pressure sensors 31 increases and is maintained, it can determine that the user 15 "pinched" the second target object 14. Moreover, if the AC information management unit 57 detects that the pressure received by a predetermined number of pressure sensors 31 repeatedly increases and decreases, it can determine that the user 15 "kneaded" the second target object 14. The user's contact state with the second target object 14 also corresponds to the AC information between the user 15 and the second target object 14.

[0056] Furthermore, the communication information management unit 57 may have a configuration for determining the level of interaction between user 15 and the first target object 13 based on communication information, and a configuration for determining the level of interaction between user 15 and the second target object 14 based on communication information. For example, the communication information management unit 57 can determine that the level of interaction between user 15 and the first target object 13, and the level of interaction between user 15 and the second target object 14, have become relatively higher as items such as the number of times user 15 has used the first target object 13 and the second target object 14, the number of conversations and conversation durations between user 15 and the first target object 13 and the second target object 14, respectively, increase.

[0057] Furthermore, the communication information management unit 57 can set the communication level in stages as communication level 1, communication level 2, communication level 3, communication level 4, and communication level 5. Here, the larger the Arabic numeral assigned to the communication level, the higher the communication level is relative to the user. A relatively high communication level means that the intimacy between user 15 and the first target object 13 and the second target object 14 is relatively high. The communication information management unit 57 may also set different types of communication levels, such as friend, best friend, lover, etc., based on the content of the conversation, the places where user 15 went out with the second target object 14, etc. The communication information management unit 57 stores the various judgment results and the settings of various communication levels in the auxiliary memory 49.

[0058] The external information processing unit 58 has a configuration that processes external information acquired by the management computer 11 from the external computer 52 and stores the processed external information in the auxiliary memory 49. The external information acquired by the management computer 11 from the external computer 52 includes external information used by the voice information management unit 60 to generate voice information, external information used by the communication information management unit 57 to determine the communication state between the user 15 and the first target object 13, and external information used to determine the communication state between the user 15 and the second target object 14. The external information processing unit 58 acquires external information from the external computer 52 based on the user 15's address, age, gender, and interests included in the user information.

[0059] The artificial intelligence unit 59 creates history data and update data for the various information stored in the auxiliary memory 49 based on the various information acquired by the processor 47, the various information stored in the auxiliary memory 49, and the learning data, and stores the created data in the auxiliary memory 49. The artificial intelligence unit 59 realizes the learning stage and the inference stage. In the learning stage, that is, the machine learning stage, the artificial intelligence unit 59 obtains output data by processing the input learning target data with the learning model. The machine learning performed by the artificial intelligence unit 59 includes three types: supervised learning, unsupervised learning, and reinforcement learning.

[0060] In supervised learning, the learning model is trained based on the error by comparing the output data resulting from the learning model's processing with the correct answer data. Specifically, the weights and biases are adjusted to minimize the error. Unsupervised learning is a method of training using training data that does not contain the correct answer, and it involves classifying the training data into groups of data with similar features (such as regularities). Reinforcement learning is a method of learning a model through trial and error based on its output data, acquiring guidance on the optimal action to take in order to achieve the objective.

[0061] The artificial intelligence unit 59 can use a neural network as a learning model to realize machine learning. A neural network has an input layer, an intermediate layer (hidden layer), and an output layer. The intermediate layer is configured to make the input information useful in the output layer. Furthermore, the artificial intelligence unit 59 can perform deep learning in realizing the intermediate layer of the neural network. In deep learning, the intermediate layer is multi-layered with multiple types of stages (nodes), and weights and biases are assigned to the functions (features) contained in the information. Deep learning can reduce noise in the information and data input to the learning model because it automatically instructs the intermediate layer to focus on the functions (features).

[0062] The artificial intelligence unit 59 is an artificial intelligence (AI) equipped with transformers including GPT (Generative Pre-trained Transformer), BERT (Bidirectional Encoder Representations from Transformers), etc., and language models such as recurrent neural networks, and is capable of performing generative artificial intelligence processing.

[0063] The artificial intelligence unit 59 outputs an inference result by processing the data to be inferred using a learning model during the inference stage. For example, when the artificial intelligence unit 59 receives information such as the user 15's address, the user 15's age, the user 15's gender, the user 15's interests, the location where the user terminal 12 is used, interaction information between the user 15 and the first target object 13, interaction information between the user 15 and the second target object 14, and external information obtained from an external computer 52 as input data to be inferred, the learning model outputs output data as the inference result. The output data output from the learning model is voice information to be sent to the user terminal 12.

[0064] The artificial intelligence unit 59 can generate voice information to send to the user terminal 12 in such a way that the level of interaction is relatively high, based on, for example, the user's address, the user's age, gender, interests, the location where the user terminal 12 is used, interaction information between the user 15 and the first target object 13, interaction information between the user 15 and the second target object 14, external information obtained from the external computer 52, etc. The voice information generated here includes conversations about events and tourist spots relatively close to the user's address, and topics such as sports, movies, and music estimated from the user's age, gender, and interests.

[0065] Then, after the audio information obtained by processing the data to be inferred is sent to the user terminal 12, the artificial intelligence unit 59 creates evaluation data for the generated learning model based on whether or not the level of interaction between the user 15 and the first target object 13 has increased. Furthermore, the artificial intelligence unit 59 may update the learning model based on the created evaluation data.

[0066] The voice information management unit 60 has a configuration that performs the processing and generation of voice information to be sent to the user terminal 12, processing of voice information acquired from the user terminal 12 and storing it in the auxiliary memory 49, etc. The voice information includes the voice itself, the conversation itself, user information associated with the voice and conversation, and target object information associated with the voice and conversation. The voice information sent to the user terminal 12 includes the voice output from the voice output device 43 of the user terminal 12, the voice output from the speaker 23 of the first target object 13, the voice output from the speaker 32 of the second target object 14, music, etc.

[0067] The voice information management unit 60 may generate voice information to send to the user terminal 12 based on the user information, including the user's address, age, gender, interests, interaction information and interaction level between the user 15 and the first target object 13, interaction information and interaction level between the user 15 and the second target object 14, external information obtained from the external computer 52, etc. When generating voice information, the voice information management unit 60 generates voice information in cooperation with the user information management unit 55, target object information management unit 56, interaction information management unit 57, external information processing unit 58, artificial intelligence unit 59, usage status determination unit 61, and auxiliary memory 49, with the aim of relatively increasing the interaction level between the user 15 and the first target object 13. When generating voice information, the voice information management unit 60 generates voice information in cooperation with the artificial intelligence unit 59, with the aim of relatively increasing the interaction level between the user 15 and the second target object 14.

[0068] The voice information generated by the voice information management unit 60 includes responses and topics corresponding to the voice and conversations generated by the user 15, topics including the user 15's address, age, gender, and interests included in the user information, topics related to external information obtained from the external computer 52, and conversations unrelated to the voice and topics generated by the user 15. Furthermore, the voice information management unit 60 can generate voice information suitable for the first usage location A1 and voice information suitable for the second usage location A2 separately. The voice information management unit 60 is configured to include topics including the user 15's hobbies and interests, responses to the user 15's complaints and dissatisfaction, etc., as voice information suitable for the first usage location A1.

[0069] In response to this, the voice information management unit 60 determines that voice information suitable for the second usage location A2 is content that does not cause any problems even if heard by a third party other than user 15. For this reason, the voice information management unit 60 can include in the voice information corresponding to the second usage location A2, for example, replies to user 15's conversations, topics related to the weather, topics related to the seasons, topics related to currently popular events, topics related to currently popular movies, topics related to currently popular music, topics related to tourist attractions around the current location of user terminal 12, etc. The voice information management unit 60 also determines that the voice information corresponding to the second usage location A2 does not include unsuitable content such as topics containing user 15's personal information or topics related to user 15's complaints or dissatisfaction.

[0070] Furthermore, the voice information management unit 60 is configured to generate voice information based on the user's 15 contact information with the first target object 13 and the second target object 14. The voice information management unit 60 can generate voice information including conversations such as "thank you" and "I'm happy" based on contact information such as "stroked". The voice information management unit 60 can generate voice information including conversations such as "ouch" and "stop" based on contact information such as "pinched". The voice information management unit 60 can generate voice information including conversations such as "feels good" based on contact information such as "rubbed".

[0071] Furthermore, the voice information management unit 60 may have a function to generate and acquire voice information, such as auxiliary voice information, that is not related to the communication information and communication level between the user 15 and the first target object 13, or the communication information and communication level between the user 15 and the second target object 14. The voice information management unit 60 stores the generated and acquired auxiliary voice information in the auxiliary memory 49. The auxiliary voice information includes greetings spoken to the user 15, voice information concerning the first target object 13 and the second target object 14, and store-side information. Store-side information includes the date and time of events held at the store, information on products sold or given as gifts from the store to customers, birthdays of store employees, days when store employees are working, etc.

[0072] The store manager or employee can input store-side information into the management computer 11 by operating the control unit 73 of the management computer 11. The store manager or employee may also set the timing for sending store-side information to the user terminal 12, and the timing for outputting the store-side information sent to the user terminal 12 from one or more of the user terminal 12, the first target object 13, or the second target object 14, by operating the control unit 73. The information, data, and various timings input by operating the control unit 73 are stored in the auxiliary memory 49.

[0073] The usage status determination unit 61 has a configuration that determines the usage status of the first target object 13, the second target object 14, the first usage location A1, and the second usage location A2. Specifically, the usage status determination unit 61 determines whether user 15 is using the first target object 13 or the second target object 14, and also determines whether user terminal 12 is actually being used at the first usage location A1 or the second usage location A2. Figure 3 is a diagram showing an example of the determination made by the usage status determination unit 61.

[0074] An example of a decision made by the usage status determination unit 61 is described below. The usage status determination unit 61 can determine that user 15 is using the first target object 13 if the main switch of the first target object 13 is ON and the main switch of the second target object 14 is OFF. The usage status determination unit 61 can determine that user 15 is using the second target object 14 if the main switch of the first target object 13 is OFF and the main switch of the second target object 14 is ON.

[0075] The usage status determination unit 61 can determine that user 15 is using the first target object 13 if the pressure sensor 22 is receiving pressure. The usage status determination unit 61 can determine that user 15 is using the second target object 14 if the pressure sensor 31 is receiving pressure. The usage status determination unit 61 can determine that user 15 is using the first target object 13 if the toggle button on the operation unit 40 is operated and the first target object 13 is selected. The usage status determination unit 61 can determine that user 15 is using the second target object 14 if the toggle button on the operation unit 40 is operated and the second target object 14 is selected. Furthermore, based on the determination result of the location determination unit 44, the usage status determination unit 61 can determine whether the user terminal 12 is being used at the first usage location A1 or the second usage location A2 during the actual inspection. The processor 47 implements a known clock function. Furthermore, the processor 47 implements the calendar function by launching the calendar application stored in the auxiliary memory 49.

[0076] (Example of management system operation) An example of the operation of the management system 10 is shown in Figure 4. Here, for convenience, we will describe an example where the application used to operate the management system 10 is a native application. The native application is downloaded to the user terminal 12 and installed and used on the user terminal 12.

[0077] In step S10, the management computer 11 manages voice information to be provided to the user terminal 12. Also in step S11, the management computer 11 provides a website to the network 16. When user 15 operates the user terminal 12 to run a web browser and access the website provided by the management computer 11, the user terminal 12 displays a user screen on the display unit 41 in step S20. The user screen displays available applications. User 15 operates the user terminal 12 to select an application in step S21 and then downloads the selected application. In step S21, user 15 further operates the user terminal 12 to install the application.

[0078] User 15 operates the user terminal 12 to run the application in step S22 and inputs user information and target object information into the user terminal 12. Then, user 15 operates the user terminal 12 to send the user information and target object information to the management computer 11. In step S12, the management computer retrieves and processes the user information and target object information from the user terminal 12 and stores the user information and target object information in the auxiliary memory 49, associating them.

[0079] In step S23, the user terminal 12 detects its location information and associates the user 15's account with the location information, then transmits it to the management computer 11. In step S13, the management computer 11 obtains and processes the location information of the user terminal 12 and the user 15's account from the user terminal 12, associates the location information of the user terminal 12 with the user 15's account, and stores it in the auxiliary memory 49. Steps S23 and S13 are repeated at predetermined time intervals.

[0080] When user 15 and the first target object 13 interact in step S30, user terminal 12 acquires interaction information about the interaction between user 15 and the first target object 13 in step S24. The interaction information between user 15 and the first target object 13 includes the identification information of the first target object 13, the date and time when the interaction took place between user 15 and the first target object 13, the voice input from user 15 to the voice output device 43 of user terminal 12, and contact information detected by the pressure sensor 22 of the first target object 13.

[0081] Furthermore, when user 15 and the second target object 14 interact in step S40, user terminal 12 acquires interaction information regarding the interaction between user 15 and the second target object 14 in step S24. The interaction information between user 15 and the second target object 14 includes the identification information of the second target object 14, the date and time when the interaction took place between user 15 and the second target object 14, the voice input by user 15 to the speaker 32 of user terminal 12, and contact information detected by the pressure sensor 31 of the second target object 14.

[0082] In step S25, the user terminal 12 associates the communication information and the user 15's account and transmits them to the management computer 11. In step S14, the management computer 11 processes the communication information and the user 15's account obtained from the user terminal 12 and stores the communication information and the user 15's account in the auxiliary memory 49, associating them with the location information of the user terminal 12. Furthermore, in step S15, the management computer 11 determines the usage status of the first target object 13, the second target object 14, the first usage location A1, and the second usage location A2.

[0083] In step S15, the management computer 11 further determines whether the location where the user terminal 12 is used matches the target object being used. Specifically, the management computer 11 determines that the locations match if the user terminal 12 is actually used at the first location A1 and the first target object 13 is being used. The management computer 11 also determines that the locations match if the user terminal 12 is actually used at the second location A2 and the second target object 14 is being used.

[0084] In response, the management computer 11 determines that the user terminal 12 is actually being used at the first usage location A1 and the second target object 14 is being used, and that there is a mismatch. The management computer 11 also determines that the user terminal 12 is actually being used at the second usage location A2 and the first target object 13 is being used, and that there is a mismatch. If the management computer 11 determines in step S15 that there is a mismatch, it sends that information to the user terminal 12.

[0085] When the user terminal 12 receives information from the management computer 11 that the information is "inconsistent" in step S26, it performs one or more processes from which it displays the information on the display unit 41 or outputs it as sound from the voice output device 43. When the user 15 recognizes the information that the user terminal 12 is "inconsistent," the user 15 takes operations and actions to align the location of use of the user terminal 12 with the usage status of the first target object 13 and the second target object 14.

[0086] Meanwhile, after step S15, the management computer 11 generates voice information corresponding to the communication information acquired from the user terminal 12 in step S16, and stores the generated voice information in the auxiliary memory 49. When the management computer 11 generates voice information in step S16, it generates voice information that is suitable for the usage state. Specifically, voice information is generated so that the communication level can be relatively increased based on the communication information exchanged between user 15 and the first target object 13 at the first usage location A1. Similarly, voice information is generated so that the communication level can be relatively increased based on the communication information exchanged between user 15 and the second target object 14 at the second usage location A2.

[0087] The audio information generated by the management computer 11 in step S16 corresponds to the content of the communication information that took place between the user 15 and either the first target object 13 or the second target object 14. For example, the content of the audio information generated when communication takes place between the user 15 and the first target object 13 will be different from the content of the audio information generated when communication takes place between the user 15 and the second target object 14.

[0088] Furthermore, if user 15 comes into contact with the first target object 13, the content of the generated audio information will differ depending on how user 15 comes into contact with the first target object 13. In addition, if user 15 comes into contact with the second target object 14, the content of the generated audio information will differ depending on how user 15 comes into contact with the second target object 14. Note that if user 15 and the first target object 13 interact at the second usage location A2, or if user 15 and the second target object 14 interact at the first usage location A1, the management computer 11 will not generate audio information in step S16. The management computer 11 will transmit the audio information generated in step S16 to the user terminal 12 in step S17.

[0089] In step S27, following step S26, the user terminal 12 acquires audio information from the management computer 11 and stores it in the auxiliary memory 39. If the audio information acquired from the management computer 11 includes audio information to be sent to the first target object 13, the user terminal 12 may transmit that audio information to the first target object 13 in step S28. When the first target object 13 acquires the audio information in step S31, it can output sound from the speaker 23 in step S32. Also, if the audio information acquired from the management computer 11 includes audio information to be sent to the second target object 14, the user terminal 12 may transmit that audio information to the second target object 14 in step S28. When the second target object 14 acquires the audio information in step S41, it can output sound from the speaker 32 in step S42.

[0090] The user terminal 12 may output audio information from the audio output device 43 in step S29 instead of processing in step S28. The management computer 11 may perform steps S10 and S11 simultaneously, or it may perform step S10 after step S11.

[0091] (Effects of the embodiment) The management computer 11 disclosed in this embodiment can control audio information, including the sound output from the speakers 23, 32 and the audio output device 43, based on the determination result of the target object being used and the determination result of the location where the target object is located. Therefore, the audio information, including the sound output from the speakers 23, 32 and the audio output device 43, can be made to correspond to the target object being used and the location where the target object is located, making it easier to match the usage status of the target object with the audio information, including the sound output from the speakers 23, 32 and the audio output device 43.

[0092] Furthermore, the management computer 11 determines whether the object being used matches the location where it is being used. If it determines that the object being used matches the location where it is being used, it generates audio information including the sound output from the speakers 23, 32 and the audio output device 43. Therefore, the management computer 11 makes it easier to match the usage status of the object with the audio information including the sound output from the speakers 23, 32 and the audio output device 43.

[0093] Furthermore, if the management computer 11 determines that the object being used and the location where it is being used are inconsistent, it sends information including this determination to the user terminal 12. Therefore, the user 15 can recognize the situation where the object being used and the location where it is being used are inconsistent.

[0094] Furthermore, the management computer 11 determines the object being used based on the operation content of the operation unit 40 obtained from the user terminal 12. Therefore, the management computer 11's ability to determine the object being used is improved.

[0095] Furthermore, the management computer 11 determines the object being used based on the pressure information obtained from the user terminal 12. Therefore, the management computer 11's ability to determine the object being used is improved.

[0096] Furthermore, the management computer 11 determines the location where the target object is located based on the determination of the location determination unit 44 obtained from the user terminal 12. Therefore, the management computer 11's ability to determine the location where the target object is being used is improved.

[0097] Furthermore, if multiple target objects are provided, and user 15 uses the first target object 13 and the second target object 14, the sound output from the speaker 23 of the first target object 13, the speaker 32 of the second target object 14, and the sound output device 43 of the user terminal 12 can be switched according to the location where user 15 is using the device.

[0098] Furthermore, if the processes from step S23 to step S28 are repeated at the user terminal 12, or if the processes from step S23 to step S27 and step S29 are repeated, and the processes from step S13 to step S17 are repeated at the management computer 11, and the processes from step S30 to step S32 are repeated at the first target object, the interaction state between the user 15 and the first target object 13 changes.

[0099] The management computer 11 can change and evolve the audio information generated in step S16 in response to changes in the interaction state between the user 15 and the first target object 13. Therefore, the audio output from the audio output device 43 of the user terminal 12 and the speaker 23 of the first target object 13 can be changed, respectively, in accordance with the interaction state between the user 15 and the first target object 13.

[0100] Furthermore, if the processes from step S23 to step S28 are repeated at the user terminal 12, or if the processes from step S23 to step S27 and step S29 are repeated, and the processes from step S13 to step S17 are repeated at the management computer 11, and the processes from step S40 to step S42 are repeated at the second target object 14, the interaction state between the user 15 and the second target object 14 changes. The management computer 11 can change and evolve the audio information generated in step S16 in accordance with the change in the interaction state between the user 15 and the second target object 14. Therefore, the audio output from the audio output device 43 of the user terminal 12 and the speaker 32 of the second target object 14 can be changed in accordance with the interaction state between the user 15 and the second target object 14.

[0101] (Other specific example 1) Another specific example is to assume that the user terminal 12 shown in Figure 1 has the configuration and functions of the management computer 11. Furthermore, the application for the user terminal 12 to execute the processes shown in Figure 4 is pre-stored in the user terminal 12's auxiliary memory 39. The user terminal 12 also has the configuration and functions of the management computer 11. In other words, in the flowchart shown in Figure 4, the user terminal 12 executes the processes, decisions, and controls performed by the management computer 11, excluding step S11. The user terminal 12 can also make the usage status determination shown in Figure 3.

[0102] (Effects of other specific example 1) The user terminal 12 can control audio information, including the audio output from speakers 23, 32 and audio output device 43, based on the determination of the object being used and the determination of the location where the object is located. Therefore, the same effects as described above can be obtained.

[0103] Furthermore, the user terminal 12 determines whether the object being used matches the location where it is being used. If it determines that the object being used matches the location where it is being used, it generates audio information including the sound output from the speakers 23, 32 and the audio output device 43. Therefore, the same effect as described above can be obtained.

[0104] Furthermore, if the user terminal 12 determines that the object being used and the location where it is being used do not match, it transmits information including the content of that determination to the user terminal 12. Therefore, the same effect as described above can be obtained. Furthermore, the user terminal 12 determines the object being used based on the operation content of the operation unit 40. Therefore, the function of the user terminal 12 in determining the object being used is improved. Furthermore, the user terminal 12 determines the object being used based on the pressure information obtained from the first object 13 and the second object 14. Therefore, the function of the user terminal 12 in determining the object being used is improved.

[0105] Furthermore, the user terminal 12 determines the location where the target object is located based on the determination of the location determination unit 44. Therefore, the user terminal 12's ability to determine the location where the target object is being used is improved. In addition, if multiple target objects are provided and the user 15 uses the first target object 13 and the second target object 14, the sound output from the speaker 23 of the first target object 13, the speaker 32 of the second target object 14, and the sound output device 43 of the user terminal 12 can be switched according to the location where the user 15 is located.

[0106] Furthermore, if the processes from step S23 to step S28 are repeated at the user terminal 12, or if the processes from step S23 to step S27 and step S29 are repeated, and the processes from step S13 to step S17 are repeated at the management computer 11, and the processes from step S30 to step S32 are repeated at the first target object, the interaction state between the user 15 and the first target object 13 changes.

[0107] The management computer 11 can change and evolve the audio information generated in step S16 in response to changes in the interaction state between the user 15 and the first target object 13. Therefore, the audio output from the audio output device 43 of the user terminal 12 and the speaker 23 of the first target object 13 can be changed, respectively, in accordance with the interaction state between the user 15 and the first target object 13.

[0108] Furthermore, if the processes from step S23 to step S28 are repeated at the user terminal 12, or if the processes from step S23 to step S27 and step S29 are repeated, the processes from step S13 to step S17 are repeated at the management computer 11, and the processes from step S40 to step S42 are repeated at the second target object 14, the interaction state between the user 15 and the second target object 14 changes. The user terminal 12 can change and evolve the audio information generated in step S16 in accordance with the change in the interaction state between the user 15 and the second target object 14. Therefore, the audio output from the audio output device 43 of the user terminal 12 and the speaker 32 of the second target object 14 can be changed in accordance with the interaction state between the user 15 and the second target object 14.

[0109] (Other examples of operation in the management system) The management system 10 may implement the operation example shown in Figure 5. The operation example in Figure 5 is based on the premise that the management system 10 can implement the operation example shown in Figure 4. The application that enables the management computer 11 to implement the processing shown in Figure 5 is stored in the auxiliary memory 49 of the management computer 11. The management computer 11 may generate voice information in step S50. The voice information generated by the management computer 11 in step S50 includes auxiliary voice information unrelated to the interaction information. The auxiliary voice information includes greetings spoken to the user 15, reminders notifying the user 15 of the timing of events, information including the user 15's interests, voice information regarding the first target object 13 and the second target object 14, store-side information, etc.

[0110] Greetings directed to user 15 may include, for example, "Good morning," "Good evening," and "Good night." These greetings may be output in accordance with the time measured by the clock function of user terminal 12. The timing of events may also include, for example, seasonal and calendar-based events, the birthdays of employees at the store where user 15 is a customer, the working days of employees at the store where user 15 is a customer, and the period during which goods are sold or given as gifts to customers at the store that manages and operates the management computer 11.

[0111] Store-side information may include, for example, "It's my birthday the day after tomorrow, so please come to the store," "My movie will be released on [date]," or "My plush toys will be released on [date] next week." Seasonal and calendar-based events may include Christmas, cherry blossom viewing, Halloween, Tanabata, etc. Audio information relating to the first object 13 and the second object 14 includes information on the release of new products for the first object 13 and the second object 14, information indicating the location and date of events relating to the first object 13 and the second object 14, and audio information of different sound qualities generated corresponding to the first object 13 and the second object 14, respectively.

[0112] The management computer 11 may send the voice information generated in step S50 to the user terminal 12 in step S51. The management computer 11 may execute steps S50 and S51 based on the contract details included in the user information. If the contract details stipulate that voice information be provided to the user terminal 12 each time a request to obtain voice information is received from the user terminal 12, the management computer 11 executes steps S50 and S51 each time a request to obtain voice information is received from the user terminal 12. On the other hand, if the contract stipulates that voice information be provided to the user terminal 12 periodically, steps S50 and S51 may be executed regardless of whether or not a request to obtain voice information has been received from the user terminal 12.

[0113] In step S60, user 15 may operate the operation unit 40 to input a request to acquire voice information and send the request to acquire voice information from the user terminal 12 to the management computer 11. Alternatively, in step S20, user terminal 12 may read the identification information display unit 71 provided on the first target object 13 or the identification information display unit 72 provided on the second target object 14 and send the request to acquire voice information to the management computer 11.

[0114] In step S61, the user terminal 12 acquires, i.e., downloads, audio information from the management computer 11 and stores the acquired audio information in the auxiliary memory 39. In step S62, following step S61, the user terminal 12 outputs the audio information from the audio output device 43. Alternatively, in step S64, the user terminal 12 may transmit the audio information to the first target object 13 and the second target object 14.

[0115] The first target object 13 may acquire voice information from the user terminal 12 in step S70. The first target object 13 may output voice information from the speaker 23 when the operation unit 24 is operated in step S71, which follows step S70. The second target object 14 may acquire voice information from the user terminal 12 in step S80. The second target object 14 may output voice information from the speaker 32 when the operation unit 33 is operated in step S71, which follows step S80. Steps S63, S70, S71, S80, and S81 shown in Figure 5 are not required to be performed.

[0116] (supplementary explanation) An example of the technical meaning disclosed in this embodiment is as follows: Management system 10 is an example of a management system. Management computer 11 is an example of a computer. Auxiliary memory 39, 49 are examples of memory. Processor 37, 47 are examples of processors. User terminal 12 is an example of a user terminal. Operation unit 40 is an example of an operation unit. Communication device 45 is an example of a communication device. Location determination unit 44 is an example of a location determination unit. First target object 13 and second target object 14 are examples of target objects. Audio output device 43 and speakers 23, 32 are examples of audio output devices, respectively. The audio output device only needs to be connected to the user terminal in a communicative manner, and the user terminal itself may also be equipped with an audio output device. Pressure sensors 22, 31 are examples of pressure sensors, respectively. First usage location A1 and second usage location A2 are examples of multiple different usage locations.

[0117] In Figures 1 and 2, a single user terminal 12 is shown for convenience, but multiple user terminals 12 used by different users 15 can be connected to the management computer 11. Also, in Figures 1 and 2, a single first target object 13 and a single second target object 14 are shown, but multiple first target objects 13 and multiple second target objects 14 may be connected to the user terminal 12. Furthermore, there may be three or more types of target objects, and three or more different usage locations may be set up.

[0118] The application that operates the management system 10 may be a native application, a web application, or a hybrid application. A web application is an application that runs on a web browser. A hybrid application is an application that combines the functions of a web application and a native application. If the application that operates the management system 10 is a native application, it is stored in the auxiliary memory 39 of the user terminal 12. If the application that operates the management system 10 is a web application, it is stored in the auxiliary memory 49 of the management computer 11. Therefore, when the management system 10 implements the operation example shown in Figure 4, the information acquired by the user terminal 12 is processed and stored by the management computer 11, and then sent from the management computer 11 to the user terminal 12.

[0119] The management system 10 disclosed in this embodiment can also be understood as a voice management system. Furthermore, the management computer 11 can also be understood as a voice management computer. Additionally, the application that enables the operation example shown in Figure 4 using the management system 10 can be understood as a management application or a voice management application.

[0120] Furthermore, the management computer 11 may consist of a single computer or may be configured distributed across multiple computers. The management computer 11 includes one or more computers, such as a server, supercomputer, mainframe, workstation, etc. Also, the applications disclosed in this embodiment can be understood as application products. [Industrial applicability]

[0121] This disclosure can be used as a management system having multiple objects used by a user, a management computer that manages multiple objects used by a user, and an application that operates a system having multiple objects used by a user. [Explanation of symbols]

[0122] 10...Management system, 11...Management computer, 12...User terminal, 13...First target object, 14...Second target object, 22,31...Pressure sensor, 23,32...Speaker, 40...Operation unit, 43...Audio output device, 44...Position determination unit, 45...Communication device, 37,47...Processor, 39,49...Auxiliary memory, A1...First usage location, A2...Second usage location

Claims

1. A target object having a configuration that allows interaction with the user, A user terminal used by a user and connected to the aforementioned target object in a manner that enables communication with it, A sound output device that outputs sound corresponding to the user's interaction state with the target object, A management computer is connected to the user terminal in a communication-enabled manner, stores information, and performs processing. A management system equipped with, The aforementioned object includes multiple objects with different configurations so that they can be used in multiple different locations. The aforementioned management computer, A process to store in memory the configuration of each of the multiple target objects, and the appropriate usage location for each of the multiple target objects, A process to determine which of the aforementioned multiple target objects is being used, A process to determine which of the usage locations the object being used is located in, A process for controlling audio information, including audio output from the aforementioned audio output device, based on the determination result of the target object being used and the determination result of the location where the target object is located. A management system configured to perform the following actions.

2. A management system according to claim 1, The aforementioned management computer, A process to determine whether the object being used and the location where it is being used are consistent, When it is determined that the object being used and the location where it is being used are consistent, the process generates audio information including the audio output from the audio output device, A management system configured to perform the following actions.

3. A management system according to claim 2, The aforementioned management computer, If it is determined that the object being used and the location where it is being used do not match, the process includes sending information containing that determination to the user terminal. A management system configured to perform the following actions.

4. A management system according to claim 1, The aforementioned user terminal is An operating unit operated by the user to select which of the aforementioned multiple target objects to use, A communication device that transmits the operation details of the control unit to the management computer, It has, The management computer is configured to perform a process to determine the target object being used based on the operation content of the operation unit obtained from the user terminal.

5. A management system according to claim 1, Each of the aforementioned multiple target objects has a pressure sensor that detects the pressure of contact with the user, The user terminal has a configuration that acquires pressure information received by the pressure sensor from the multiple target objects and transmits the acquired pressure information to the management computer. The management computer is configured to perform a process to determine the target object being used based on the pressure information obtained from the user terminal.

6. A management system according to claim 1, The aforementioned user terminal is A location determination unit that determines which of the multiple usage locations shown in the map information the user terminal is located in, A communication device that transmits the determination content of the position determination unit to the management computer, It has, The management computer is configured to perform a process to determine the location of the object based on the determination of the location determination unit obtained from the user terminal, thereby determining the location of the object.

7. A management computer having a user terminal that is communicatively connected to an object used by a user, and an audio output device that outputs audio corresponding to the user's interaction status with the object, and having a memory for storing information and a processor for executing processing, The aforementioned object includes multiple objects with different configurations so that they can be used in multiple different locations. The aforementioned processor, A process to store in memory the configuration of each of the multiple target objects, and the appropriate usage location for each of the multiple target objects, A process to determine which of the aforementioned multiple target objects is being used, A process to determine which of the usage locations the object being used is located in, A process for controlling audio information, including audio output from the aforementioned audio output device, based on the determination result of the target object being used and the determination result of the location where the target object is located. A management computer configured to perform the following actions.

8. The management computer according to claim 7, The processor is configured to generate audio information, including audio output from the audio output device, based on user information acquired from the user terminal, and is a management computer.

9. A non-temporary application that causes a management computer to perform processing in a management system comprising: an object having a configuration that allows interaction with a user; a user terminal used by the user and connected to the object in a communicative manner; an audio output device that outputs sound according to the user's interaction status with the object; and a management computer connected to the user terminal in a communicative manner, which stores information and performs processing, The aforementioned object includes multiple objects with different configurations so that they can be used in multiple different locations. The aforementioned management computer, A process to store in memory the configuration of each of the multiple target objects, and the appropriate usage locations for each of the multiple target objects, A process to determine which of the aforementioned multiple target objects is being used, A process to determine which of the usage locations the object being used is located in, A process for controlling audio information, including audio output from the aforementioned audio output device, based on the determination result of the target object being used and the determination result of the location where the target object is located. An application that has a configuration to execute [something].

10. A user terminal that is connected to an object used by the user and an audio output device that outputs sound, and has a memory for storing information and a processor for executing processing, The aforementioned object includes multiple objects with different configurations so that they can be used in multiple different locations. The aforementioned processor, A process to store in memory the configuration of each of the multiple target objects, and the appropriate usage location for each of the multiple target objects, A process to determine which of the aforementioned multiple target objects is being used, A process to determine which of the usage locations the object being used is located in, A process for controlling audio information, including audio output from the aforementioned audio output device, based on the determination result of the target object being used and the determination result of the location where the target object is located. A user terminal configured to execute [the command / function].

11. A non-temporary application that causes a user terminal, which is connected to a target object used by the user and an audio output device that outputs sound, to perform processing, The aforementioned object includes multiple objects with different configurations so that they can be used in multiple different locations. On the aforementioned user terminal, A process for storing the configuration of each of the multiple target objects, and the appropriate usage locations for each of the multiple target objects, A process to determine which of the aforementioned multiple target objects is being used, A process to determine which of the usage locations the object being used is located in, A process for controlling audio information, including audio output from the aforementioned audio output device, based on the determination result of the target object being used and the determination result of the location where the target object is located. An application that has a configuration to execute [something].

Images