Management system, management computer, user terminal, and application

Abstract

[Problem] To provide a management system capable of controlling voice information to be outputted from a voice generation device when a plurality of target objects are separately used in a plurality of use places. [Solution] A management system 10 comprises: a first target object 13 and a second target object 14; a user terminal 12; speakers 23, 32 and a voice output device 43 that output voice; and a management computer 11 that is communicably connected to the user terminal 12 and executes processing. The management system 10 is configured such that the management computer 11 executes processing for storing the first target object 13, the second target object 14, a first use place A1, and a second use place A2 in an auxiliary memory 49, processing for determining which of target objects in the first use place A1 and the second use place A2 is used, processing for determining the use place of the used target object, and processing for controlling voice information including voice to be outputted from the speakers 23, 32 and a the voice output device 43 on the basis of the determination result of the used target object and the determination result of the use place.

Description

Management system, management computer, user terminal, and application

[0001] The present disclosure relates to a management system having a plurality of objects used by users, a management computer that manages the plurality of objects used by users, a user terminal, and an application.

[0002] An example of a voice generation program for controlling voices output from multiple robots is described in Patent Document 1. Patent Document 1 describes a robot that is used by a user and includes a voice generation unit that generates voices and a voice output unit that outputs the generated voices. Furthermore, when multiple robots are present in a home, the voice generation unit described in Patent Document 1 generates voices for each robot so that the voices of the robots are not the same.

[0003] JP 2024-40159 A

[0004] The inventor of the present application found that the voice generation program described in Patent Document 1 did not recognize how to control the voice information output from the voice generating device when multiple target objects are used separately in multiple usage locations, and that there was room for improvement in this regard.

[0005] The object of the present disclosure is to provide a management system, a management computer, a user terminal, and an application that can appropriately control the audio information output from an audio generating device when multiple target objects are used separately in multiple usage locations.

[0006] This embodiment discloses a management system comprising a target object having a configuration that allows interaction with a user, a user terminal used by the user and communicatively connected to the target object, an audio output device that outputs audio according to the user's interaction state with the target object, and a computer communicatively connected to the user terminal, storing information, and executing processing, wherein the target objects include a plurality of target objects each having a different configuration so that they can be used at a plurality of different usage locations, and the computer is configured to execute the following processes: storing in memory the configuration of each of the plurality of target objects and the respective usage locations suitable for the plurality of target objects; determining which of the plurality of target objects is being used; determining in which usage location the target object being used is located; and controlling audio information including audio output from the audio output device based on the determination result of the target object being used and the determination result of the usage location where the target object being used is located.

[0007] In the management system disclosed in this embodiment, audio information including audio output from an audio output device can be made to correspond to the target object being used and the usage location where the target object being used is located, making it easier to align the usage status of the target object with the audio information including audio output from the audio output device.

[0008] Fig. 1 is a block diagram showing an example of the configuration of a management computer and a user terminal included in the management system; Fig. 2 is a block diagram showing an example of the configuration of a target object included in the management system; Fig. 3 is a diagram showing an example in which the management computer determines the usage state of the target object; Fig. 4 is a flowchart showing an example of the operation of the management system; Fig. 5 is a flowchart showing another example of the operation of the management system.

[0009] (Explanation of Management System) FIGS. 1 and 2 show the overall configuration of a management system 10. The management system 10 includes devices such as a management computer 11, a user terminal 12, a first target object 13, and a second target object 14 as components. The user terminal 12, the first target object 13, and the second target object 14 are used by a user 15. The first target object 13 and the second target object 14 may be sold by a store where the user 15 is a customer, or may be given as a gift from a store where the user 15 is a customer. The user terminal 12 and the management computer 11 are configured to be able to communicate bidirectionally via a network 16. The user terminal 12 and the first target object 13 are configured to be able to communicate bidirectionally via a network 17. The user terminal 12 and the second target object 14 are configured to be able to communicate bidirectionally via a network 18.

[0010] Information is transmitted and received between the management computer 11 and the user terminal 12. Information is also transmitted and received between the user terminal 12 and the first target object 13. Information is also transmitted and received between the user terminal 12 and the second target object 14. The format of information handled by one or more of the devices includes information itself, data, signals, commands, etc. The type of information handled by one or more of the devices includes voice, conversation, music, still images, videos, text, photographs, movies, illustrations, letters, numbers, symbols, sentences, maps, language models, evaluation data, learning models, etc. The management computer 11 is configured to be connectable to an external computer 52 other than the user terminal 12 via the network 16. The external computer 52 is configured to provide various types of information to the network 16. Therefore, the management computer 11 can access the external computer 52 to acquire various types of external information and process the acquired external information.

[0011] The network 16 is composed of one or more communication systems, either a wireless communication system or a wired communication system. Wireless communication systems include radio wave communication, optical communication, infrared communication, radio wave communication, satellite communication, etc. Wired communication systems include communication circuits and communication cables. The network 16 includes one or more networks of the Internet, an intranet, a wide area network, and an intranet. Both networks 17 and 18 include short-range wireless communication systems. A short-range wireless communication system is a wireless communication system that uses radio waves or light to exchange information, with a communication distance of 5 to 15 meters. Examples of short-range wireless communication systems include wireless LAN (Wi-Fi (registered trademark)) and Bluetooth (registered trademark). A short-range wireless communication system is composed of a communication device provided in the user terminal 12, an access point provided at the location where the user terminal 12 is used, etc.

[0012] (Explanation of the First Target Object and the Second Target Object) Comparing the first target object 13 and the second target object 14, the first target object 13 is an object used at a first use location A1. The first use location A1 is an environment in which the user 15 is less restricted in using the target object. The first use location A1 is, for example, the home of the user 15, inside a car owned by the user 15, the home of a relative of the user 15, etc. The second target object 14 is used at a second use location A2, which is an environment different from the first use location A1. The second use location A2 is an environment in which the user 15 is restricted in using the target object. The second use location A2 is, for example, a park where a third party other than the user 15 is present, on a train where a third party other than the user 15 is present, inside a building where the user 15 works, etc.

[0013] The configuration of the first target object 13 and the second target object 14 also includes size, mass, and appearance. For example, the size and mass of the first target object 13 may be larger than the size and mass of the second target object 14. The first target object 13 may have an appearance such as a life-size body pillow of the user 15, a robot, a stuffed animal, etc. The second target object 14 may have an appearance such as a cushion, a doll, a pad, etc. The surfaces of the first target object 13 and the second target object 14 are both made of cloth, leather, urethane resin, sponge, etc. When the user 15 touches the surfaces of the first target object 13 and the second target object 14 with a part of their body, they can be deformed by the contact pressure. The size of the first target object 13 is larger than the size of the second target object 14. Therefore, the first target object 13 is suitable for use at a first use location A1 where the first target object 13 can be prevented from contacting surrounding objects. Furthermore, the second target object 14 is relatively small in size, relatively lightweight, and easy to transport, making it suitable for use at the second use location A2.

[0014] The user 15 can heal his or her mind by interacting with the first target object 13 and the second target object 14. The interaction includes bringing a part of the user 15's body into contact with the first target object 13 and the second target object 14, having a simulated conversation between the user 15 and the first target object 13 and the second target object 14, the user 15 carrying the second target object 14 and going out to the second use location A2, and the like.

[0015] The first target object 13 has a main body, which is provided with a processor 19, a main memory 20, an auxiliary memory 21, a pressure sensor 22, a speaker 23, an operation unit 24, a communication device 25, a power supply 26, and an identification information display unit 71, as shown in FIG. 2 . The identification information is unique information that identifies the first target object 13, and includes, for example, a serial number, a two-dimensional code, etc. The electrical and electronic components provided in the main body function with power supplied from the power supply 26. The processor 19 is communicatively connected to the main memory 20, the auxiliary memory 21, the pressure sensor 22, the speaker 23, the operation unit 24, and the communication device 25 via a bus 27. The processor 19 may be provided inside the main body and configured as a central processing unit (CPU) that integrates an arithmetic unit (arithmetic circuit) and a control device (control circuit).

[0016] Furthermore, instead of or in addition to a central processing unit, the processor 19 may be realized by an arithmetic processing circuit such as a digital signal processor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or a micro processing unit (MPU). Image data and audio data may be processed by the GPU.

[0017] The main memory 20 is a volatile storage device that functions as a work area and a buffer area when the processor 19 executes processing. The auxiliary memory 21 is a non-volatile storage device and a non-transitory storage medium. Non-transitory applications are stored in the auxiliary memory 21. Information to be processed by the processor 19 is also stored in the auxiliary memory 21. The processor 19 runs the non-transitory applications to execute various processes. The processor 19 has a configuration for controlling the sound output from the speaker 23, a configuration for processing signals detected by the pressure sensor 22, a configuration for determining the operation of the operation unit 24, a configuration for processing information to be sent to the user terminal 12 via the communication device 25, and a configuration for processing information obtained from the user terminal 12 via the communication device 25.

[0018] A plurality of pressure sensors (pressure-sensitive devices) 22 are provided on the main body of the first target object 13. The plurality of pressure sensors 22 are arranged at different positions in a direction along the surface of the main body of the first target object 13. The pressure sensors 22 are, for example, capacitive, resistive, optical, or the like, and output a signal corresponding to the pressure received when a part of the user's 15 body comes into contact with them.

[0019] The speaker 23 is an electronic device that converts an electrical signal into sound and outputs it. The speaker 23 outputs sound contained in audio information generated by the management computer 11 and acquired via the user terminal 12. The operation unit 24 is composed of various switches, buttons, levers, touch panels, etc. provided on the body of the first target object 13. The user 15 can manually start and stop using the first target object 13 by operating the main switch of the operation unit 24. The user 15 can also output audio information from the speaker 23 by operating the operation unit 24. The communication device 25 is composed of a communication circuit, cable, antenna, etc. The communication device 25 is connected to the user terminal 12 via the network 17. The power source 26 may be either a DC power source or an AC power source. The DC power source may be either a primary battery or a secondary battery.

[0020] The second target object 14 has a main body, which is provided with a processor 28, a main memory 29, an auxiliary memory 30, a pressure sensor 31, a speaker 32, an operation unit 33, a communication device 34, a power supply 35, and an identification information display unit 72, as shown in FIG. 2 . The identification information is unique information that identifies the second target object 14, and includes, for example, a serial number, a two-dimensional code, etc. The electrical and electronic components provided in the main body function with power supplied from the power supply 35. The processor 28 is communicatively connected to the main memory 29, the auxiliary memory 30, the pressure sensor 31, the speaker 32, the operation unit 33, and the communication device 34 via a bus 36. The processor 28 is provided inside the main body and may be configured as a central processing unit (CPU) that integrates an arithmetic unit (arithmetic circuit) and a control device (control circuit).

[0021] Furthermore, instead of or in addition to a central processing unit, the processor 28 may be realized by an arithmetic processing circuit such as a digital signal processor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or a micro processing unit (MPU). Image data and audio data may be processed by the GPU.

[0022] The main memory 29 is a volatile storage device that functions as a work area and a buffer area when the processor 28 executes processing. The auxiliary memory 30 is a non-volatile storage device and a non-transitory storage medium. Non-transitory applications are stored in the auxiliary memory 30. The auxiliary memory 30 also stores information and various data to be processed by the processor 28. The processor 28 runs the non-transitory applications to execute various processes. The processor 28 has a configuration for controlling the sound output from the speaker 32, a configuration for processing signals detected by the pressure sensor 31, a configuration for determining the operation of the operation unit 33, a configuration for processing information to be sent to the user terminal 12 via the communication device 34, and a configuration for processing information obtained from the user terminal 12 via the communication device 34.

[0023] A plurality of pressure sensors (pressure-sensitive devices) 31 are provided on the body of the second target object 14. The plurality of pressure sensors 31 are arranged at different positions in a direction along the surface of the body of the second target object 14. The pressure sensors 31 may be, for example, capacitive, resistive, optical, or the like, and output a signal corresponding to the pressure received when a part of the body of the user 15 comes into contact with the pressure sensor 31.

[0024] The speaker 32 is an electronic device that converts an electrical signal into sound and outputs it. The speaker 32 outputs sound generated by the management computer 11 and contained in audio information acquired via the user terminal 12. The operation unit 33 is composed of various switches, buttons, levers, touch panels, etc. provided on the body of the second target object 14. The user 15 can manually switch between starting and ending use of the second target object 14 by operating the main switch of the operation unit 33. The user 15 can also output audio information from the speaker 32 by operating the operation unit 33. The communication device 34 is composed of a communication circuit, cable, antenna, etc. The communication device 34 is connected to the user terminal 12 via the network 18. The power source 35 is composed of a DC power source. The DC power source may be either a primary battery or a secondary battery.

[0025] (Description of User Terminal) The user terminal 12 is a computer used and operated by the user 15. The user terminal 12 is configured so that the user 15 can use it at both the first use location A1 and the second use location A2. In other words, the user terminal 12 is a portable computer that the user 15 can carry and move between locations. Portable computers include smartphones, tablet computers, notebook computers, etc.

[0026] The user terminal 12 includes a main body (casing), a processor 37, a main memory 38, an auxiliary memory 39, an operation unit 40, a display unit 41, a microphone 42, an audio output device 43, a position determination unit 44, a communication device 45, etc. The processor 37 is provided inside the main body and is configured by a central processing unit (CPU (Central Processing Unit)) that integrates an arithmetic unit (arithmetic circuit) and a control device (control circuit). The processor 37 is communicably connected to the main memory 38, the auxiliary memory 39, the operation unit 40, the display unit 41, the microphone 42, the audio output device 43, the image acquisition unit 70, the position determination unit 44, the communication device 45, etc. via a bus 46.

[0027] The main memory 38 is a volatile storage device that functions as a work area and a buffer area when the processor 37 executes processing. The auxiliary memory 39 is a non-volatile storage device and a non-transitory storage medium. Non-transitory applications are stored in the auxiliary memory 39. The applications stored in the auxiliary memory 39 include applications downloaded from the management computer 11 and installed.

[0028] The auxiliary memory 39 has a larger capacity than the main memory 38, and operates in response to input and output instructions from the processor 37. The auxiliary memory 39, which is a non-transitory storage medium, is realized by, for example, a magnetic disk, an optical disk, a flash memory, etc. An example of a magnetic disk is a hard disk drive. An example of an optical disk is a compact disk, a digital video disk, a Blu-ray disk, etc. A flash memory is a type of semiconductor memory, and examples of flash memory include an SD memory card, a USB flash drive, a solid state drive, etc. One or more elements included in the auxiliary memory 39 can be defined as a storage medium 39A that can be attached to and detached from the main body.

[0029] The operation unit 40 includes at least one element, for example, a liquid crystal display, an organic electroluminescence display, a touchpad, etc. This element has a display screen on which various buttons, tabs, etc. are displayed. The operation unit 40 may be configured with operation buttons and operation switches provided on the main body of the user terminal 12. By operating the operation unit 40, the user 15 can input various information to the user terminal 12, cause the user terminal 12 to execute various processes, and exchange various information between the user terminal 12 and the management computer 11.

[0030] Moreover, by operating the operation unit 40, the user 15 can exchange various information between the user terminal 12 and the first target object 13, exchange various information between the user terminal 12 and the second target object 14, etc. Furthermore, by operating the switching button on the operation unit 40, the user 15 can switch between starting and ending use of the first target object 13 and starting and ending use of the second target object 14, respectively.

[0031] The display unit 41 is a display that is viewed by the user 15, and the display may have a structure such as a liquid crystal display or an organic electroluminescence display. The display unit 41 displays various screens, information on websites on the network 16, etc. The display unit 41 can also display various buttons, tabs, the current time, etc. on the screen. In other words, the display unit 41 can also function as the operation unit 40.

[0032] The microphone 42 is a device that acquires voice uttered by the user 15, converts the voice into an electrical signal, and outputs the electrical signal. The audio output device 43 is an electronic device that converts the electrical signal into audio and outputs the audio, and is realized by a speaker, headphones, earphones, etc. The audio output device 43 is a device that outputs audio included in audio information acquired from the management computer 11. The audio output device 43 may be connected to the bus 46 via wired or wireless communication.

[0033] The image acquisition unit 70 captures an image of an object, converts the generated image into an electrical signal, and outputs it. The image acquisition unit 70 includes, for example, a camera, a scanner, a barcode reader, etc. The object captured by the image acquisition unit 70 includes scenery, people, documents, code information, etc. Code information includes one-dimensional codes and two-dimensional codes. One-dimensional codes include barcodes. Two-dimensional codes include QR Codes (registered trademark). Images include videos, still images, photographs, etc. The images acquired by the image acquisition unit 70 are stored in the auxiliary memory 39.

[0034] The position determination unit 44 is composed of, for example, a detection circuit and a determination circuit. The position determination unit 44 processes signals received from satellites via the communication device 45 and signals from a gyro sensor and a magnetic sensor provided in the user terminal 12. The position determination unit 44 determines the current location of the user terminal 12 and the movement route of the user terminal 12 based on the processing results and map information stored in the auxiliary memory 39. The current location and movement route of the user terminal 12 are the current location and movement route within the map information. By operating the operation unit 40, the user 15 can display the map information on the display unit 41 and identify the location of the first use location A1. Therefore, the position determination unit 44 can determine the location information of the user terminal 12, i.e., whether the user terminal 12 is being used at the first use location A1 or the second use location A2.

[0035] The position determination unit 44 can also process the communication status between the communication device 45 and multiple access points constituting the short-range communication system, and determine whether the user terminal 12 is located at the first use location A1 or the second use location A2 based on the processing result. Furthermore, the position determination unit 44 can indirectly determine whether the user terminal 12 is located at the first use location A1 or the second use location A2 based on the operation of the operation unit 40. For example, when the operation unit 40 is operated and use of the first target object 13 is started, the position determination unit 44 can determine that the user terminal 12 is located at the first use location A1. When use of the second target object 14 is started, the position determination unit 44 can determine that the user terminal 12 is located at the second use location A2. The position determination unit 44 may be integrated into the processor 37.

[0036] The communication device 45 includes a communication circuit, a communication cable, an antenna, etc. The communication device 45 is connected to the management computer 11 via the network 16, the communication device 45 is connected to the first target object 13 via the network 17, and the communication device 45 is connected to the second target object 14 via the network 18.

[0037] The processor 37 executes various processes by running applications stored in the auxiliary memory 39. The processes executed by the processor 37 include calculation, judgment, comparison, control, storage in the auxiliary memory 39, etc. The processes executed by the processor 37 include processing of reading information from the auxiliary memory 39, processing of information acquired from the management computer 11, processing of information acquired from the first target object 13, and processing of information acquired from the second target object 14.

[0038] The information acquired by the processor 37 from the first target object 13 includes first contact information determined from the signal of the pressure sensor 22. The information acquired by the processor 37 from the second target object 14 includes second contact information determined from the signal of the pressure sensor 31. The processing executed by the processor 37 includes processing of information to be sent to the management computer 11, processing of information to be sent to the first target object 13, processing of information to be sent to the second target object 14, processing of audio input to the microphone 42, processing of audio output from the audio output device 43, determining the operation performed on the operation unit 40, processing of the screen and information to be displayed on the display unit 41, etc.

[0039] (Description of the Management Computer) The management computer 11 may be managed and operated by an administrator or employee of a store (including a company, organization, enterprise, etc.) where the user 15 is a customer. The management computer 11 includes a main body (casing), a processor 47, a main memory 48, an auxiliary memory 49, a communication device 50, an operation unit 73, etc. The processor 47 may be provided inside the main body and configured as a central processing unit (CPU) that integrates an arithmetic unit (arithmetic circuit) and a control device (control circuit). The processor 47 is communicatively connected to the main memory 48, the auxiliary memory 49, the communication device 50, the operation unit 73, etc. via a bus 51. The processor 47 comprehensively controls other devices and circuits provided inside the main body and devices and circuits provided outside the main body. The processor 47 runs non-transitory applications to perform various processes based on instructions input by operating the operation unit 73, information and data obtained from the user terminal 12, etc. The processing performed by the processor 47 includes calculation, judgment, comparison, control, etc. The processing performed by the processor 47 includes processing to read information from the auxiliary memory 49, processing of information obtained via the network 16, addition, change, and update of information stored in the auxiliary memory 49, etc.

[0040] Furthermore, instead of or in addition to a central processing unit, the processor 47 may be realized by an arithmetic processing circuit such as a digital signal processor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or a micro processing unit (MPU). Image data and audio data may be processed by the GPU.

[0041] The main memory 48 is a volatile storage device that functions as a work area and a buffer area when the processor 47 executes processing. The auxiliary memory 49 is a non-volatile storage device and a non-transitory storage medium. Non-transitory applications are stored in the auxiliary memory 49. The applications include programs, configuration files, files for storing data, various libraries, etc. The auxiliary memory 49 also stores information used by the processor 47 to execute processing, information before being processed by the processor 47, and information after being processed by the processor 47. The auxiliary memory 49 also stores language models used by the artificial intelligence unit of the processor 47 to execute processing, information and data used by the artificial intelligence unit to perform machine learning, learning models and evaluation data used by the artificial intelligence unit, etc. The configuration and function of the artificial intelligence unit will be described later.

[0042] The auxiliary memory 49 has a larger capacity than the main memory 48, and operates in response to input and output instructions from the processor 47. The auxiliary memory 49 includes, for example, a magnetic disk, an optical disk, a flash memory, etc. Examples of magnetic disks include hard disk drives, and examples of optical disks include compact disks, digital video disks, and Blu-ray disks. Flash memory is a type of semiconductor memory, and examples of flash memory include SD memory cards, USB flash drives, and solid state drives. One or more elements included in the auxiliary memory 49 can be defined as a storage medium 49A that can be attached to and removed from the main body.

[0043] The communication device 50 includes devices, equipment, and standards that connect the management computer 11 to the network 16 via at least one of a wireless communication system and a wired communication system. Devices and equipment that connect the management computer 11 to the network 16 include communication circuits, communication cables, antennas, routers, etc. The operation unit 73 may be operated by a store manager or employee, etc. The operation unit 73 is realized by, for example, a mouse, keyboard, touch pen, touch panel, etc. The store manager or employee, etc. can input information, data, and instructions to the management computer 11 by operating the operation unit 73. The information, data, and instructions input by operating the operation unit 73 are stored in the auxiliary memory 49.

[0044] The processor 47 is configured to realize functional units such as a website management unit 54, a user information management unit 55, a target object information management unit 56, an interaction information management unit 57, an external information processing unit 58, an artificial intelligence unit 59, an audio information management unit 60, and a usage status determination unit 61 by running non-temporary applications.

[0045] The website management unit 54 is configured to manage the websites that the management computer 11 provides to the network 16, manage the screens provided to the websites, provide applications on the websites, and so on.

[0046] The user information management unit 55 processes user information acquired from the user terminal 12 and stores the user information in the auxiliary memory 49. The user information includes the user 15's name, gender, date of birth (birthday), age, address, telephone number, contract details, identification information for the first target object 13, identification information for the second target object 14, email address, password, and the user 15's interests. The contract details include items such as the "information provision timing" for providing voice information from the management computer 11 to the user terminal 12, the usage fee (cost) for the user terminal 12 to acquire the voice information, and the account for payment of the fee. The information provision timing refers to the timing for providing voice information from the management computer 11 to the user terminal 12, regardless of whether the management computer 11 acquires interaction information from the user terminal 12. The technical meaning of interaction information will be described later.

[0047] The timing of providing information may include timing each time the management computer 11 receives a request to acquire voice information from the user terminal 12, timing when the management computer 11 periodically provides voice information to the user terminal 12 based on a contract, etc. A usage fee is incurred each time the user terminal 12 acquires voice information from the management computer 11, or timing when the user terminal 12 periodically acquires voice information from the management computer 11. The timing of periodically providing voice information is managed by a calendar function.

[0048] The email address and password are an account for identifying different users. The interests of the users 15 include the hobbies of the users 15, the favorite foods of the users 15, the sports that the users 15 are interested in, the music that the users 15 are interested in, the movies that the users 15 are interested in, the tourist spots that the users 15 are interested in, the events that the users 15 are interested in, and the products that the store that operates the management computer 11 sells or gives as gifts to customers (users 15).

[0049] The target object information management unit 56 is configured to process information regarding the first target object 13 and the second target object 14 obtained from the user terminal 12, and to store the target object information regarding the first target object 13 and the second target object 14 in the auxiliary memory 49.

[0050] The target object information includes identification information of the first target object 13, a first use location A1 where the first target object 13 is expected to be used, identification information of the second target object 14, and a second use location A2 where the second target object 14 is expected to be used. The identification information of the first target object 13 and the identification information of the second target object 14 are unique information assigned to each of the target objects in order to distinguish between them. The use locations of the first target object 13 and the second target object 14 refer to locations suitable for the user 15 to use the first target object 13 and the second target object 14 and to interact with the first target object 13 and the second target object 14. The first use location A1 of the first target object 13 and the second use location A2 of the second target object 14 are different from each other.

[0051] The user 15 can operate the user terminal 12 to input that the first target object 13 will be used at the first use location A1 and that the second target object 14 will be used at the second use location A2, but does not input a specific example of the first use location A12 or the second use location A2. The target object information management unit 56 processes the target object information in association with the account of the user 15 and stores it in the auxiliary memory 49.

[0052] The interaction information management unit 57 is configured to acquire, from the user terminal 12, interaction information between the user 15 and the first target object 13, and interaction information between the user 15 and the second target object 14. The interaction information management unit 57 also processes the interaction information acquired from the user terminal 12, and stores the processed interaction information in the auxiliary memory 49 in association with the account of the user 15, the identification information of the first target object 13, and the identification information of the second target object 14, respectively.

[0053] The interaction information between the user 15 and the first target object 13 includes the date, time and duration of use when the user 15 used the first target object 13 at the first use location A1, the date, time and duration of use when the user 15 used the second target object 14 at the second use location A2, the voice input by the user 15 to the voice output device 43 of the user terminal 12, the first contact information obtained from the first target object 13, and the second contact information obtained from the second target object 14.

[0054] The interaction information management unit 57 can then determine the contact state of the user 15 with the second target object 14 by processing the signals of the pressure sensors 31 included in the first contact information. For example, when multiple pressure sensors 22 arranged in a predetermined direction sequentially detect an increase and decrease in pressure, the interaction information management unit 57 can determine that the user 15 has “stroked” the first target object 13. Furthermore, when the interaction information management unit 57 detects that the pressure received by a predetermined number of pressure sensors 22 has increased and that this pressure is being maintained, the interaction information management unit 57 can determine that the user 15 has “grabbed” the first target object 13. Furthermore, when the interaction information management unit 57 detects that the pressure received by a predetermined number of pressure sensors 22 has repeatedly increased and decreased, the interaction information management unit 57 can determine that the user 15 has “kneaded” the first target object 13. The contact state of the user 15 with the first target object 13 also corresponds to interaction information between the user 15 and the first target object 13.

[0055] Furthermore, the interaction information management unit 57 can determine the contact state of the user 15 with the second target object 14 by processing the signals of the pressure sensors 31 included in the second contact information. For example, when multiple pressure sensors 31 arranged in a predetermined direction detect an increase and decrease in pressure in sequence, the interaction information management unit 57 can determine that the user 15 has “stroked” the second target object 14. When the interaction information management unit 57 detects that the pressure received by a predetermined number of pressure sensors 31 has increased and that this pressure is being maintained, the interaction information management unit 57 can determine that the user 15 has “grabbed” the second target object 14. When the interaction information management unit 57 detects that the pressure received by a predetermined number of pressure sensors 31 has repeatedly increased and decreased, the interaction information management unit 57 can determine that the user 15 has “kneaded” the second target object 14. The contact state of the user 15 with the second target object 14 also corresponds to interaction information between the user 15 and the second target object 14.

[0056] Furthermore, the interaction information management unit 57 may have a configuration for determining the interaction level between the user 15 and the first target object 13 based on the interaction information, and a configuration for determining the interaction level between the user 15 and the second target object 14 based on the interaction information. The interaction information management unit 57 can determine that the interaction level between the user 15 and the first target object 13 and the interaction level between the user 15 and the second target object 14 have become relatively higher, respectively, as items such as, for example, the number of times the user 15 has used the first target object 13 and the second target object 14, the number of conversations and conversation durations the user 15 had with the first target object 13 and the second target object 14, the content of positive voices uttered by the user 15, the number of times the user 15 uttered positive voices, and the number of times the user 15 stroked the first target object 13 and the second target object 14 increase.

[0057] Furthermore, the interaction information management unit 57 can set the interaction level in stages, such as interaction level 1, interaction level 2, interaction level 3, interaction level 4, and interaction level 5. Here, the larger the Arabic numeral assigned to the interaction level, the higher the interaction level. A relatively high interaction level means that the degree of intimacy between the user 15 and the first target object 13 and the second target object 14 is relatively high. Furthermore, the interaction information management unit 57 may set different types of interaction levels, such as friend, best friend, lover, etc., based on the content of the conversation, the places where the user 15 goes out while carrying the second target object 14, etc. The interaction information management unit 57 stores various determination results and the setting results of various interaction levels in the auxiliary memory 49.

[0058] The external information processing unit 58 is configured to process external information acquired by the management computer 11 from the external computer 52 and store the processed external information in the auxiliary memory 49. The external information acquired by the management computer 11 from the external computer 52 includes external information used by the voice information management unit 60 to generate voice information, external information used by the interaction information management unit 57 to determine the interaction state between the user 15 and the first target object 13, and external information used to determine the interaction state between the user 15 and the second target object 14. The external information processing unit 58 acquires external information from the external computer 52 based on the address, age, gender, and interests of the user 15 included in the user information.

[0059] The artificial intelligence unit 59 creates history data and update data for the various information stored in the auxiliary memory 49 based on the various information acquired by the processor 47, the various information stored in the auxiliary memory 49, and the learning data, and stores the created data in the auxiliary memory 49. The artificial intelligence unit 59 realizes a learning stage and an inference stage. In the learning stage, i.e., the machine learning stage, the artificial intelligence unit 59 processes input learning target data using a learning model to obtain output data. The machine learning performed by the artificial intelligence unit 59 includes three types: supervised learning, unsupervised learning, and reinforcement learning.

[0060] In supervised learning, the output data resulting from the processing of the learning model is compared with the correct data, and the learning model is trained based on the error. Specifically, the weights and biases are adjusted to reduce the error. Unsupervised learning is a learning method that uses training data that does not contain the correct answer, and classifies the training data into groups of data with similar features (regularities, etc.). Reinforcement learning is a method that uses trial and error based on the output data of the learning model to acquire optimal guidelines for action, such as "this is what you should do in this case" to achieve a goal.

[0061] The artificial intelligence unit 59 can use a neural network as a learning model to realize machine learning. A neural network has an input layer, an intermediate layer (hidden layer), and an output layer. The intermediate layer is configured to convert input information into a form useful for the output layer. The artificial intelligence unit 59 can also perform deep learning to realize the neural network intermediate layer. In deep learning, the intermediate layer is multi-layered with multiple types of stages (nodes), and weighting and biasing are applied to functions (features) contained in the information. Deep learning automatically indicates the functions (features) to be focused on by the intermediate layer, thereby reducing noise in the information and data input to the learning model.

[0062] The artificial intelligence unit 59 is an artificial intelligence (AI) that includes transformers such as a Generative Pre-trained Transformer (GPT) and a Bidirectional Encoder Representations from Transformers (BERT), and a language model such as a recurrent neural network, and is capable of performing processing as a generative artificial intelligence.

[0063] In the inference stage, the artificial intelligence unit 59 processes the inference target data using a learning model to output an inference result. For example, when information such as the address of the user 15, the age of the user 15, the gender of the user 15, the interests of the user 15, the location where the user terminal 12 is used, information on interactions between the user 15 and the first target object 13, information on interactions between the user 15 and the second target object 14, and external information obtained from the external computer 52 is input as inference target data, the artificial intelligence unit 59 outputs output data as an inference result from the learning model. The output data output from the learning model is voice information to be sent to the user terminal 12.

[0064] The artificial intelligence unit 59 can generate voice information to be sent to the user terminal 12 so as to maintain a relatively high level of interaction, based on, for example, the address of the user 15, the age of the user 15, the gender of the user 15, the interests of the user 15, the location where the user terminal 12 is used, interaction information between the user 15 and the first target object 13, interaction information between the user 15 and the second target object 14, external information obtained from the external computer 52, etc. The voice information generated here includes conversations about events and tourist spots relatively close to the address of the user 15, and topics such as sports, movies, music, etc. that are estimated from the age, gender, and interests of the user 15.

[0065] Then, after the voice information obtained by processing the inference target data is sent to the user terminal 12, the artificial intelligence unit 59 creates evaluation data for the generated learning model based on whether or not the level of interaction between the user 15 and the first target object 13 has increased. Furthermore, the artificial intelligence unit 59 may update the learning model based on the created evaluation data.

[0066] The voice information management unit 60 is configured to process and generate voice information to be sent to the user terminal 12, process voice information obtained from the user terminal 12 and store the voice information in the auxiliary memory 49, etc. The voice information includes the voice itself, the conversation itself, user information associated with the voice and the conversation, and target object information associated with the voice and the conversation. The voice information sent to the user terminal 12 includes the voice output from the voice output device 43 of the user terminal 12, the voice output from the speaker 23 of the first target object 13, the voice output from the speaker 32 of the second target object 14, music, etc.

[0067] The voice information management unit 60 may generate voice information to be sent to the user terminal 12 based on the address, age, and gender of the user 15, the interests of the user 15, interaction information and interaction level between the user 15 and the first target object 13, interaction information and interaction level between the user 15 and the second target object 14, external information acquired from the external computer 52, etc. When generating the voice information, the voice information management unit 60 generates the voice information in cooperation with the user information management unit 55, the target object information management unit 56, the interaction information management unit 57, the external information processing unit 58, the artificial intelligence unit 59, the usage state determination unit 61, and the auxiliary memory 49, with the aim of relatively increasing the interaction level between the user 15 and the first target object 13. When generating the voice information, the voice information management unit 60 generates the voice information in cooperation with the artificial intelligence unit 59, with the aim of relatively increasing the interaction level between the user 15 and the second target object 14.

[0068] The voice information generated by the voice information management unit 60 includes replies and topics corresponding to the voice and conversations generated by the user 15, topics including the address, age, and gender of the user 15 and topics of interest of the user 15 included in the user information, topics related to external information obtained from the external computer 52, and conversations unrelated to the voice and topics generated by the user 15. The voice information management unit 60 can also separately generate voice information suitable for the first use location A1 and voice information suitable for the second use location A2. The voice information management unit 60 is configured to be able to include, as voice information suitable for the first use location A1, topics including the hobbies and interests of the user 15, replies to complaints and dissatisfaction of the user 15, etc.

[0069] In contrast, the voice information management unit 60 determines that voice information suitable for the second use location A2 is content that will not cause any problems if heard by a third party other than the user 15. For this reason, the voice information management unit 60 can include, in the voice information corresponding to the second use location A2, for example, replies to conversations with the user 15, topics related to the weather, topics related to the seasons, topics related to currently popular events, topics related to currently popular movies, topics related to currently popular music, topics related to tourist spots around the current location of the user terminal 12, etc. Note that the voice information management unit 60 does not include inappropriate content, such as topics containing personal information of the user 15 or topics related to complaints or dissatisfaction of the user 15, in the voice information corresponding to the second use location A2.

[0070] Furthermore, the voice information management unit 60 is configured to generate voice information based on contact information of the user 15 with the first target object 13 and the second target object 14. The voice information management unit 60 can generate voice information including conversations such as "thank you" and "happy" for contact information such as "stroked." The voice information management unit 60 can generate voice information including conversations such as "it hurts" and "stop" for contact information such as "kneaded." The voice information management unit 60 can generate voice information including conversations such as "it feels good" for contact information such as "kneaded."

[0071] Furthermore, the voice information management unit 60 may have a function to generate and acquire voice information, for example, auxiliary voice information, that is unrelated to the interaction information and interaction level between the user 15 and the first target object 13 and the interaction information and interaction level between the user 15 and the second target object 14. The voice information management unit 60 stores the generated and acquired auxiliary voice information in the auxiliary memory 49. The auxiliary voice information includes greetings spoken to the user 15, voice information related to the first target object 13 and the second target object 14, and store information. The store information includes the date and time of events held at the store, information on products sold or given as gifts by the store to customers, birthdays of store employees, work days of store employees, etc.

[0072] A store manager or employee can input store information to the management computer 11 by operating the operation unit 73 of the management computer 11. Furthermore, the store manager or employee may operate the operation unit 73 to set the timing for sending the store information to the user terminal 12 and the timing for outputting the store information sent to the user terminal 12 from one or more of the user terminal 12, the first target object 13, and the second target object 14. The information, data, and various timings input by operating the operation unit 73 are stored in the auxiliary memory 49.

[0073] The usage state determination unit 61 is configured to determine the usage state of each of the first target object 13, the second target object 14, the first use location A1, and the second use location A2. Specifically, the usage state determination unit 61 determines whether the user 15 is using the first target object 13 or the second target object 14, and also determines whether the user terminal 12 is actually being used at the first use location A1 or the second use location A2. Figure 3 is a diagram showing an example of determination made by the usage state determination unit 61.

[0074] An example of the determination made by the usage state determination unit 61 will be described. When the main switch of the first target object 13 is turned on and the main switch of the second target object 14 is turned off, the usage state determination unit 61 can determine that the user 15 is using the first target object 13. When the main switch of the first target object 13 is turned off and the main switch of the second target object 14 is turned on, the usage state determination unit 61 can determine that the user 15 is using the second target object 14.

[0075] The usage state determination unit 61 can determine that the user 15 is using the first target object 13 when the pressure sensor 22 is receiving pressure. The usage state determination unit 61 can determine that the user 15 is using the second target object 14 when the pressure sensor 31 is receiving pressure. The usage state determination unit 61 can determine that the user 15 is using the first target object 13 when the switching button of the operation unit 40 is operated to select the first target object 13. The usage state determination unit 61 can determine that the user 15 is using the second target object 14 when the switching button of the operation unit 40 is operated to select the second target object 14. Furthermore, the usage state determination unit 61 can determine whether the user terminal 12 is being used at the first use location A1 or the second use location A2 during the actual inspection based on the determination result of the position determination unit 44. The processor 47 implements a known clock function. The processor 47 also activates a calendar application stored in the auxiliary memory 49 to implement a calendar function.

[0076] (Example of Operation of Management System) An example of operation of the management system 10 is shown in Figure 4. For convenience, an example will be described in which the application used to operate the management system 10 is a native application. The native application is downloaded to the user terminal 12 and installed on the user terminal 12 for use.

[0077] In step S10, the management computer 11 manages voice information to be provided to the user terminal 12. In step S11, the management computer 11 also provides a website to the network 16. When the user 15 operates the user terminal 12 to run a web browser and access a website provided by the management computer 11, the user terminal 12 displays a usage screen on the display unit 41 in step S20. The usage screen displays available applications. The user 15 operates the user terminal 12 to select an application in step S21 and download the selected application. In step S21, the user 15 further operates the user terminal 12 to install the application.

[0078] The user 15 operates the user terminal 12 to run an application in step S22 and inputs user information and target object information to the user terminal 12. The user 15 then operates the user terminal 12 to send the user information and target object information to the management computer 11. In step S12, the management computer acquires and processes the user information and target object information from the user terminal 12, and stores the user information and target object information in the auxiliary memory 49 in association with each other.

[0079] In step S23, the user terminal 12 detects the location information of the user terminal 12, associates the location information with the account of the user 15, and transmits the location information to the management computer 11. In step S13, the management computer 11 acquires and processes the location information of the user terminal 12 and the account of the user 15 from the user terminal 12, associates the location information of the user terminal 12 with the account of the user 15, and stores the information in the auxiliary memory 49. Steps S23 and S13 are repeated at predetermined time intervals.

[0080] When the user 15 and the first target object 13 interact in step S30, the user terminal 12 acquires in step S24 interaction information about the interaction between the user 15 and the first target object 13. The interaction information between the user 15 and the first target object 13 includes identification information of the first target object 13, the date and time when the interaction between the user 15 and the first target object 13 occurred, the voice input by the user 15 to the voice output device 43 of the user terminal 12, and contact information detected by the pressure sensor 22 of the first target object 13.

[0081] Furthermore, when the user 15 and the second target object 14 interact in step S40, the user terminal 12 acquires in step S24 interaction information about the interaction between the user 15 and the second target object 14. The interaction information between the user 15 and the second target object 14 includes identification information of the second target object 14, the date and time when the interaction between the user 15 and the second target object 14 occurred, the voice input by the user 15 to the speaker 32 of the user terminal 12, and contact information detected by the pressure sensor 31 of the second target object 14.

[0082] In step S25, the user terminal 12 associates the interaction information with the account of the user 15 and transmits the associated information to the management computer 11. In step S14, the management computer 11 processes the interaction information and the account of the user 15 acquired from the user terminal 12, and stores the interaction information and the account of the user 15 in the auxiliary memory 49 in association with the location information of the user terminal 12. Furthermore, in step S15, the management computer 11 determines the usage status of the first target object 13, the second target object 14, the first use location A1, and the second use location A2.

[0083] In step S15, the management computer 11 further determines whether the use location of the user terminal 12 and the target object being used are consistent. Specifically, the management computer 11 determines that there is consistency if the user terminal 12 is actually used at the first use location A1 and the first target object 13 is being used. The management computer 11 determines that there is consistency if the user terminal 12 is actually used at the second use location A2 and the second target object 14 is being used.

[0084] In response to this, the management computer 11 determines that there is a mismatch if the user terminal 12 is actually used at the first use location A1 and the second target object 14 is also used. The management computer 11 also determines that there is a mismatch if the user terminal 12 is actually used at the second use location A2 and the first target object 13 is also used. If the management computer 11 determines that there is a mismatch in step S15, it sends that information to the user terminal 12.

[0085] When the user terminal 12 receives the information that the information is "not consistent" from the management computer 11 in step S26, the user terminal 12 executes one or more of the following processes: displaying the information on the display unit 41, or outputting the information as audio from the audio output device 43. When the user 15 recognizes the information that the information is "not consistent" on the user terminal 12, the user 15 performs an operation or takes an action to align the usage location of the user terminal 12 with the usage status of the first target object 13 and the second target object 14.

[0086] Meanwhile, after step S15, the management computer 11 generates voice information corresponding to the interaction information acquired from the user terminal 12 in step S16 and stores the generated voice information in the auxiliary memory 49. When the management computer 11 generates voice information in step S16, it generates voice information that is suited to the usage state. Specifically, the voice information is generated based on the interaction information of the interaction between the user 15 and the first target object 13 at the first use location A1 so as to relatively increase the interaction level. Furthermore, the voice information is generated based on the interaction information of the interaction between the user 15 and the second target object 14 at the second use location A2 so as to relatively increase the interaction level.

[0087] The voice information generated by the management computer 11 in step S16 has content corresponding to information about an interaction between the user 15 and either the first target object 13 or the second target object 14. For example, the voice information generated when an interaction occurs between the user 15 and the first target object 13 differs in content from the voice information generated when an interaction occurs between the user 15 and the second target object 14.

[0088] Furthermore, when the user 15 touches the first target object 13, the content of the generated voice information will differ depending on how the user 15 touches the first target object 13. Furthermore, when the user 15 touches the second target object 14, the content of the generated voice information will differ depending on how the user 15 touches the second target object 14. Note that if the user 15 and the first target object 13 interact at the second use location A2, or if the user 15 and the second target object 14 interact at the first use location A1, the management computer 11 does not generate voice information in step S16. The management computer 11 transmits the voice information generated in step S16 to the user terminal 12 in step S17.

[0089] In step S27 following step S26, the user terminal 12 acquires voice information from the management computer 11 and stores it in the auxiliary memory 39. If the voice information acquired from the management computer 11 includes voice information to be sent to the first target object 13, the user terminal 12 may transmit the voice information to the first target object 13 in step S28. If the first target object 13 acquires voice information in step S31, it can output voice from the speaker 23 in step S32. Furthermore, if the voice information acquired from the management computer 11 includes voice information to be sent to the second target object 14, the user terminal 12 may transmit the voice information to the second target object 14 in step S28. If the second target object 14 acquires voice information in step S41, it can output voice from the speaker 32 in step S42.

[0090] Instead of the process of step S28, the user terminal 12 may output audio information from the audio output device 43 in step S29. The management computer 11 may perform steps S10 and S11 simultaneously, or may perform step S10 after step S11.

[0091] Effect of the embodiment The management computer 11 disclosed in this embodiment can control the audio information including audio output from the speakers 23, 32 and the audio output device 43 based on the determination result of the target object being used and the determination result of the usage location where the target object being used is located. Therefore, the audio information including audio output from the speakers 23, 32 and the audio output device 43 can be made to correspond to the target object being used and the usage location where the target object being used is located, making it easier to match the usage state of the target object with the audio information including audio output from the speakers 23, 32 and the audio output device 43.

[0092] Furthermore, the management computer 11 determines whether the target object being used matches the use location where it is being used, and if it determines that the target object being used matches the use location where it is being used, it generates audio information including audio output from the speakers 23, 32 and the audio output device 43. Therefore, the management computer 11 can more easily match the use state of the target object with the audio information including audio output from the speakers 23, 32 and the audio output device 43.

[0093] Furthermore, when the management computer 11 determines that the target object being used does not match the usage location, it transmits information including the determination content to the user terminal 12. Therefore, the user 15 can recognize the situation in which the target object being used does not match the usage location.

[0094] Furthermore, the management computer 11 determines the target object being used based on the operation content of the operation unit 40 acquired from the user terminal 12. Therefore, the management computer 11 has an improved function of determining the target object being used.

[0095] Furthermore, the management computer 11 determines the target object being used based on the pressure information acquired from the user terminal 12. Therefore, the management computer 11 has an improved function of determining the target object being used.

[0096] Furthermore, the management computer 11 determines the usage location where the target object is located based on the determination content of the position determination unit 44 obtained from the user terminal 12. Therefore, the management computer 11 has an improved function of determining the usage location where the target object is being used.

[0097] Furthermore, when multiple target objects are provided and the user 15 uses the first target object 13 and the second target object 14, the audio output from the speaker 23 of the first target object 13, the speaker 32 of the second target object 14, and the audio output device 43 of the user terminal 12 can be switched depending on the location where the user 15 is located.

[0098] Furthermore, when the processing of steps S23 to S28 is repeated on the user terminal 12, or the processing of steps S23 to S27 and step S29 is repeated, the processing of steps S13 to S17 is repeated on the management computer 11, and the processing of steps S30 to S32 is repeated on the first target object, the interaction state between the user 15 and the first target object 13 changes.

[0099] The management computer 11 can change and evolve the audio information generated in step S16 in accordance with changes in the state of interaction between the user 15 and the first target object 13. Therefore, the audio output from the audio output device 43 of the user terminal 12 and the speaker 23 of the first target object 13 can be changed in accordance with the state of interaction between the user 15 and the first target object 13.

[0100] Furthermore, when the processes of steps S23 to S28 are repeated in the user terminal 12, or the processes of steps S23 to S27 and step S29 are repeated, the processes of steps S13 to S17 are repeated in the management computer 11, and the processes of steps S40 to S42 are repeated in the second target object 14, the interaction state between the user 15 and the second target object 14 changes. The management computer 11 can change and evolve the audio information generated in step S16 in accordance with changes in the interaction state between the user 15 and the second target object 14. Therefore, the audio output from the audio output device 43 of the user terminal 12 and the speaker 32 of the second target object 14 can be changed in accordance with the interaction state between the user 15 and the second target object 14, respectively.

[0101] (Another Specific Example 1) In another specific example, the user terminal 12 shown in Fig. 1 has the configuration and functions of the management computer 11. Furthermore, an application for the user terminal 12 to execute the processing shown in Fig. 4 is stored in advance in the auxiliary memory 39 of the user terminal 12. Furthermore, the user terminal 12 has the configuration and functions of the management computer 11. In other words, in the flowchart shown in Fig. 4, the user terminal 12 executes the processing, judgment, and control other than step S11 among the processing, judgment, and control executed by the management computer 11. Furthermore, the user terminal 12 can also determine the usage status shown in Fig. 3.

[0102] (Effects of Other Specific Example 1) The user terminal 12 can control the audio information, including the audio output from the speakers 23, 32 and the audio output device 43, based on the determination result of the target object being used and the determination result of the usage location where the target object being used is located. Therefore, the same effects as those described above can be obtained.

[0103] Furthermore, the user terminal 12 determines whether the target object being used matches the usage location where it is being used, and if it determines that the target object being used matches the usage location where it is being used, it generates audio information including audio to be output from the speakers 23, 32 and the audio output device 43. Therefore, it is possible to obtain the same effect as described above.

[0104] Furthermore, when the user terminal 12 determines that the target object being used and the usage location are inconsistent, the user terminal 12 transmits information including the determination content to the user terminal 12. Therefore, the same effect as described above can be obtained. Furthermore, the user terminal 12 determines the target object being used based on the operation content of the operation unit 40. Therefore, the user terminal 12 has an improved function of determining the target object being used. Furthermore, the user terminal 12 determines the target object being used based on pressure information acquired from the first target object 13 and the second target object 14. Therefore, the user terminal 12 has an improved function of determining the target object being used.

[0105] Furthermore, the user terminal 12 determines the usage location where the target object is located based on the determination made by the position determination unit 44. Therefore, the user terminal 12 has an improved function of determining the usage location where the target object is being used. Furthermore, when multiple target objects are provided and the user 15 uses a first target object 13 and a second target object 14, the audio output from the speaker 23 of the first target object 13, the speaker 32 of the second target object 14, and the audio output device 43 of the user terminal 12 can be switched depending on the usage location where the user 15 is located.

[0106] Furthermore, when the processing of steps S23 to S28 is repeated on the user terminal 12, or the processing of steps S23 to S27 and step S29 is repeated, the processing of steps S13 to S17 is repeated on the management computer 11, and the processing of steps S30 to S32 is repeated on the first target object, the interaction state between the user 15 and the first target object 13 changes.

[0107] The management computer 11 can change and evolve the audio information generated in step S16 in accordance with changes in the state of interaction between the user 15 and the first target object 13. Therefore, the audio output from the audio output device 43 of the user terminal 12 and the speaker 23 of the first target object 13 can be changed in accordance with the state of interaction between the user 15 and the first target object 13.

[0108] Furthermore, when the processes of steps S23 to S28 are repeated in the user terminal 12, or the processes of steps S23 to S27 and step S29 are repeated, the processes of steps S13 to S17 are repeated in the management computer 11, and the processes of steps S40 to S42 are repeated in the second target object 14, the interaction state between the user 15 and the second target object 14 changes. The user terminal 12 can change and evolve the audio information generated in step S16 in accordance with changes in the interaction state between the user 15 and the second target object 14. Therefore, the audio output device 43 of the user terminal 12 and the audio output from the speaker 32 of the second target object 14 can each be changed in accordance with the interaction state between the user 15 and the second target object 14.

[0109] (Other Operational Examples of the Management System) The operational example shown in Fig. 5 may be realized in the management system 10. The operational example of Fig. 5 is based on the premise that the operational example shown in Fig. 4 can be realized in the management system 10. An application that causes the management computer 11 to realize the processing shown in Fig. 5 is stored in the auxiliary memory 49 of the management computer 11. The management computer 11 may generate audio information in step S50. The audio information generated by the management computer 11 in step S50 includes auxiliary audio information unrelated to interaction information. The auxiliary audio information includes greetings to the user 15, reminders notifying the user 15 of event times, information including the user 15's interests, audio information related to the first target object 13 and the second target object 14, store information, etc.

[0110] Greetings issued to the user 15 include, for example, "good morning," "good evening," "good night," etc. These greetings may be output according to the time measured by the clock function of the user terminal 12. The time of the event may also be, for example, a seasonal or calendar-based event, the birthday of an employee of the store where the user 15 is a customer, a work day of an employee of the store where the user 15 is a customer, or a period during which products are sold or given as gifts to customers at a store managed and operated by the management computer 11.

[0111] Examples of store-side information include "The day after tomorrow is my birthday, so come to the store," "My movie will be released on XX month," "My stuffed animal will be released on XX day next week," etc. Seasonal and calendar-based events may be Christmas, cherry blossom viewing, Halloween, Tanabata, etc. The audio information related to the first target object 13 and the second target object 14 includes information about the release of new products for the first target object 13 and the second target object 14, information indicating the location and date and time when an event related to the first target object 13 and the second target object 14 will be held, audio information of different sound quality generated corresponding to each of the first target object 13 and the second target object 14, etc.

[0112] The management computer 11 may transmit the voice information generated in step S50 to the user terminal 12 in step S51. The management computer 11 may execute steps S50 and S51 based on the contract details included in the user information. If the contract details stipulate that voice information be provided to the user terminal 12 each time a request to acquire voice information is received from the user terminal 12, the management computer 11 executes steps S50 and S51 each time a request to acquire voice information is received from the user terminal 12. On the other hand, if voice information is to be provided to the user terminal 12 periodically based on the contract, steps S50 and S51 may be executed regardless of whether a request to acquire voice information is received from the user terminal 12.

[0113] In step S60, the user 15 may operate the operation unit 40 to input a request to acquire voice information, and may also transmit the request to acquire voice information from the user terminal 12 to the management computer 11. Note that in step S20, the user terminal 12 may read the identification information display unit 71 provided on the first target object 13 or the identification information display unit 72 provided on the second target object 14, and transmit the request to acquire voice information to the management computer 11.

[0114] In step S61, the user terminal 12 acquires, i.e., downloads, the audio information from the management computer 11 and stores the acquired audio information in the auxiliary memory 39. In step S62 following step S61, the user terminal 12 outputs the audio information from the audio output device 43. Note that the user terminal 12 may transmit the audio information to the first target object 13 and the second target object 14 in step S64.

[0115] The first target object 13 may acquire audio information from the user terminal 12 in step S70. The first target object 13 may output audio information from the speaker 23 when the operation unit 24 is operated in step S71 following step S70. The second target object 14 may acquire audio information from the user terminal 12 in step S80. The second target object 14 may output audio information from the speaker 32 when the operation unit 33 is operated in step S71 following step S80. Note that steps S63, S70, S71, S80, and S81 shown in FIG. 5 do not have to be executed.

[0116] (Supplementary Explanation) An example of the technical meaning disclosed in this embodiment is as follows. The management system 10 is an example of a management system. The management computer 11 is an example of a computer. The auxiliary memories 39, 49 are examples of memories. The processors 37, 47 are examples of processors. The user terminal 12 is an example of a user terminal. The operation unit 40 is an example of an operation unit. The communication device 45 is an example of a communication device. The position determination unit 44 is an example of a position determination unit. The first target object 13 and the second target object 14 are examples of target objects. The audio output device 43 and the speakers 23, 32 are each an example of an audio output device. The audio output device only needs to be communicably connected to the user terminal, and the user terminal itself may be equipped with an audio output device. The pressure sensors 22, 31 are each an example of a pressure sensor. The first use location A1 and the second use location A2 are examples of a plurality of different use locations.

[0117] 1 and 2 show a single user terminal 12 for convenience, but multiple user terminals 12 used by different users 15 can be connected to the management computer 11. Also, while a single first target object 13 and a single second target object 14 are shown in FIGS. 1 and 2, multiple first target objects 13 may be connected to the user terminal 12, and multiple second target objects 14 may be connected to the user terminal 12. Furthermore, three or more types of target objects may be provided, and three or more types of different usage locations may be set.

[0118] The application that runs the management system 10 may be a native application, a web application, or a hybrid application. A web application is an application that runs on a web browser. A hybrid application is an application that combines the functions of a web application and a native application. If the application that runs the management system 10 is a native application, it is stored in the auxiliary memory 39 of the user terminal 12. If the application that runs the management system 10 is a web application, it is stored in the auxiliary memory 49 of the management computer 11. Therefore, when the management system 10 realizes the operation example of FIG. 4, the information acquired by the user terminal 12 is processed and stored in the management computer 11 and then sent from the management computer 11 to the user terminal 12.

[0119] The management system 10 disclosed in this embodiment can also be understood as a voice management system. The management computer 11 can also be understood as a voice management computer. Furthermore, the application that realizes the operation example of Fig. 4 in the management system 10 can also be understood as a management application or a voice management application.

[0120] Furthermore, the management computer 11 may be configured as a single computer or may be configured as a plurality of distributed computers. The management computer 11 includes one or more computers selected from the group consisting of a server, a supercomputer, a mainframe, a workstation, etc. The application disclosed in this embodiment may also be understood as an application product.

[0121] The present disclosure can be used as a management system having multiple objects used by users, a management computer that manages multiple objects used by users, and an application that operates a system having multiple objects used by users.

[0122] 10...Management system, 11...Management computer, 12...User terminal, 13...First target object, 14...Second target object, 22, 31...Pressure sensor, 23, 32...Speaker, 40...Operation unit, 43...Sound output device, 44...Position determination unit, 45...Communication device, 37, 47...Processor, 39, 49...Auxiliary memory, A1...First usage location, A2...Second usage location

Claims

1. A management system comprising: a target object configured to be able to interact with a user; a user terminal used by the user and communicatively connected to the target object; an audio output device that outputs audio according to the user's interaction with the target object; and a management computer communicatively connected to the user terminal, storing information, and executing processing, wherein the target objects include a plurality of target objects each having a different configuration so that they can be used at a plurality of different use locations, and the management computer is configured to execute the following processes: storing in memory the configuration of each of the plurality of target objects and the respective use locations suitable for the plurality of target objects; determining which of the plurality of target objects is being used; determining in which use location the target object being used is located; and controlling audio information including audio output from the audio output device based on the determination result of the target object being used and the determination result of the use location where the target object being used is located.

2. A management system according to claim 1, wherein the management computer is configured to execute the following processes: determining whether the target object being used matches the usage location where it is being used; and, if it is determined that the target object being used matches the usage location where it is being used, generating audio information including audio to be output from the audio output device.

3. A management system as described in claim 2, wherein the management computer is configured to execute the process of transmitting information including the content of the determination to the user terminal when it determines that the target object being used and the usage location are inconsistent.

4. A management system as described in claim 1, wherein the user terminal has an operation unit that the user operates to select which of the plurality of target objects to use, and a communication device that transmits the operation content of the operation unit to the management computer, and the management computer is configured to execute a process to determine the target object being used based on the operation content of the operation unit obtained from the user terminal.

5. A management system as described in claim 1, wherein the plurality of target objects each have a pressure sensor that detects the pressure of contact with the user through interaction therewith, the user terminal is configured to acquire information on the pressure received by the pressure sensor from the plurality of target objects and transmit the acquired pressure information to the management computer, and the management computer is configured to execute a process to determine which of the target objects is being used based on the pressure information acquired from the user terminal.

6. A management system as described in claim 1, wherein the user terminal has a location determination unit that determines which of multiple usage locations indicated in map information the user terminal is located at, and a communication device that transmits the determination content of the location determination unit to the management computer, and the management computer is configured to execute a process to determine the usage location where the target object is located based on the determination content of the location determination unit obtained from the user terminal.

7. A management computer having a user terminal communicatively connected to a target object used by a user and an audio output device communicatively connected to output audio according to the user's interaction with the target object, a memory for storing information, and a processor for executing processing, wherein the target objects include a plurality of target objects each having a different configuration for use at a plurality of different use locations, and the processor is configured to execute the following processes: storing in memory the configuration of each of the plurality of target objects and the respective use locations suitable for each of the plurality of target objects; determining which of the plurality of target objects is being used; determining in which use location the target object being used is located; and controlling audio information including audio output from the audio output device based on the determination result of the target object being used and the determination result of the use location where the target object being used is located.

8. A management computer according to claim 7, wherein the processor is configured to generate audio information including audio to be output from the audio output device based on user information acquired from the user terminal.

9. A non-transient application that causes a management computer to execute processing in a management system comprising: a target object configured to be able to interact with a user; a user terminal that is used by the user and communicatively connected to the target object; an audio output device that outputs audio according to the user's interaction with the target object; and a management computer that is communicatively connected to the user terminal, stores information, and executes processing, wherein the target objects include a plurality of target objects that have different configurations so that they can be used at a plurality of different use locations, and the application is configured to cause the management computer to execute the following processing: a process of storing in memory the configurations of each of the plurality of target objects and the respective use locations suitable for each of the plurality of target objects; a process of determining which of the plurality of target objects is being used; a process of determining in which use location the target object being used is located; and a process of controlling audio information including audio output from the audio output device based on the determination result of the target object being used and the determination result of the use location where the target object being used is located.

10. A user terminal that is communicatively connected to a target object used by a user and an audio output device that outputs audio, and has a memory that stores information, and a processor that executes processing, wherein the target objects include a plurality of target objects that have different configurations so that they can be used in a plurality of different usage locations, and the processor is configured to execute the following processes: a process of storing in memory the configuration of each of the plurality of target objects and the usage locations suitable for each of the plurality of target objects; a process of determining which of the plurality of target objects is being used; a process of determining in which usage location the target object being used is located; and a process of controlling audio information including audio that is output from the audio output device based on the determination result of the target object being used and the determination result of the usage location where the target object being used is located.

11. A non-transient application that causes a user terminal that is communicatively connected to a target object used by a user and to an audio output device that outputs audio to execute processing, wherein the target objects include a plurality of target objects that have different configurations so that they can be used in a plurality of different usage locations, and the application is configured to cause the user terminal to execute the following processes: a process of storing the configuration of each of the plurality of target objects and the usage locations suitable for each of the plurality of target objects; a process of determining which of the plurality of target objects is being used; a process of determining in which usage location the target object being used is located; and a process of controlling audio information including audio that is output from the audio output device based on the determination result of the target object being used and the determination result of the usage location in which the target object being used is located.

Images