Spatial emotion estimation system and spatial emotion estimation method

Abstract

The wireless device collection control unit 101 collects, as wireless device information for each wireless device, the space ID set in the wireless device, the terminal ID of the user terminal that has communicated with the wireless device by short-range wireless communication, and the time of communication with the user terminal. The camera collection control unit 102 collects, as camera information for each camera, the space ID set in the camera, the user's emotion information captured by the camera, and the time of capture by the camera. The movement calculation control unit 103 calculates spatial movement information indicating that a user terminal with a specific terminal ID has moved from one space to another at a specified communication time. The change calculation control unit 104 calculates emotion change information indicating that the user's emotion information has changed when the user moved from one space to another at a specified capture time. The estimation control unit 105 estimates the relationship between the user's movement in space and changes in the user's emotion.

Description

[Technical Field]

[0001] The present invention relates to a spatial emotion estimation system and a spatial emotion estimation method. [Background technology]

[0002] Conventionally, there are technologies that use wireless devices (WiFi sensors or beacon terminals) or cameras to collect and analyze information about users, such as customers or employees, present in a given space. For example, Japanese Patent Application Laid-Open Publication No. 2017-60164 (Patent Document 1) discloses a street information processing system including a client and a server. Here, the client includes a street information collection unit that collects street information, a provision determination unit that determines whether the street information needs to be provided to the server, and a first transmission unit that transmits street information determined to need to be provided to the server. The server also includes an analysis unit that analyzes the street information, a determination unit that determines a client to which the information should be provided based on the analysis results, and a second transmission unit that transmits the information to the determined client. As a result, the system can be utilized in a variety of applications by actively utilizing clients connected to the server via a communication network and connected to microphones, sensors, cameras, etc.

[0003] Furthermore, Japanese Patent Application Laid-Open Publication No. 2017-68681 (Patent Document 2) discloses a service providing system including a storage means, a determination means, an acquisition means, a generation means, and a transmission means. The storage means stores personal attribute information of users using the service and communication connection information of multiple electrical appliances, and the determination means determines whether the users are located near the electrical appliances. When the acquisition means determines that a user is located near an electrical appliance, it acquires user status information from the electrical appliance, including the environment of the location where the determined target user is located and the target user's facial expression, behavior, voice, and physical condition at that location. Based on the user status information and personal attribute information, the generation means generates control information for controlling the operation of the electrical appliance so as to provide a sense of security and satisfaction to the target user located near the electrical appliance. The transmission means then transmits the control information to the electrical appliance based on the communication connection information. This reduces the burden on users of inputting operations while providing a sense of security to the users.

[0004] Japanese Patent Application Laid-Open Publication No. 2020-47044 (Patent Document 3) discloses a work management system including a beacon, a camera, a terminal, a storage processing device, and a search processing device. The beacons are installed in each area where workers work and transmit signals identifying the area. The cameras are installed in each area and photograph the workers within the area. The terminals are carried by the workers and have the function of receiving signals from the beacons. Based on image data from the cameras and information from the terminals that received the signals from the beacons, the storage processing device associates the image data of the workers with the area where the image was captured and the ID of the worker in the image, and stores the image data in an image database. The search processing device then accepts input of search parameters, including the worker's ID, searches the image database, and displays the image data as a result of the search, corresponding to the area where the image was captured. This allows managers to efficiently confirm the work content of the workers from the photographed data.

[0005] Furthermore, Japanese Patent Application Laid-Open Publication No. 2020-76596 (Patent Document 4) discloses an employee location movement display system that includes an employee's mobile terminal, a USB beacon communication unit, a control unit, and a display unit. Here, the USB beacon communication unit is capable of communicating with the mobile terminal and is provided in plurality, and the control unit determines the relationship between the plurality of USB beacon communication units and the mobile terminal. Furthermore, the display unit displays the employee's current location, and the control unit includes a processing unit that smoothly processes the display of the movement when displaying the movement of the mobile terminal on the display unit. This allows for smooth display of employee location movements, and is said to enable business improvements based on the employee's movement history.

[0006] Furthermore, Japanese Patent Application Laid-Open Publication No. 2021-33485 (Patent Document 5) discloses a location confirmation system comprising a beacon, a user portable mobile terminal, a server, and a location display terminal. Here, the beacon is installed indoors and emits a predetermined radio wave. When the user portable mobile terminal receives the radio wave from the beacon, the user portable mobile terminal or the server estimates the current location of the user portable mobile terminal within the indoors based on the beacon radio wave reception status. Furthermore, the browser on the location display terminal communicates with the server and displays the estimated location information of the user portable mobile terminal based on the beacon radio wave reception status of the user portable mobile terminal. This enables easy attendance management of work arrival and departure times without or with reduced user operation.

[0007] Furthermore, Japanese Patent Application Laid-Open Publication No. 2021-128370 (Patent Document 6) discloses an information provision system equipped with a photographing means and an information providing means. Here, the photographing means is capable of photographing the facial expression of the stroller occupant, and the information providing means provides information to the user via a mobile terminal. Furthermore, when a specific facial expression of the occupant is photographed by the photographing means, the information providing means provides the user with information indicating the facial expression and information on the stroller's location when the specific facial expression was photographed, in association with each other. This makes it possible to understand the condition of the stroller occupant.

[0008] Furthermore, Japanese Patent Application Laid-Open Publication No. 2021-128644 (Patent Document 7) discloses a cleaning management device including a receiving unit, a cleaner identifier detection unit, an inspector identifier detection unit, a determination unit, and a transmission unit. Here, the receiving unit is capable of receiving a cleaner beacon signal from a cleaner communication terminal carried by a cleaner entering a room and an inspector beacon signal from an inspector communication terminal carried by an inspector entering the room. The cleaner identifier detection unit detects the cleaner identifier included in the cleaner beacon signal, and the inspector identifier detection unit detects the inspector identifier included in the inspector beacon signal. The determination unit determines at least one status of the room: not cleaned, cleaning in progress, cleaning completed, inspection in progress, or inspection completed, and the transmission unit transmits the status. This is said to enable the completion of cleaning and inspection work to be managed with greater accuracy and reliability, while minimizing the effort and burden on workers.

[0009] Furthermore, Japanese Patent Laid-Open Publication No. 2023-37366 (Patent Document 8) discloses a work monitoring system including an identification information storage medium and a monitoring device. Here, the identification information storage medium is carried by the worker, stores medium identification information, and includes a beacon signal receiving unit, a work status determination unit, a work performance information transmitting unit, a work status determination unit, and a work performance information transmitting unit. The beacon signal receiving unit receives a beacon signal output from a beacon terminal provided for each piece of work equipment. The work status determination unit estimates the distance between the identification information storage medium and the beacon terminal based on the received beacon signal and determines whether the worker is working by comparing the distance with a predetermined threshold. When the work status determination unit determines that the worker is working, the work performance information transmitting unit transmits work performance information that associates the medium identification information with the beacon terminal identification information included in the received beacon signal. The monitoring device also includes a work performance information acquiring unit and a monitoring camera identifying unit, and the work performance information acquiring unit acquires the work performance information. The surveillance camera identification unit identifies, based on the work performance information, a surveillance camera that can capture footage of the worker from among multiple surveillance cameras installed within the work area, and outputs surveillance camera identification information that can identify the identified surveillance camera. This reduces the burden on the supervisor in terms of monitoring and improves monitoring efficiency.

[0010] Furthermore, Japanese Patent Laid-Open Publication No. 2023-42682 (Patent Document 9) discloses an advertising device including a first information acquisition means, a second information acquisition means, a linked information storage means, a user information integration means, an advertisement information creation means, and an advertisement information notification means. Here, the first information acquisition means associates and acquires first user information and first user identification information collected from a user terminal. Furthermore, the second information acquisition means associates and acquires second user information and second user identification information collected from the user terminal, and the associated information storage means associates and stores predetermined first user identification information, predetermined second user identification information, and user terminal identification information. Furthermore, the user information integration means integrates first user information associated with the predetermined first user identification information and second user information associated with the predetermined second user identification information, and the advertisement information creation means creates advertisement information based on analysis results of the integrated user information. Then, the advertisement information notification means notifies the advertisement information by specifying the user terminal identification information. This will allow the system to provide advertisements that are likely to be effective, taking into account the user's current situation and preferences. [Prior art documents] [Patent documents]

[0011] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-60164 [Patent Document 2] Japanese Patent Application Laid-Open No. 2017-68681 [Patent Document 3] Japanese Patent Publication No. 2020-47044 [Patent Document 4] Japanese Patent Publication No. 2020-76596 [Patent Document 5] Patent Publication No. 2021-33485 [Patent Document 6] Patent Publication No. 2021-128370 [Patent Document 7] Patent Publication No. 2021-128644 [Patent Document 8] Japanese Patent Application Publication No. 2023-37366 [Patent Document 9] Japanese Patent Application Publication No. 2023-42682 Summary of the Invention [Problem to be solved by the invention]

[0012] Wireless devices can specifically analyze a user's movement information (roaming information), and cameras can specifically analyze a user's emotional information, but most analyses are performed using only wireless devices or cameras, which means that both perspectives are lacking.

[0013] Here, by using both wireless devices and cameras installed in a specific space (location) and integrating the user's movement information and emotional information, it may be possible to estimate the characteristics of the space relative to the user.

[0014] The techniques described in the above Patent Documents 1 to 9 are specialized for analysis using a single camera or a single beacon terminal, and therefore cannot solve the above-mentioned problems.

[0015] Therefore, the present invention has been made to solve the above-mentioned problems, and has an object to provide a spatial emotion estimation system and a spatial emotion estimation method that are capable of estimating what kind of space led to a change in a user's emotion by using wireless terminals and cameras installed in each space. [Means for solving the problem]

[0016] The spatial emotion estimation system according to the present invention comprises a wireless device collection control unit, a camera collection control unit, a movement calculation control unit, a change calculation control unit, and an estimation control unit. The wireless device collection control unit uses a wireless device provided for each space to collect, for each wireless device, the space ID set for that wireless device, the terminal ID of a user terminal that has communicated with that wireless device via short-range wireless communication, and the time of communication with that user terminal as wireless device information. The camera collection control unit uses a camera provided for each space to collect, for each camera, the space ID set for that camera, emotion information of the user captured by that camera, and the time of capture by that camera as camera information. The movement calculation control unit arranges the space ID, the terminal ID, and the communication time of the collected wireless device information in chronological order, and calculates spatial movement information indicating that a user terminal with a specific terminal ID has moved from one space to another at a predetermined communication time. The change calculation control unit arranges the space ID of the collected camera information, the user's emotion information, and the shooting time in chronological order, and calculates emotion change information indicating a change in the user's emotion information when the user moves from one space to another at a predetermined shooting time. The estimation control unit arranges the calculated spatial movement information at the predetermined communication time and the calculated emotion change information at the predetermined shooting time in chronological order, and estimates the relationship between the user's movement in space and the change in the user's emotion.

[0017] The spatial emotion estimation method according to the present invention comprises a wireless device collection control step, a camera collection control step, a movement calculation control step, a change calculation control step, and an estimation control step. Each control step of the spatial emotion estimation method corresponds to each control unit of the spatial emotion estimation system. [Effects of the Invention]

[0018] According to the present invention, it is possible to estimate what kind of space led to a change in a user's emotion by using a wireless terminal and a camera installed in each space. [Brief explanation of the drawings]

[0019] [Figure 1]1 is a conceptual diagram and a functional block diagram of a spatial emotion estimation system according to an embodiment of the present invention.

[0023] FIG. [Figure 2] 1 is a flowchart showing the execution procedure of a spatial emotion estimation method according to an embodiment of the present invention. [Figure 3] FIG. 3A shows an example of a case where wireless devices and cameras are installed in multiple spaces, and FIG. 3B shows an example of a case where a user terminal enters a first space. [Figure 4] FIG. 4A shows an example of a wireless device information table when a user terminal enters a first space, and FIG. 4B shows an example of a camera information table when a user terminal enters a first space. [Figure 5] FIG. 5A shows an example of a case where a user terminal moves from a first space to a second space, and FIG. 5B shows an example of a wireless device information table when the user terminal enters the second space. [Figure 6] FIG. 6A shows an example of a camera information table when a user terminal enters a second space, and FIG. 6B shows an example of a camera information table when a user terminal moves from the second space to a third space. [Figure 7] FIG. 7A shows an example of a wireless device information table when a user terminal enters a third space, and FIG. 7B shows an example of a camera information table when a user terminal enters a third space. [Figure 8] 8A is a diagram showing an example of a wireless device information time-series table, and FIG. 8B is a diagram showing an example of a camera information time-series table. [Figure 9] FIG. 10 is a diagram illustrating an example of an estimation table. [Figure 10] FIG. 10A is a diagram showing an example of when an object is placed in or removed from the second space, and FIG. 10B is a diagram showing an example of when a person is placed in the second space. [Figure 11] FIG. 11A is a diagram showing an example of a space attribute information table and a user attribute information table, and FIG. 11B is a diagram showing an example of a wireless device information time-series table in which space attribute information and user attribute information are associated with each other. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings to help understand the present invention. Note that the following embodiment is an example of the present invention, and is not intended to limit the technical scope of the present invention.

[0021] 1, a spatial emotion estimation system 1 according to an embodiment of the present invention includes a user terminal 10, a server 11, a wireless device 12, a camera 13, and a network 14. In the spatial emotion estimation system 1, a wireless device 12 and a camera 13 are installed in each of a plurality of spaces (places).

[0022] Here, space means a space large enough for users to move around or stay in, and there is no particular limitation on the type of space, and it includes, for example, the interior and exterior of buildings, apartment buildings, stores, corridors, stations, subways, offices, restaurants, cafes, department stores, supermarkets, product sales areas, product shelves, cash registers, shopping malls, etc.

[0023] Furthermore, there is no particular limit to the number of spaces. For example, Figure 1 shows a case where there are three spaces, but this is not limited to this. As long as there is a space where wireless devices 12 and cameras 13 can be installed as a pair, the number of spaces can be increased to tens or hundreds.

[0024] The user terminal 10 includes, for example, a mobile terminal device (smartphone) with a touch panel, a tablet terminal device, a portable notebook computer, etc., and includes a display unit (output unit) that displays a screen (window, screen), a reception unit (input unit) that receives predetermined instructions input by a user's operation, a communication unit for wireless and wired communication, a storage unit that stores data, and a control unit that controls each unit. The communication unit of the user terminal 10 is capable of communicating with the wireless device 12 via short-range wireless communication such as Wi-Fi (registered trademark) or Bluetooth (registered trademark).

[0025] The server 11 is a commonly used computer or the like, and includes, for example, a communication unit for wireless and wired communication, a storage unit for storing data, and a control unit for controlling each unit. The server 11 mainly transmits and receives data from the wireless device 12 and the camera 13 via the network 14.

[0026] The wireless device 12 may include, for example, a Wi-Fi sensor, a Beacon terminal, or a combination thereof. The Wi-Fi sensor may utilize, for example, a wireless device at a public wireless LAN spot (Wi-Fi spot, free spot, hot spot).

[0027] Here, short-distance wireless communication means transmitting or receiving data to or from a wireless device 12 within a range of several tens of centimeters to several hundred meters using radio waves of a wireless signal. When the user terminal 10 enters the reception area (reception range) of the radio waves of the wireless device 12, it performs short-distance wireless communication with the wireless device 12.

[0028] Furthermore, camera 13 is generally a camera capable of continuous shooting, and examples thereof include an indoor camera, an outdoor camera, a camera capable of night shooting, a one-cable camera, a wireless surveillance camera, a security camera, a small camera, a terminal camera attached to a terminal device, etc. Camera 13 is installed at the top of a space, and examples thereof include a box-type camera, a pallet-type camera, a dome-type camera, an omnidirectional camera, etc., like a security camera. Camera 13 includes a shooting unit that shoots images (video) and videos, a communication unit for wireless and wired communication, and a control unit that controls each unit.

[0029] The camera 13 may also be an AI camera. An AI camera is equipped with artificial intelligence (AI) and automatically analyzes images and videos according to predetermined rules. The analysis content of an AI camera can include, for example, user face authentication, user skeletal structure estimation, user flow, and user congestion status. In an embodiment of the present invention, it is preferable to analyze user emotional information from user images and videos.

[0030] The network 14 is communicably connected to each of the server 11, the wireless device 12, and the camera 13. Here, the network 14 includes wireless communication networks such as Wi-Fi (registered trademark), a LAN (Local Area Network) via an access point, a WAN (Wide Area Network) via a wireless base station, a third generation (3G) communication method, a fourth generation (4G) communication method such as LTE, a fifth generation (5G) or later communication method, Bluetooth (registered trademark), and a specified low power wireless method.

[0031] The server 11, the wireless device 12, and the camera 13 each incorporate a CPU, ROM, RAM, SSD, etc. (not shown), and the CPU uses, for example, the RAM as a work area to execute programs stored in the ROM, SSD, etc. The CPU also executes programs to realize the functions of each control unit, which will be described later.

[0032] Next, the configuration and execution procedure according to an embodiment of the present invention will be described with reference to Figures 1 to 10. First, an analyst (administrator, estimator, etc.) using the spatial emotion estimation system 1 selects multiple spaces for which he or she wishes to estimate spatial characteristics for users, and installs a wireless device 12 and a camera 13 as a pair for each space.

[0033] Here, in the multiple spaces, there are installed objects and staff members in each space. For example, as shown in FIG. 3A , assume that three spaces (rooms) exist as estimation targets, with an acoustic device 301 installed as an installed object in the first space (N=1), a work storage tool 302 installed as an installed object in the second space (N=2), and a receptionist 303 existing as a staff member in the third space (N=3). In this case, for example, assume that the analyst installs the wireless device 12 and the camera 13 above the first space, facing the acoustic device 301 and overlooking the entrance to the first space, installs the wireless device 12 and the camera 13 above the second space, overlooking the storage tool 302, and installs the wireless device 12 and the camera 13 above the third space, overlooking the receptionist 303. In other words, the analyst installs the camera 13 in a position where it can capture an image of the user within the space.

[0034] The types of the above-mentioned installed objects and installers are not particularly limited, and are changed depending on the type of space. For example, if the space is the front desk of a building, the installed objects could be the lighting or objects at the front desk, and the installer could be a receptionist or building manager. If the space is an office room, the installed objects could be desks, chairs, filing cabinets, or work tools, and the installer could be an employee, business partner, or related party. If the space is a room in a store, the installed objects could be a product or service providing device, and the installer could be a seller or manufacturer. If the space is a restaurant, the installed objects could be food and drink, a dining table, or chairs, and the installer could be a chef or a customer.

[0035] Now, when the administrator completes the installation of the wireless devices 12 and cameras 13 for each space, the administrator turns on the power to the server 11 of the spatial emotion estimation system 1 to start the server 11. Then, the wireless device collection control unit 101 of the server 11 uses the wireless devices 12 provided for each space to collect, as wireless device information, for each wireless device 12, the space ID set in the wireless device 12, the terminal ID of the user terminal 10 that has communicated with the wireless device 12 by short-range wireless communication, and the time of communication with the user terminal 10 (FIG. 2: S101). Here, the communication time means the time of contact communication between the user terminal 10 and the wireless device 12. For example, when the user terminal 10 enters and leaves a space, the communication time means the time of contact start communication and the time of contact end communication between the user terminal 10 and the wireless device 12.

[0036] Here, there is no particular limitation on the collection method of the wireless device collection control unit 101. For example, as shown in Fig. 3B, when a user terminal 10 with a terminal ID "a01" enters a first space, it performs short-range wireless communication with a wireless device 12 in the first space. Then, the wireless device collection control unit 101 acquires the space ID "ab01" of the wireless device 12, the terminal ID "a01" of the user terminal 10, and the contact start communication time of the user terminal 10 via short-range wireless communication (for example, "2025 / 2 / 1 13:00").

[0037] Furthermore, when a user terminal 10 with a terminal ID of "a01" leaves the radio wave reception range of a wireless device 11 with a space ID of "ab01", the wireless device collection control unit 101 acquires the space ID "ab01" of the wireless device 12, the terminal ID "a01" of the user terminal 10, and the contact end communication time of the user terminal 10 via short-range wireless communication (for example, "2025 / 2 / 1 13:25").

[0038] Then, the wireless device collection control unit 101 creates a wireless device information table that associates the space ID of the wireless device 12, "ab01", the terminal ID of the user terminal 10, "a01", the contact start communication time ("2025 / 2 / 1 13:00"), and the contact end communication time ("2025 / 2 / 1 13:25").

[0039] Here, there is no particular limitation on the wireless device information table 400, but for example, as shown in Fig. 4A, the wireless device information table 400 stores a space ID 401, a terminal ID 402, and a detection time 403 in association with each other. The space ID 401 is identification information for identifying the space in which the wireless device 12 is installed, and can be, for example, setting information previously set in the MAC address of the wireless device 12. The terminal ID is identification information for identifying the user terminal 10, and can similarly be, for example, a MAC address.

[0040] Then, when the wireless device collection control unit 101 refers to the wireless device information table 400, it associates and stores the space ID 401 ("ab01"), the terminal ID 402 ("a01"), and the detection time 403 (contact start communication time "2025 / 2 / 1 13:00") in the referenced wireless device information table 400. The wireless device collection control unit 101 also associates and stores the space ID 401 ("ab01"), the terminal ID 402 ("a01"), and the detection time 403 (contact end communication time "2025 / 2 / 1 13:25") in the wireless device information table 400. This makes it possible to record the entry and exit of the user carrying the user terminal 10 in the first space as wireless device information.

[0041] Incidentally, there is no particular limitation on the method by which the wireless device collection control unit 101 acquires the space ID, the terminal ID, and the contact start communication time or the contact end communication time. For example, when the wireless device 12 is a Wi-Fi sensor, the user terminal 10 normally periodically transmits radio waves (beacons, omnidirectional radio waves) to search for access points for wireless LAN (Local Area Network) communication. These radio waves contain the terminal ID ("a01") of the user terminal 10. Therefore, when the user terminal 10 enters the radio wave reception range of the Wi-Fi sensor 12, the Wi-Fi sensor 12 receives the radio waves from the user terminal 10 and transmits the space ID ("ab01") for identifying the Wi-Fi sensor 12, the terminal ID ("a01") in the radio waves, and the contact start communication time "2025 / 2 / 1 13:00" to the server 11 via the network 14. At this time, radio wave intensity may also be transmitted. Furthermore, when the user terminal 10 leaves the radio wave reception range of the Wi-Fi sensor 12, the Wi-Fi sensor 12 stops receiving radio waves from the user terminal 10, and transmits the space ID ("ab01"), the terminal ID ("a01") whose radio wave reception has stopped, and the contact end communication time ("2025 / 2 / 1 13:25") to the server 11 via the network 14. Note that the Wi-Fi sensor 12 may also identify the contact end communication time by detecting that the user terminal 10 has stopped receiving radio waves.

[0042] On the other hand, when the wireless device 12 is a Beacon terminal, the Beacon terminal 12 periodically transmits radio waves for short-range wireless communication. These radio waves include a space ID ("ab01") for identifying the Beacon terminal 12. When the user terminal 10 enters the radio wave reception range of the Beacon terminal 12, the user terminal 10 receives the radio waves from the Beacon terminal 12 and transmits the space ID ("ab01") in the radio waves, the terminal ID ("a01") of the user terminal 10, and the contact start communication time ("2025 / 2 / 1 13:00") to the server 11 via the network 14. In this case, a specific program (application, etc.) that (automatically) transmits information from the Beacon terminal 12 to the server 11 may be installed in the user terminal 10 in advance. Furthermore, when the user terminal 10 leaves the radio wave reception range of the Beacon terminal 12, the user terminal 10 stops receiving radio waves from the Beacon terminal 12, and transmits the space ID ("ab01") where radio wave reception stopped, the terminal ID ("a01"), and the contact end communication time ("2025 / 2 / 1 13:25") to the server 11 via the network 14. Note that the user terminal 10 may also identify the contact end communication time by detecting that reception of radio waves from the Beacon terminal 12 has stopped.

[0043] In this way, regardless of whether the wireless device 12 is a Wi-Fi sensor or a Beacon terminal, the wireless device collection control unit 101 of the server 11 can receive and acquire the space ID, terminal ID, and communication time (contact start communication time, contact end communication time) from one wireless device 12 via the network 14 by short-range wireless communication with a user terminal 10 located in the vicinity of the wireless device 12.

[0044] Now, when the wireless device collection control unit 101 collects wireless device information, the camera collection control unit 102 of the server 11 uses the cameras 13 provided for each space to collect, for each camera, the space ID set for the camera 13, the emotional information of the user photographed by the camera 13, and the time of photographing by the camera 13 as camera information (FIG. 2: S102).

[0045] Here, there is no particular limitation on the collection method of the camera collection control unit 102. For example, as shown in Fig. 3B, when a user terminal 10 with a terminal ID of "a01" enters a first space, the camera 13 in the first space starts up at a predetermined timing and captures an image of the user carrying the user terminal 10. Then, the camera collection control unit 102 acquires the space ID "ab01" preset for the camera 13, the image (or video) of the user captured by the camera 13, and the capture time of the camera 13 (for example, "2025 / 2 / 1 13:20").

[0046] Next, the camera collection control unit 102 calculates user emotion information estimated from the user's facial expression based on the acquired user image. For example, as shown in FIG. 4B , when the camera collection control unit 102 acquires a user image P1, it detects the user's facial expression from the user image P1 and calculates the degree of the user's emotion (e.g., joy, trust, fear, surprise, sadness, disgust, anger, expectation, etc.) from the detected user expression. Here, the degree of each emotion is set, for example, within a range of 0-100, and the sum of the degrees of all emotions becomes 100. Here, for example, since the audio device 301 is present in the first space, the user's emotions improve upon entering the first space, and the degree of motivation is calculated to be "30" and the degree of inspiration is calculated to be "10." Note that the emotion information is expressed as the degree of each emotion.

[0047] Then, the camera collection control unit 102 creates a camera information table that associates the space ID "ab01" of the camera 13 with the user's emotional information ("Motivation: 30", "Inspiration: 10", etc.), and the shooting time ("2025 / 2 / 1 13:20").

[0048] Here, there are no particular limitations on the camera information table 404, but for example, as shown in Fig. 4B, the camera information table 404 stores a space ID 405, emotion information 406 (such as "Motivation" 406a and "Inspiration" 406b), and a detection time 407 in association with each other. The space ID 405 is identification information for identifying the space in which the camera 13 is installed, and may be, for example, setting information that has been set in advance in the camera 13.

[0049] Then, the camera collection control unit 102 refers to the camera information table 404, and stores the space ID 405 ("ab01"), the degree of "Motivation" 406a of "30" and the degree of "Inspiration" 406b of "10" in the emotion information 406, and the detection time 407 (photographing time "2025 / 2 / 1 13:20") in association with each other in the referenced camera information table 404. This makes it possible to record the user's emotions in the first space as camera information.

[0050] Now, when the camera collection control unit 102 completes the collection of camera information, the movement calculation control unit 103 of the server 11 refers to the wireless device information table 400 and the camera information table 404 and determines whether a predetermined amount of data has been accumulated (Figure 2: S103).

[0051] If the result of the determination is that the predetermined amount of data has not been accumulated (FIG. 2: S103 NO), the movement calculation control unit 103 determines that it is not the timing for estimation, returns to S101, and repeats accumulating data in the wireless device information table 400 and the camera information table 404.

[0052] For example, as shown in FIG. 5A, when a user terminal 10 with a terminal ID "a01" leaves a first space and enters a second space, the wireless device collection control unit 101 uses a wireless device 12 in the second space to collect, as wireless device information, the space ID set in the wireless device 12, the terminal ID of the user terminal 10 that has communicated with the wireless device 12 by short-range wireless communication, and the time of communication with the user terminal 10 (FIG. 2: S101).

[0053] Here, the wireless device collection control unit 101 performs short-range wireless communication with the wireless device 12 in the second space, and the wireless device collection control unit 101 acquires the space ID "ab02" of the wireless device 12, the terminal ID "a01" of the user terminal 10, and the contact start communication time of the user terminal 10 via short-range wireless communication (e.g., "2025 / 2 / 1 14:00"). Furthermore, when the user terminal 10 with the terminal ID "a01" leaves the radio wave reception range of the wireless device 12 with the space ID "ab02", the wireless device collection control unit 101 acquires the space ID "ab02" of the wireless device 12, the terminal ID "a01" of the user terminal 10, and the contact end communication time of the user terminal 10 via short-range wireless communication (e.g., "2025 / 2 / 1 14:15").

[0054] 5B, the wireless device collection control unit 101 refers to the wireless device information table 400 and stores the space ID 401 ("ab02"), the terminal ID 402 ("a01"), and the detection time 403 (contact start communication time "2025 / 2 / 1 14:00") in the referenced wireless device information table 400 in association with each other. The wireless device collection control unit 101 also stores the space ID 401 ("ab02"), the terminal ID 402 ("a01"), and the detection time 403 (contact end communication time "2025 / 2 / 1 14:15") in the wireless device information table 400 in association with each other. This makes it possible to record the entry and exit of the user in the second space as wireless device information.

[0055] In addition, the camera collection control unit 102 uses the camera 13 installed in the second space to collect the space ID set in the camera 13, the emotional information of the user photographed by the camera 13, and the time of photographing by the camera 13 as camera information (Figure 2: S102).

[0056] For example, as shown in FIG. 5B, when a user terminal 10 with a terminal ID of "a01" enters the second space, the camera 13 in the second space starts up at a predetermined timing to capture an image of the user carrying the user terminal 10, and the camera collection control unit 102 acquires the space ID "ab02" previously set for the camera 13, the image of the user captured by the camera 13, and the time of capture by the camera 13 (e.g., "2025 / 2 / 1 14:10").

[0057] Next, the camera collection control unit 102 calculates the user's emotional information estimated from the user's facial expression based on the acquired image of the user. For example, as shown in FIG. 6A, when the camera collection control unit 102 acquires image P2 of the user, it calculates the level of the user's emotional state from image P2 of the user. Here, for example, since storage equipment 302 is present in the second space, the user's emotional state becomes negative upon entering the second space, and the level of motivation "Motivation" is calculated to be "10" and the level of inspiration "Inspiration" is also calculated to be "10".

[0058] Then, the camera collection control unit 102 references the camera information table 404, and as shown in Fig. 6A, stores the space ID 405 ("ab02"), the degree of "10" for "Motivation" 406a and the degree of "10" for "Inspiration" 406b of emotion information 406, and the detection time 407 (photographing time "2025 / 2 / 1 14:10") in association with each other in the referenced camera information table 404. This makes it possible to record the user's emotion in the second space as camera information.

[0059] Now, the movement calculation control unit 103 refers to the wireless device information table 400 and the camera information table 404 at a predetermined timing, and determines whether or not a predetermined amount of data has been accumulated (FIG. 2: S103).

[0060] If the result of the determination is that sufficient data has not yet been accumulated (FIG. 2: S103 NO), the movement calculation control unit 103 determines that it is not time for estimation, and returns to S101, as described above, to repeat the accumulation of data in the wireless device information table 400 and the camera information table 404.

[0061] Here, for example, as shown in FIG. 6B, when a user terminal 10 with a terminal ID of "a01" leaves the second space and enters a third space, the wireless device collection control unit 101 then uses a wireless device 12 in the third space to collect, as wireless device information, the space ID set in the wireless device 12, the terminal ID of the user terminal 10 that has communicated with the wireless device 12 by short-range wireless communication, and the time of communication with the user terminal 10 (FIG. 2: S101).

[0062] Here, the wireless device collection control unit 101 performs short-range wireless communication with the wireless device 12 in the third space, and the wireless device collection control unit 101 acquires the space ID "ab03" of the wireless device 12, the terminal ID "a01" of the user terminal 10, and the contact start communication time of the user terminal 10 via short-range wireless communication (e.g., "2025 / 2 / 1 15:00"). Furthermore, when the user terminal 10 with the terminal ID "a01" leaves the radio wave reception range of the wireless device 12 with the space ID "ab03", the wireless device collection control unit 101 acquires the space ID "ab03" of the wireless device 12, the terminal ID "a01" of the user terminal 10, and the contact end communication time of the user terminal 10 via short-range wireless communication (e.g., "2025 / 2 / 1 15:40").

[0063] 7A, the wireless device collection control unit 101 refers to the wireless device information table 400 and stores the space ID 401 ("ab03"), the terminal ID 402 ("a01"), and the detection time 403 (contact start communication time "2025 / 2 / 1 15:00") in the referenced wireless device information table 400 in association with each other. The wireless device collection control unit 101 also stores the space ID 401 ("ab03"), the terminal ID 402 ("a01"), and the detection time 403 (contact end communication time "2025 / 2 / 1 15:40") in the wireless device information table 400 in association with each other. This makes it possible to record the entry and exit of the user in the third space as wireless device information.

[0064] In addition, the camera collection control unit 102 uses a camera 13 installed in a third space to collect the space ID set in the camera 13, the emotional information of the user photographed by the camera 13, and the time of photographing by the camera 13 as camera information (FIG. 2: S102).

[0065] For example, as shown in FIG. 6B, when a user terminal 10 with a terminal ID of "a01" enters a third space, the camera collection control unit 102 determines that since a reception desk 303 exists in the third space, the user who has entered the third space interacts with the reception desk 303, the user's mood improves, and the degree of motivation "Motivation" is calculated to be "10" and the degree of inspiration "Inspiration" is calculated to be "30".

[0066] Then, the camera collection control unit 102 references the camera information table 404, and as shown in Fig. 7B, stores the space ID 405 ("ab03"), the degree of "10" for "Motivation" 406a and the degree of "30" for "Inspiration" 406b of the emotion information 406, and the detection time 407 (photographing time "2025 / 2 / 1 15:15") in association with each other in the referenced camera information table 404. This makes it possible to record the user's emotion in the third space as camera information.

[0067] Now, the movement calculation control unit 103 refers to the wireless device information table 400 and the camera information table 404 at a predetermined timing, and determines whether or not a predetermined amount of data has been accumulated (FIG. 2: S103).

[0068] If the result of the determination is that sufficient data has been accumulated (FIG. 2: S103 YES), the movement calculation control unit 103 determines that it is time for estimation, and calculates spatial movement information indicating that a user terminal 10 with a specific terminal ID has moved from one space to another at a specified communication time (FIG. 2: S104).

[0069] Here, there is no particular limitation on the calculation method of the movement calculation control unit 103. For example, the movement calculation control unit 103 refers to the wireless device information table 400, collects the space ID 401 and the detection time 403 for each terminal ID 402 from the referred wireless device information table 400, and generates a wireless device information time-series table 800 for each terminal ID 402 in which the space IDs 401 are arranged in chronological order based on the collected detection times 403.

[0070] For example, the movement calculation control unit 103 collects the space ID 401 and the detection time 403 associated with a specific (identical) terminal ID 402 (e.g., "a01"), and generates a wireless device information time-series table 800 in which the space ID 401 is arranged in chronological order for the specific terminal ID 402. Here, as shown in FIG. 8A, the wireless device information time-series table 800 stores the detection time 802, the space ID 803, and the spatial movement information 804 associated with the specific terminal ID 801 ("a01"). The movement calculation control unit 103 then references the space ID 803 in the wireless device information time-series table 800 in chronological order, and determines whether the space ID 803 at the time of reference has changed compared to the space ID 803 immediately before the reference.

[0071] If the determination result shows that the space ID 803 at the time of reference has not changed compared to the space ID 803 immediately before reference, for example, if the space ID 803 immediately before reference does not exist for the initially referenced space ID 803, or if the space ID 803 at the time of reference (e.g., "ab01") is the same as the space ID 803 immediately before reference (e.g., "ab01"), the movement calculation control unit 103 determines that the user terminal 10 with a specific terminal ID 801 ("a01") is staying in the space with space ID 803 ("ab01"). In this case, the movement calculation control unit 103 does not store anything in the spatial movement information 804 (leaving it blank), and makes the above-mentioned determination by referring to the next space ID 803 from the space ID 803 at the time of reference.

[0072] On the other hand, if the space ID 803 at the time of reference has changed compared to the space ID 803 immediately before reference, for example, if the space ID 803 at the time of reference (e.g., "ab02") is different from the space ID 803 immediately before reference (e.g., "ab01"), the movement calculation control unit 103 determines that the user terminal 10 with a specific terminal ID 801 ("a01") has moved from the space with the space ID 803 ("ab01") immediately before reference to the space with the space ID 803 ("ab02") at the time of reference. In this case, the movement calculation control unit 103 calculates spatial movement information (e.g., "ab01->ab02") indicating movement from the space with the space ID 803 ("ab01") immediately before the reference to the space with the space ID 803 ("ab02") at the time of the reference, and stores this spatial movement information ("ab01->ab02") in association with the spatial movement information 804 in the wireless device information time-series table 800 with respect to the detection time 802 at the time of the reference ("2025 / 2 / 1 14:00"). This makes it possible to calculate the spatial movement information of the user terminal 10 with a specific terminal ID ("a01") from the wireless device information table 800. Note that after the movement calculation control unit 103 calculates the spatial movement information, if a next space ID 803 exists from the space ID 803 at the time of the reference, the movement calculation control unit 103 refers to the next space ID 803 and makes the above-mentioned determination.

[0073] In this way, the movement calculation control unit 103 repeats the above-mentioned judgment in time series for all spatial IDs 803 in the wireless device information time series table 800, thereby being able to calculate spatial movement information for the user terminal 10 with a specific terminal ID ("a01") at the point when sufficient data has been accumulated.

[0074] Here, as shown in FIG. 8A, for a user terminal 10 with a specific terminal ID ("a01"), the detection time 802 is calculated to be "2025 / 2 / 1 14:00" and the spatial movement information 804 is calculated to be "ab01->ab02", and the detection time 802 is calculated to be "2025 / 2 / 1 15:00" and the spatial movement information 804 is calculated to be "ab02->ab03".

[0075] Once the movement calculation control unit 103 has completed the calculation, the change calculation control unit 104 of the server 11 then arranges the collected space ID of the camera 13, the user's emotional information, and the shooting time in chronological order, and calculates emotion change information indicating that the user's emotional information has changed when the user moved from one space to another at a specific shooting time (FIG. 2: S105).

[0076] Here, there are no particular limitations on the calculation method of the change calculation control unit 104. For example, the change calculation control unit 104 references the camera information table 404, and in the referenced camera information table 404, arranges the emotion information 406 and the shooting time 407 for each space ID 405 in chronological order to generate a camera information time-series table 805.

[0077] Here, when consecutive space IDs 405 are arranged in chronological order, if there are consecutive identical space IDs 405, then multiple different emotion information 406 will be associated with different detection times 407 for the same space IDs 405 arranged consecutively in chronological order. Accordingly, change calculation control section 104 calculates average emotion information 406 by averaging the multiple pieces of emotion information 406 for the same space IDs 405 arranged consecutively in chronological order, and calculates average detection time 407 by averaging the multiple detection times 407, and sets these as average emotion information 406 and average detection time 407 for the same space ID 405. In this way, even if multiple pieces of emotion information 406 and multiple detection times 407 are associated with the same space ID 405 arranged consecutively in chronological order, it is possible to group together emotion information 406 and detection times 407 for the same space ID 405.

[0078] As described above, emotion information 406 is expressed as the degree of each emotion, so when change calculation control unit 104 averages multiple pieces of emotion information 406, it can, for example, add up the degrees of specific emotions and divide the total by the number of pieces of emotion information 406 to calculate the quotient as the degree of the specific emotion.

[0079] Here, in the above description, the space IDs 405 arranged in chronological order in the camera information table 404 are all different space IDs, so the change calculation control unit 104 arranges the space IDs 405, emotion information 406, and shooting times 407 from the camera information table 404 in chronological order as they are, to generate a camera information chronological table 805.

[0080] 8B, camera information time-series table 805 stores space ID 806, detection time 807, emotion information 808, and emotion change information 809 in association with each other. Here, emotion information 808 is expressed as a degree of each emotion, such as the degree of motivation "Motivation" 808a (for example, "30") and the degree of emotion "Inspiration" 808b (for example, "10"). Change calculation control unit 104 then references emotion information 808 in camera information time-series table 805 for each different space ID 806 along the time series, and determines whether emotion information 808 at the time of reference has changed compared with emotion information 808 immediately before reference.

[0081] If the result of this determination is that emotion information 808 at the time of reference has not changed compared with emotion information 808 immediately before reference, for example, if emotion information 808 immediately before reference does not exist for emotion information 808 initially referenced, or if emotion information 808 at the time of reference is the same as emotion information 808 immediately before reference, change calculation control unit 104 determines that the emotion will not change when the user moves between spaces. In this case, change calculation control unit 104 does not store anything in emotion change information 809 (leaving it blank), and makes the above determination by referring from emotion information 808 of a given space ID 806 at the time of reference to emotion information 808 of the next space ID 806.

[0082] Here, there are no particular limitations on the method by which change calculation control section 104 determines whether emotion information 808 at the time of reference is the same as emotion information 808 immediately before reference, but for example, as described above, when emotion information 808 is expressed as the degree of each emotion, it is rare for the degree of each emotion at the time of reference to match the degree of each emotion immediately before reference, and therefore change calculation control section 104, for example, calculates a subtraction value for each emotion by subtracting the degree of emotion immediately before reference from the degree of emotion at the time of reference, and determines for each emotion whether the absolute value of the subtraction value exceeds a prescribed threshold value (for example, "10").

[0083] If the result of this determination is that the absolute value of the subtraction value does not exceed a predetermined threshold for any emotion, then change calculation control unit 104 determines that emotion information 808 at the time of reference is the same as emotion information 808 immediately before reference. On the other hand, if the absolute value of the subtraction value exceeds a predetermined threshold for any emotion, change calculation control unit 104 determines that emotion information 808 at the time of reference is not the same as emotion information 808 immediately before reference. In this way, even if emotion information 808 is expressed as the degree of each emotion, it is possible to determine whether or not an emotion has changed when the user moves around in space.

[0084] On the other hand, if emotion information 808 at the time of reference has changed compared to emotion information 808 immediately before reference, for example, if emotion information 808 at the time of reference (level of motivation "Motivation" 808a: "10") has changed compared to emotion information 808 immediately before reference (level of motivation "Motivation" 808a: "30"), change calculation control section 104 determines that the emotion has changed when the user moved within the space. In this case, the change calculation control unit 104 calculates emotion change information (for example, "Motivation -20") indicating that emotion information 808 immediately before reference (level of motivation "Motivation" 808a: "30") has changed to emotion information 808 at the time of reference (level of motivation "Motivation" 808a: "10"), and stores this emotion change information ("Motivation -20") in association with emotion change information 809 in the camera information time-series table 805 for the detection time 807 at the time of reference ("2025 / 2 / 1 14:10"). Here, the emotion change information corresponds to the subtraction value "-20" obtained by subtracting the level of emotion (motivation "Motivation" 808a) immediately before reference ("30") from the level of emotion (motivation "Motivation" 808a) at the time of reference ("10"). In this way, the user's emotion change information can be calculated from the camera information table 800.

[0085] In this way, the change calculation control unit 104 repeats the above-mentioned determination for all of the emotion information 808 in the camera information time series table 805 in chronological order, thereby being able to calculate the user's emotion change information at the point when sufficient data has been accumulated.

[0086] Here, as shown in FIG. 8B, the detection time 807 is "2025 / 2 / 1 14:10", emotion change information 809 is calculated as "Motivation -20", and the detection time 807 is "2025 / 2 / 1 15:15", emotion change information 809 is calculated as "Inspiration +10".

[0087] Once the change calculation control unit 104 has completed the calculation, the estimation control unit 105 of the server 11 then arranges the calculated spatial movement information at the predetermined communication time (detection time) and the calculated emotion change information at the predetermined shooting time (detection time) in chronological order, and estimates the relationship between the user's spatial movement and the change in the user's emotion (FIG. 2: S106).

[0088] Here, the estimation method of the estimation control unit 105 is not particularly limited, but for example, the estimation control unit 105 acquires detection time 802 and spatial movement information 804 from wireless device time series table 800, and acquires detection time 807 and emotion change information 809 from camera time series table 805. Then, using detection times 802 and 807 as keys, the estimation control unit 105 arranges detection times 802 and 807, spatial movement information 804, and emotion change information 809 in chronological order to create estimation table 900.

[0089] 9, estimation table 900 stores detection time 901, spatial movement information 902, and emotion change information 903 in association with each other. It can be seen that emotion change information 903 is stored within a predetermined threshold time (e.g., 30 minutes) from detection time 901 at which spatial movement information 902 is stored. This indicates that the emotion of a given user changed after the user moved through space.

[0090] Therefore, the estimation control unit 804 estimates the relationship between the user's movement in space and changes in the user's emotions by associating emotion change information 903 for a detection time 901 that is within a predetermined threshold time from the detection time 901 at which predetermined spatial movement information 902 is stored with the spatial movement information 902 from the estimation table 900.

[0091] 9, spatial movement information 902 for detection time 901 of "2025 / 2 / 1 14:00" stores "ab01->ab02," and emotion change information 809 for "2025 / 2 / 1 14:10," which is within the threshold time from detection time 901 of "2025 / 2 / 1 14:00," stores "Motivation -20." Therefore, estimation control unit 105 associates spatial movement information 902 "ab01->ab02" with emotion change information 809 "Motivation -20." This specifically means that when moving from the first space to the second space, the motivation "Motivation" decreases.

[0092] Furthermore, spatial movement information 902 for detection time 901 of "2025 / 2 / 1 15:00" stores "ab02 -> ab03," and emotion change information 809 for "2025 / 2 / 1 15:15," within the threshold time from detection time 901 of "2025 / 2 / 1 15:00," stores "Inspiration +10." Therefore, estimation control unit 105 associates spatial movement information 902 "ab02 -> ab03" with emotion change information 809 "Inspiration +10." This specifically means that moving from the second space to the third space increases the emotion "Inspiration." This makes it possible to estimate what kind of space led to the user's emotional change using wireless terminals and cameras installed in each space.

[0093] In the above description, it was estimated that moving from the first space to the second space would decrease motivation. Therefore, for example, as shown in FIG. 10A , objects 1000 (e.g., books) that improve motivation are placed in the second space (N=2), and work storage items 302 are removed from the second space. Then, using the camera 13, emotional information of the user who next enters the second space is acquired, and a camera information table 1001 is created, which makes it possible to examine whether the placement of the objects 1000 or the removal of the storage items 302 has changed the user's emotional information. Here, it is sufficient to examine whether the level of motivation has changed among the user's emotional information.

[0094] 10B, for example, a person 1002 (e.g., a receptionist) who improves the emotion "Inspiration" can be placed in the second space (N=2), and emotional information of the next user who enters the second space can be obtained using the camera 13. A camera information table 1003 can be created to examine whether the user's emotional information has changed due to the placement of the person 1002. Here, it is sufficient to examine whether the degree of the motivation "Inspiration" has changed among the user's emotional information.

[0095] In this way, if we can estimate the characteristics of a space relative to the user (changes in the user's emotions), we can make specific changes to the space that suit the user, such as arranging objects and people, etc. This makes it possible to implement specific measures to encourage changes in the user's emotions in a positive direction.

[0096] In this invention, we estimated what kind of space led to a change in the user's emotions based on the user's movement and emotions, without taking into account the spatial attributes of the space ID or the user attributes of the terminal ID. However, this is not limited to this, and the characteristics of the space for the user may be estimated by taking into account the spatial attributes of the space ID or the user attributes of the terminal ID.

[0097] 11A, a space attribute information table 1100 is prepared in advance, and a space ID 1101 and space attribute information 1102 are stored in association with each other in the space attribute information table 1100. The space attribute information 1102 means information indicating the attributes of a space, and stores, for example, specific information about the space in which the wireless device 12 or the camera 13 is installed, such as information about a building, an apartment building, a store, etc.

[0098] 11A, a user attribute information table 1103 is prepared in advance, and terminal IDs 1104 and user attribute information 1105 are stored in association with each other in the user attribute information table 1103. The user attribute information 1105 means information indicating the attributes of a user, and can include, for example, information on social status such as young, middle-aged, elderly, male or female, office worker, housewife, etc., and tendency information on hobbies, etc.

[0099] For example, in S104, when the movement calculation control unit 103 generates the wireless device information time-series table 800 for each terminal ID 402 from the wireless device information table 400, it uses the space attribute information table 1100 and the user attribute information table 1103. Specifically, when the movement calculation control unit 103 generates the wireless device information time-series table 800 in which space IDs are arranged in chronological order for a specific terminal ID (for example, "a01"), as shown in Fig. 11B, a detection time 802, a space ID 803, and space movement information 804 are stored in association with each other for the specific terminal ID 801 ("a01").

[0100] Here, the movement calculation control unit 103 refers to the space ID ("ab01") in the space attribute information table 1100 in correspondence with the space ID (for example, "ab01") in the wireless device information time series table 800, and newly associates the space attribute information 1102 associated with the referenced space ID with the space attribute information 1106 in the wireless device information time series table 800. Also, the movement calculation control unit 103 refers to the terminal ID ("a01") in the user attribute information table 1103 in correspondence with the terminal ID (for example, "a01") in the wireless device information time series table 800, and newly associates the user attribute information 1105 associated with the referenced terminal ID with the user attribute information 1107 in the wireless device information time series table 800. This allows the spatial attribute information 1106 and user attribute information 1107 to be taken into consideration for the spatial movement information 804 calculated by the movement calculation control unit 103, making it possible to specifically estimate how the emotions of a user with a predetermined user attribute will change when the user moves from a space with a certain spatial attribute to a space with another spatial attribute. Also, it is possible to estimate the influence of the spatial attribute information and user attribute information on spatial movement and changes in emotions.

[0101] Although the above description has been given of the wireless device information time series table 800 of the movement calculation control unit 103, the present invention is not limited to this. For example, the spatial attribute information table 1100 can also be used for the camera information time series table 805 of the change calculation control unit 104, which is associated with a spatial ID.

[0102] Furthermore, in the embodiment of the present invention, the spatial emotion estimation system 1 is configured to include each control unit, but it may also be configured to store a program that realizes each control unit on a storage medium and provide the storage medium. In this configuration, the program is read out by a device, and the device realizes each control unit. In that case, the program itself read out from the recording medium achieves the effects of the present invention. Furthermore, it is also possible to provide it as a method of storing the steps executed by each control unit on a hard disk. [Industrial Applicability]

[0103] As described above, the spatial emotion estimation system and spatial emotion estimation method according to the present invention are applicable not only to the marketing field but also to a wide variety of fields such as promotion, architecture, construction, and redevelopment, and are effective as a spatial emotion estimation system and spatial emotion estimation method that can estimate what kind of space led to a change in the user's emotion using wireless terminals and cameras installed in each space. [Explanation of symbols]

[0104] 1. Spatial Emotion Estimation System 10 User terminal 11 Server 12 Wireless equipment 13 Camera 14 Network 101 Wireless device information collection control unit 102 Camera collection control unit 103 Movement calculation control unit 104 Change calculation control unit 105 Estimation control unit

Claims

1. a wireless device collection control unit that uses a wireless device provided for each space to collect, as wireless device information for each wireless device, a space ID set in the wireless device, a terminal ID of a user terminal that has communicated with the wireless device by short-range wireless communication, and a communication time with the user terminal; a camera collection control unit that uses a camera provided for each of the spaces to collect, for each of the cameras, a space ID set in the camera, emotion information of the user photographed by the camera, and a photographing time of the camera as camera information; a movement calculation control unit that arranges the space ID of the collected wireless device information, the terminal ID, and the communication time in chronological order and calculates spatial movement information indicating that a user terminal with a specific terminal ID has moved from one space to another at a predetermined communication time; a change calculation control unit that arranges the space ID of the collected camera information, the user's emotion information, and the shooting time in chronological order, and calculates emotion change information that indicates a change in the user's emotion information when the user moves from one space to another at a predetermined shooting time; an estimation control unit that arranges the calculated spatial movement information at the predetermined communication time and the calculated emotion change information at the predetermined shooting time in chronological order to estimate a relationship between the spatial movement of the user and a change in the emotion of the user; A spatial emotion estimation system comprising:

2. the movement calculation control unit refers to the space ID in chronological order for a specific terminal ID based on the wireless device information, and determines whether the space ID at the time of reference has changed compared with the space ID immediately before the reference; If the result of the determination is that the space ID at the time of the reference has not changed compared to the space ID immediately before the reference, the determination is made by referring to the next space ID from the space ID at the time of the reference; If the result of the determination is that the space ID at the time of the reference has changed compared to the space ID immediately before the reference, spatial movement information indicating that the user has moved from the space of the space ID immediately before the reference to the space of the space ID at the time of the reference is calculated, and the spatial movement information is associated with the detection time at the time of the reference; the change calculation control unit references the emotion information in chronological order for each different space ID based on the camera information, and determines whether the emotion information at the time of reference has changed compared with the emotion information immediately before the reference; If, as a result of said determination, the emotion information at the time of reference has not changed compared to the emotion information immediately before said reference, said determination is made by referring to the next emotion information from the emotion information at the time of reference; if the result of said determination is that the emotional information at the time of reference has changed compared with the emotional information immediately before said reference, emotion change information indicating that the emotional information immediately before reference has changed to the emotional information at the time of reference is calculated, and this emotion change information is associated with the detection time at said reference. The spatial emotion estimation system according to claim 1 .

3. the estimation control unit estimates a relationship between the spatial movement of the user and a change in the emotion of the user by associating emotion change information at a shooting time that is within a predetermined threshold time from a communication time at which predetermined spatial movement information is stored with the spatial movement information; The spatial emotion estimation system according to claim 1 .

4. a wireless device collection control step of using a wireless device provided for each space to collect, as wireless device information for each wireless device, a space ID set in the wireless device, a terminal ID of a user terminal that has communicated with the wireless device by short-range wireless communication, and a communication time with the user terminal; a camera collection control step of collecting, for each camera, a space ID set in the camera, emotion information of the user photographed by the camera, and a photographing time of the camera as camera information; a movement calculation control process for arranging the space ID of the collected wireless device information, the terminal ID, and the communication time in chronological order, and calculating spatial movement information indicating that a user terminal having a specific terminal ID has moved from one space to another at a predetermined communication time; a change calculation control step of chronologically arranging the space ID of the collected camera information, the user's emotion information, and the shooting time, and calculating emotion change information indicating a change in the user's emotion information when the user moves from one space to another at a predetermined shooting time; an estimation control step of estimating a relationship between the user's spatial movement and the user's emotional change by arranging the calculated spatial movement information at the predetermined communication time and the calculated emotional change information at the predetermined shooting time in chronological order; A spatial emotion estimation method comprising:

Images