Behavior analysis device, and behavior analysis method

Abstract

To provide a behavior analysis device capable of quantifying the relationship between the behavior of a person and other behaviors of the same person, the time of day, the day of the week, the operating status of the device, and the behaviors of others, etc. as a degree of similarity, and a behavior analysis method.SOLUTION: A behavior analysis device 1 includes: an information acquisition unit 11 that acquires information including at least sensor data generated by human behaviors, and converts the acquired information into words; a calculation unit 12 quantifies the similarity, as the relationship between words, among multiple words associated with human behaviors, which are converted into words by the information acquisition unit; and a display unit 14 that displays the similarity among the words calculated by the calculation unit or the time series change of the similarity among the words.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a human behavior analysis device and a behavior analysis method. [Background technology]

[0002] In recent years, global warming has become an issue, and awareness and behavioral change are required in the activities of ordinary households, which account for approximately 60% of CO2 emissions. Recommendations that encourage behavioral change to reduce CO2 emissions in ordinary households are prone to status quo bias, such as "it is difficult to accept things that we do not know or have not experienced," if the content is uniform. This makes it difficult to lead to awareness and behavioral change. Therefore, there is a need for human behavior analysis technology to make more convincing recommendations.

[0003] For example, Patent Document 1 states that "based on the power consumption of residents (electricity users), what kind of "living behavior (resident behavior)" each user performed at what time or time period is estimated, and further, living behavior comparison information is generated in which a "comparison label" is added to the "living behavior" estimated for a specific user based on the comparison result with other similar users." The technology in Patent Document 1 makes it possible to provide recommendations that are compatible with each user's living behavior pattern through comparison with other users. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-191325 A Summary of the Invention [Problem to be solved by the invention]

[0005] According to the above-mentioned prior art, by using "behavioral attributes" and their "attribute values" as elements of a feature vector, similarity can be calculated by Euclidean distance or inner product. In addition, users whose feature vectors have a similarity equal to or greater than a predetermined threshold can be considered "similar users."

[0006] However, this "behavioral attribute" is, for example, "wake up" estimated based on power consumption data, and the "attribute value" is a specific value of the behavioral attribute, for example, "6:30." Therefore, the similarity calculated using a vector with these elements determines whether the word spellings match, and does not capture the magnitude of the relationship between words.

[0007] In addition, since comparisons could only be made between the "behavioral attributes" of the same person, it was difficult to grasp the relationship between the "behavioral attributes" and the time of day, day of the week, the operating status of home appliances, or the "behavioral attributes" of others.

[0008] An object of the present invention is to provide a behavior analysis device and a behavior analysis method that quantify the relationship between a human behavior and other behaviors of the same person, time, day of the week, operating status of a device, behavior of others, etc. [Means for solving the problem]

[0009] In order to solve the above problems, the behavioral analysis device of the present invention includes an information acquisition unit that acquires information including at least sensor data generated in conjunction with human behavior and converts the acquired information into words, a calculation unit that quantifies a similarity as a relationship between multiple words associated with human behavior that have been converted into words by the information acquisition unit, and a display unit that displays the similarity between words calculated by the calculation unit or the time series change in the similarity between words. Effect of the Invention

[0010] According to the present invention, it is possible to quantify the similarity as the relationship between a human behavior and other behaviors of the same person, the time of day, the day of the week, the operating state of a device, the behavior of others, and the like. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of a behavior analysis device according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating a configuration of a calculation unit. [Diagram 3] 13 is a flowchart illustrating an operation of the behavior analysis device. [Figure 4] FIG. 1 is a configuration diagram of a smart home service system to which a behavior analysis device is applied. [Figure 5A] This is an example of visualizing the similarity between cooking behavior and each day of the week using a bar graph. [Figure 5B] This is an example of visualizing the similarity between laundry behavior and other behaviors using a polygonal chart (radar chart). [Figure 6] This is an example of a screen of an app that provides smart home services as a display unit. [Figure 7] 13 is another example screen of an app that provides smart home services as a display unit. [Figure 8A] 13 is an example of an app screen showing the relationship between the operation status of home appliances and the actions of a user. [Figure 8B] 13 is an example of an app screen showing the relationship between the operation status of home appliances and the actions of a user. [Figure 9] FIG. 1 is a system configuration diagram of a smart life service to which a behavior analysis device is applied. [Figure 10A] 13 is an example of a screen display of an app that provides smart life services. [Figure 10B] 13 is an example of a screen display of an app that provides smart life services. [Figure 11] FIG. 1 is a system configuration diagram of a smart life service to which a behavior analysis device is applied. [Figure 12A] FIG. 13 is a diagram illustrating an output screen of a smart life service. [Figure 12B] FIG. 13 is a diagram illustrating an output screen of a smart life service. [Figure 13] 13 is an example of a screen for selecting a recommendation policy. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiment, and various modifications and application examples within the technical concept of the present invention are also included in its scope. EXAMPLES

[0013] 1 is a block diagram showing the configuration of a behavior analysis apparatus 1 according to this embodiment. The behavior analysis apparatus 1 is made up of an information acquisition unit 11, a calculation unit 12, a storage unit 13, and a display unit 14, and quantifies human behavior.

[0014] The information acquisition unit 11 acquires data 21 generated in association with human behavior and information 31 to be analyzed, and transmits them to the calculation unit 12 and the storage unit 13. Here, the data 21 generated in association with human behavior includes sensor data 21a (not shown) such as human presence, illuminance, temperature and humidity, air pressure, acceleration, and door opening and closing, image data 21b (not shown), voice data 21c (not shown), facility entry and exit records 21d (not shown), reservation information 21e (not shown) for vehicles and events, payment information 21f (not shown) including product names and prices, and date, day of the week, and time information thereof. In addition, the information 31 to be analyzed includes a symbol, facility name, room name, place name, place name, time information, etc., which identify the person to be analyzed.

[0015] Specifically, the sensor data 21a is acquired by installing various sensors in the house, using a wearable sensor such as a smart watch, using a sensor that is already installed in a home appliance, or the like.

[0016] Since the human presence sensor outputs a numerical value of 1 / 0 indicating the presence / absence of a human, the information acquiring unit 11 converts the output result of the human presence sensor again into a word indicating presence / absence, or converts it into an action name linked to the place where the human presence sensor reacted, and transmits it to the calculation unit 12 and the storage unit 13. For example, the information acquiring unit 11 converts the reaction of the human presence sensor in the living room to relaxation, the reaction of the human presence sensor in the bedroom to sleep, the reaction of the human presence sensor in the kitchen to cooking, and a state where there is no reaction from the human presence sensor to going out. These words (presence, absence, relaxation, sleep, cooking, going out) are words that convert information including sensor data generated in accordance with human actions.

[0017] The information acquisition unit 11 converts the analog values ​​of the sensor outputs into words at a certain interval, since the outputs of the illuminance sensor, temperature and humidity sensor, air pressure sensor, and acceleration sensor are continuous analog values. For example, 0°C to 10°C is converted into words as low temperature, 10°C to 30°C is converted into words as appropriate temperature, and 30°C to 40°C is converted into words as high temperature. Alternatively, 20% or less may be converted into words as dry, 20% to 60% is converted into words as comfortable, and 60% to 100% is converted into words as wet. Similarly, air pressure is converted into words as low, normal, and high pressure at arbitrary thresholds. The output of the acceleration sensor is also converted into words as active, normal, and resting at arbitrary thresholds using its magnitude. Alternatively, the output of acceleration in three axial directions and the amount of change thereof are converted into words as walking, running, falling, and the like. For example, the illuminance changes depending on whether the light is turned on or off, and the values ​​of the illuminance, temperature and humidity, air pressure, and acceleration change according to human behavior. The changes are acquired by the information acquisition unit 11 as information including sensor data and converted into words.

[0018] As described above, the information acquiring unit 11 converts the output value of the sensor data 21a into words under any condition, which has the advantage of making it easier for the user to understand the physical meaning indicated by the data.

[0019] The sensor data 21a may be acquired using a wearable sensor such as a smart watch. Unlike the above-mentioned sensors, wearable sensors are always worn by the user, so data can be collected even in situations where it is difficult to install a sensor, such as outdoors. In addition, because the sensor is in close contact with the body, data such as heart rate and sleep quality that are different from those obtained by the above-mentioned non-contact sensors can be acquired.

[0020] The sensors that are provided in advance in home appliances include temperature sensors, door opening / closing sensors, current sensors, etc. The information acquisition unit 11 can use these sensors to infer the status of people and home appliances, and convert them into words. For example, cooking and eating can be inferred using the sensors of a refrigerator and a microwave, laundry can be inferred from the sensor of a washing machine, and cleaning can be inferred from the sensor of a vacuum cleaner.

[0021] The image data 21b can be acquired by a surveillance camera in a store, public facility, etc. The image data 21b includes information such as the number of people captured, their actions, and the direction of people flow, in addition to the date and time of capture and the location, so the information acquisition unit 11 converts this information into words such as "10 people, walking, south direction" using image analysis technology.

[0022] The voice data 21c can be acquired by a telephone call record, a conversational smart device, a camera equipped with a microphone, etc. The information acquisition unit 11 converts the voice data 21c into sentences using a natural language processing technique, and further divides the sentences into words.

[0023] The facility entrance and exit records 21d include handwritten entrance and exit records, room entrance and exit cards, and authentication records based on fingerprints, veins, etc., and are managed by the facility's security management device. The information acquisition unit 11 acquires the entrance and exit records 21d from the security management device. These data include words such as the location, date, day of the week, and time when the entrance and exit was recorded, and information that identifies an individual. For example, there is "February 10th, Monday, Conference Room A, Person B."

[0024] The reservation information 21e for an event or the like includes information such as reservation date and time, number of people, event name, date and time, venue, organizer, genre, etc. as words, and the information acquisition unit 11 can acquire the reservation information 21e from the management device of the event reservation site or the reservation history of the reservation person's terminal. For example, the reservation information 21e may be "February 11, Tuesday, music event, 3 people, Venue A, Sponsored by B."

[0025] The payment information 21f including the product name, price, etc. includes information such as the payment date and time, place, product name, price, etc. as words, and the information acquisition unit 11 can acquire it from the history of the payment processing device or the terminal of the person who made the payment. An example of the data of the payment information 21f is "February 12, Wednesday, Department Store A, Product B, Price C".

[0026] The calculation unit 12, which will be described in detail later, quantifies the similarity as the relationship between words for the information obtained by converting data 21 generated by human behavior notified by the information acquisition unit 11 into words. Then, the calculation unit 12 transmits the processing result to the display unit 14 and the storage unit 13. The calculation unit 12 may also determine the time series change of the vector between words, for example, by dividing the period into days or seasons.

[0027] The display unit 14, the details of which will be described later, displays the current relationship between words calculated by the calculation unit 12 or a time-series change in the relationship between words. The storage unit 13 stores the relationships between the words quantified by the calculation unit 12 for each subject to be analyzed.

[0028] Next, the configuration of the calculation unit 12 will be described with reference to the block diagram of FIG. The calculation unit 12 comprises a text creation unit 121, a word division unit 122, a word digitization unit 123, and a similarity calculation unit .

[0029] The sentence creation unit 121 creates sentences by listing, in a specific order, words and the like capable of identifying dates, days of the week, times, locations, human behavior, environmental conditions, individuals, and the like received from the information acquisition unit 11. Here, a sentence in this specification is defined as a combination of words connected using periods, commas, spaces, particles, auxiliary verbs, and the like, such as "February 13th, Thursday, 14:00, Room A, Relax, Sunny, Mr. B."

[0030] The sentence creation unit 121 adds date, day, and time information to each created document and describes them in multiple sentences in chronological order, thereby clarifying the relationship between the date, day, and time information and other words. For example, when image data is received from the information acquisition unit 11 at 15:00 on Friday, February 14th, and reservation information for an event or the like is received at 16:00 on Friday, February 14th, the sentence creation unit 121 creates sentences with the date, day, and time information for each data item, such as "February 14th, Friday, 15:00, 10 people, walking, south direction, music event, venue A, sponsored by B", making it easier to quantify the relationship between the date, day, and time information and other words.

[0031] Furthermore, the sentence creation unit 121 creates sentences by standardizing at least a part of the order in which words are written. This makes it possible to sort multiple sentences based on a certain word. For example, by defining the order in which sentences are written as "words that can identify dates, days of the week, times, locations, human behavior, environmental conditions, individuals, etc.", it is possible to create multiple sentences from multiple data, obtain date and time information from the first and third words, and sort the sentences in chronological order based on the obtained information. This makes it possible to quantify the relationships between words in chronological order, making the chronological relationships between each piece of data clearer.

[0032] The word division unit 122 receives the sentences from the sentence creation unit 121 and divides each sentence into words again. The division is performed mechanically using periods, commas, spaces, particles, auxiliary verbs, etc. Alternatively, a word division algorithm used in the field of natural language processing such as Janome or MeCab may be applied. Note that if it can be confirmed that all the information transmitted from the information acquisition unit 11 has already been divided into words in advance and that the order of description is standardized, the processing of the sentence creation unit 121 and the word division unit 122 may be omitted.

[0033] The word quantification unit 123 receives words for each sentence divided by the word division unit 122, analyzes words present around a certain word when it occurs in data consisting of at least one sentence, and quantifies the words in vector format so that the more frequently words occur together, the greater the similarity. For example, a word vectorization algorithm used in the field of natural language processing, such as Word2Vec or BERT, is applied, and a vector representation of each word is acquired by learning.

[0034] The similarity calculation unit 124 receives the words expressed in vector format by the word quantification unit 123, and quantifies the relationship between the words using the similarity of the vectors. For example, the relationship between the words is quantified using the cosine similarity defined by the following formula (1).

[0035]

number

[0036] Here, in formula (1), a and b are vectors, a·b is the dot product, and represents the product of norms. Using this cosine similarity, it is possible to quantify the angle between words expressed in vector format. The angle between vectors represents the similarity between the words represented by those two vectors, and it can be inferred that the smaller the absolute value, the more similar the tendency is, and that if the absolute value is less than 90 degrees, the tendency is the same, and if it is 90 degrees or more, the tendency is opposite. Cosine similarity is useful in that it can express these tendencies numerically, including positive and negative signs.

[0037] As another method for evaluating the relationship between words, the occurrence probability defined by the following formula (2) may be used.

number

[0038] Here, in formula (2), L is the distance between vectors, and n indicates that there are a total of n vectors whose distance to a certain vector is to be calculated. The subscript i indicates that the vector is the i-th vector among the n vectors. The value of n is a value that is inevitably determined when the analysis content is determined, as in the following specific example.

[0039] (Specific example 1) When evaluating the relationship between cooking behavior and each day of the week, n ​​is the number of different days of the week, 7. (Specific example 2) In a building with five conference rooms, if you want to evaluate the relationship between a person who works there and the conference rooms, n is 5, which is the number of types of conference rooms. (Example 3) In a family of four, when evaluating the relationship between the father and the other family members, n is the number of family members other than the father.

[0040] By using this occurrence probability, the distance between words expressed in vector format can be quantified and the relationship between the words can be evaluated. In addition to expressing the similarity between the words indicated by the two vectors, just like the angle formed by the vector, the distance between the vectors also expresses the similarity between the words indicated by the two vectors, and since formula (2) takes into account the total number n to be calculated, it can appropriately express the proportion of all possible cases. This is expected to more appropriately express the influence of the number of occurrences, frequency of occurrence, etc., compared to cosine similarity.

[0041] Another method for evaluating the relationship between words is to use the results of the calculation of formula (1) and formula (2). For example, by evaluating the relationship between words using formula (3), which is the product of the two, it is possible to reflect the information on the positive and negative signs of the cosine similarity and express the similarity in the tendency between words, while at the same time taking into account the influence of the number of occurrences and frequency of occurrence expressed by the occurrence probability.

[0042]

number

[0043] The relationships between words calculated by the calculation unit 12 are linked to the information 31 to be analyzed and transmitted to the storage unit 13, and are also transmitted to the display unit 14, where they are visualized by means of a bar graph, a polygonal chart, or the like. The calculation unit 12 may use not only the data 21 generated by human behavior, but also the data received from the storage unit 13 to quantify the relationships between words.

[0044] The storage unit 13 transmits the relationship between words calculated by the calculation unit 12 to the calculation unit 12 together with past accumulated data on the subject of analysis or other people, other facilities, etc. This enables the calculation unit 12 to utilize the current relationship between words and the past accumulated data to calculate time-series changes in the relationship between words and transmit it to the display unit 14.

[0045] Next, the operation of behavior analysis apparatus 1 will be described with reference to the flowchart in FIG. The flowchart in FIG. 3 is started each time a behavioral analysis is performed or periodically.

[0046] In step S11, the information acquisition unit 11 acquires data 21 generated in association with human behavior and information 31 to be analyzed.

[0047] In step S12, the information acquiring unit 11 converts the acquired information into words and transmits them to the calculation unit 12 and the storage unit 13.

[0048] In step S13, the calculation unit 12 uses the information received from at least one of the information acquisition unit 11 and the storage unit 13 to convert the relationship between the words into a numerical value.

[0049] In step S14, the calculation unit 12 transmits the calculated relationship between the words to the display unit 14 and the storage unit 13.

[0050] In step S15, the storage unit 13 transmits the relationship between the words calculated by the calculation unit 12 to the calculation unit 12 together with previously stored data on the subject of analysis or other people, other facilities, and the like.

[0051] In step S16, the calculation unit 12 uses the current relationships between the words and the past accumulated data to calculate time-series changes in the relationships between the words and transmits them to the display unit .

[0052] In step S17, the display unit 14 displays the current relationships between words and the changes over time. EXAMPLES

[0053] In the following, a case will be described in which the behavior analysis device 1 of the embodiment is applied to a general home in which connected home appliances and various sensors are installed, to provide a smart home service. FIG. 4 is a system configuration diagram of a smart home service to which the behavior analysis device 1 is applied.

[0054] 4, a service provider provides smart home services and products in cooperation with the behavior analysis device 1. Sensors are built into the products in advance, and the behavior analysis device 1 collects information about the products via the Internet or an external server 61. The products include connected home appliances and various sensors.

[0055] When a subscription application for a service or product is made, the business operator sends the necessary connected home appliances and various sensors to the user's home. If the user's home already has connected home appliances or monitoring sensors, the external server 61 may collect the data on those appliances to obtain information required for the smart home service.

[0056] The sensors include, for example, a body composition scale 41, a weight scale 42, a wearable sensor 43, a pressure sensor 44, a vibration sensor 45, an opening / closing sensor 46, an environmental sensor 47 including a temperature, humidity and air pressure sensor, a human presence sensor 48, an image sensor 49, a radio wave sensor 50, an illuminance sensor 51, etc., and the data thereof is collected by the behavioral analysis device 1 via the Internet or an external server 61.

[0057] The connected home appliances include a refrigerator 52, a washing machine 53, a microwave oven 54, a robot cleaner 55, an induction cooking heater 56, and a water heater 57, and the behavior analysis device 1 can collect information on operation modes and the like in addition to the indication values ​​of pre-installed door opening / closing sensors, current sensors, etc. After the sensors are installed, they secure power by connecting to a power source, using batteries, or by environmental power generation such as sunlight or vibration, and start measurement.

[0058] The owner of the external server 61 is not limited, and it may be a server managed by a business operator, a server managed by a sensor manufacturer, or a server that can be generally rented.

[0059] Also, the smart home service provider and the provider of the behavior analysis device 1 may store data in different environments, read the data into the behavior analysis device 1, and connect within the system. However, these servers must be able to communicate with a behavior analysis device 1 built in either the in-house environment of the provider of the behavior analysis device 1, the in-house environment of the provider, or a cloud environment.

[0060] The operation of the behavior analysis device 1 in the smart home service system will be described in detail below.

[0061] The information acquisition unit 11 of the behavior analysis device 1 acquires the sensor data, connected home appliance data, and analysis target information 31 collected by the external server 61, converts the data into words, and transmits the words to the calculation unit 12 and the storage unit 13. Methods for acquiring the information 31 to be analyzed include inputting the information into a sensor, entering the information on an application form for a service or product, allowing additional input via the user's personal page on the business's website, obtaining the information through collaboration with third parties such as other business operators, the local government to which the user belongs, or non-profit organizations.

[0062] The calculation unit 12 quantifies the relationship between words using the data received from the information acquisition unit 11 and the analysis target information 31. In detail, first, a sentence is created in the sentence creation unit 121 of the calculation unit 12 using the data of the sensor and the connected home appliance that have been converted into words by the information acquisition unit 11.

[0063] For example, as an example of converting sensor data into words, the values ​​of the environmental sensor 47 including the temperature, humidity and air pressure sensor can be converted into words in certain ranges, such as 0°C to 10°C being "low temperature," 10°C to 30°C being "optimum temperature," and 30°C to 40°C being "high temperature." In addition, the location where the human presence sensor 48 reacts is associated with human behavior, and if the human presence sensor reacts in the kitchen, it is converted into a word such as "kitchen" or "cooking." As an example of converting data from connected home appliances into words, when a microwave oven is operating in some operating mode such as heating or cooking, it can be converted into words such as "microwave oven" or "cooking."

[0064] The sentence creation unit 121 uses these wordization results and the date, time, and day of the week information on when the sensor or connected home appliance operated to create a sentence such as "February 15, Saturday, 12:00, low temperature, kitchen, microwave, cooking." This sentence can be created for each time when the sensor or connected home appliance reacts, and they may be linked to the information 31 to be analyzed and stored in the storage unit 13.

[0065] The word division section 122 of the calculation section 12 accumulates the sentences created by the sentence creation section 121 for a certain period of time, and then arranges them in chronological order and divides them into words again. Then, the word digitizing unit 123 of the calculation unit 12 expresses the words in vector format.

[0066] Finally, the similarity calculation unit 124 of the calculation unit 12 calculates the similarity between the words. These operations make it possible to quantify the relationships between words contained in sentences obtained from data from sensors or connected home appliances, so that, for example, other actions, days of the week, times, temperatures, locations, home appliances, etc. that are closely related to cooking behavior can be evaluated numerically.

[0067] Next, a specific example of the inter-word relationship being quantified by the word quantification unit 123 of the calculation unit 12 will be described.

[0068] FIG. 5A is an example of the similarity between cooking behavior and each day of the week calculated using the above formula (1) or formula (2) and visualized using a bar graph. When the similarity is calculated using formula (1) or formula (2), if the trends are similar, the similarity will be positive, and if the trends are not similar, the similarity will be negative. For this reason, as shown in FIG. 5A, the similarity can be clearly indicated by changing the way the bar graph is colored depending on the similarity, and it can be intuitively confirmed that cooking behavior is not similar to the trends on Wednesdays and Thursdays. Alternatively, instead of using a bar graph, the background color of the graph can be changed depending on the similarity.

[0069] FIG. 5B is an example of visualizing the similarity between laundry behavior and other behaviors using a polygonal chart (radar chart). In FIG. 5B, the similarity is calculated using formula (1) or formula (3), and the line format is changed to clearly show positive similarity indicating similar trends and negative similarity indicating dissimilar trends. This allows intuitive confirmation that laundry behavior is dissimilar to breakfast and lunch trends. Instead of changing the line format, the background color of the area may be changed.

[0070] Next, a display example of display unit 14 of behavior analysis apparatus 1 will be described. In the smart home service system of the embodiment, the display unit 14 may be a display connected to the behavior analysis device 1, or may be a terminal such as a smartphone that allows a user using the smart home service system to check the display at hand.

[0071] Fig. 6 shows an example of a screen of an app that provides a smart home service on an external terminal 71 such as a user's smartphone as the display unit 14. In Fig. 6, the similarity between each type of behavior and other behaviors is visualized in a polygonal chart (radar chart). This allows the user to check the relationships between their own behaviors and reflect on their lifestyle habits.

[0072] In Fig. 6, the smart home service further provides energy saving advice based on the relationship between actions. In detail, the service links the time when laundry is performed with information on the electricity rate plan subscribed to by the user, which is an example of the information 31 to be analyzed, and provides advice (recommendations) on how to save on electricity bills.

[0073] FIG. 7 is another example of a screen of an app that provides a smart home service on an external terminal 71 such as a user's smartphone as the display unit 14. FIG. 7 displays the relationship between information such as sleep quality and heart rate, which can be acquired by a wearable sensor such as a smartwatch, and behavior. The sleep quality is measured by a smartwatch, for example, into REM sleep, non-REM sleep (light), and non-REM sleep (deep), and is classified as sleep quality: low, sleep quality: medium, and sleep quality: high, respectively. The wearable sensor such as a smartwatch may be sent by a business operator, or a device owned by the user may be used.

[0074] In detail, the behavioral analysis device 1 combines the sleep quality with information on the user's behavior, location, date, day of the week, time, etc. detected by the sensor to create a sentence such as "February 15th, Saturday, 22:00, bedroom, sleep, sleep quality: low, heart rate: 100", and inputs this into the calculation unit 12, thereby quantifying the relationship between information such as sleep quality and heart rate and behavior.

[0075] Then, the behavioral analysis device 1 shows the user, using a polygonal chart (radar chart) shown in Figure 7, that there is a high correlation between high sleep quality and relaxation behavior, going out behavior, and cooking behavior, and at the same time, shows that there is a high correlation between low sleep quality and napping behavior.

[0076] In addition, based on this information, the behavioral analysis device 1 may provide advice (recommendations) to improve the quality of sleep, such as, "It seems that the quality of your sleep does not improve on the days when you take a nap. After you move your body, relax sufficiently and prepare for sleep. It may also be effective to enjoy cooking."

[0077] 8A and 8B are examples of screens of an app that shows the relationship between the operation status of home appliances and the like and the user's actions on an external terminal 71 such as a user's smartphone as the display unit 14. The operation status of the home appliances shows the refrigerator temperature settings of low, medium, and high detected by a sensor that is provided in advance in the connected refrigerator.

[0078] The operating status of the home appliance may also be the status detected by a sensor pre-installed in the connected washing machine, which detects the washing modes of the washing machine, such as the muddy soil course, the delicates course, and the tub cleaning course. Furthermore, in order to quantify the relationship between seasonal operation status and user behavior, a year may be classified into four seasons (spring, summer, autumn, and winter) based on the date on which the operation status of the home appliances, etc. was obtained, and the information may be added. In addition, the environmental temperature may be measured using a sensor attached to the connected home appliance or a separately installed environmental sensor, and the measured value may be classified into three categories: low temperature, suitable temperature, and high temperature, and added to the information.

[0079] By combining these pieces of information to create a sentence such as "February 16th, winter, Sunday, 13:00, refrigerator setting: medium, environment: low temperature" and inputting it into the calculation unit 12, the relationship between the refrigerator setting and each piece of environmental information, as shown in Figs. 8A and 8B, is quantified. By comparing Fig. 8A, in which the refrigerator settings remain constant throughout the year, with Fig. 8B, in which the system displays the most energy-efficient refrigerator settings, the user can recognize that they have not been making full use of the refrigerator settings up until now, and at the same time, understand the ideal refrigerator settings for energy conservation.

[0080] At this time, the user may refer to the home appliance utilization advice (recommendation) in Figures 8A and 8B. Alternatively, the user can check the display in Figure 8A again after starting to use the refrigerator settings, and thereby check the difference with the display in Figure 8B, which shows the ideal settings, and confirm his / her own energy saving level.

[0081] In the above, the temperature setting of a refrigerator has been described as an example of the operating status of a home appliance, but other than temperature, a current value, a compressor rotation speed, etc. may be used. Alternatively, fault information may be separately obtained from repair requests, returns, etc., and the relationship with the operating status of the home appliance may be quantified. EXAMPLES

[0082] In the following, a case where a smart life service is implemented by applying the behavior analysis device 1 of the embodiment will be described. Fig. 9 is a system configuration diagram of a smart life service to which the behavior analysis device 1 is applied. In the following explanation, the explanation of the contents that overlap with the system configuration diagram in Fig. 4 will be omitted, and only the different parts will be explained.

[0083] The information acquisition unit 11 of the behavior analysis device 1 of a business providing a smart life service acquires data 21 generated in association with human behavior in a home 73 of a user of the smart life service. Here, a sensor for acquiring the data 21 generated in association with human behavior is installed in the user's home by the business, or a device owned by the user is utilized.

[0084] The behavior analysis device 1 of this embodiment also acquires information on external businesses related to the user in his / her daily life. Here, the external businesses are, for example, companies that manage transportation infrastructure, building / facility management companies, event companies, etc., and the information that the user has authorized is acquired by the information acquisition unit 11 from external servers 61a, 61b, 61c managed by these external businesses.

[0085] The behavior analysis device 1 (information acquisition unit 11) acquires information such as the time, section, and fare when a user uses a transportation infrastructure such as a bus or a train from an external server 61a of a company that manages the transportation infrastructure. The behavior analysis device 1 (computation unit 12) uses this information and information obtained by analyzing the sensor data 21a in the home to document information on behavior in the home and movement outside the home in chronological order, such as "February 17, Monday, 14:00, meal," "February 17, Monday, 14:30, out of home," or "February 17, Monday, 15:00, out of home, bus ride (to XX)." The behavior analysis device 1 (computation unit 12) then quantifies the relationship between eating behavior in the home, outing, and bus ride including destination on Monday, together with time information, and clarifies the relationship between lifestyle habits in the home and movement information outside the home.

[0086] The behavior analysis device 1 (information acquisition unit 11) acquires information such as the office building, sports gym, library, etc., the building entry / exit record, and the conference room entry / exit record used by the user from the external server 61b of the building / facility management company. The behavior analysis device 1 (computation unit 12) uses this information, the sensor data 21a in the home, or the information obtained by analyzing the user's wearable sensor 43 to document information on behavior in the home and behavior in facilities outside the home in chronological order, such as "February 18, Tuesday, 8:00, wake up, heart rate: low", "February 18, Tuesday, 9:00, go out, heart rate: low", and "February 18, Tuesday, 9:30, go out, enter sports gym, heart rate: medium". The behavioral analysis device 1 (calculation unit 12) then quantifies the relationship between the wake-up time at home on Tuesday, going outside, entering the gym, and heart rate, and evaluates the lifestyle habits in relation to the lifestyle habits at home, the usage information of buildings and facilities outside the home, and health conditions such as fatigue and fever that can be inferred from the heart rate.

[0087] The behavior analysis device 1 (information acquisition unit 11) acquires the types of events, reservation records, and participation information, such as tour trips and concerts, that the user participated in, from the external server 61c of the event company. The behavior analysis device 1 (computation unit 12) uses these pieces of information and information obtained by analyzing the sensor data 21a in the home to document information on behaviors in the home and behaviors related to events outside the home in chronological order, such as "February 19, Wednesday, 21:00, sleep," "February 20, Thursday, 7:00, wake up," "February 20, Thursday, 7:30, go out," and "February 20, Thursday, 9:00, go out, participate in concert event." The behavior analysis device 1 (computation unit 12) then quantifies the relationship between the sleep time in the home on Wednesday night and the wake-up, go out, and participate in a concert event on Thursday morning, and evaluates the lifestyle habits in the home and the lifestyle habits related to participation in a concert event outside the home.

[0088] The behavior analysis device 1 may use a combination of information acquired from the external servers 61a, 61b, and 61c. For example, the behavior analysis device 1 acquires information that a user has entered an office building from the external server 61b of a building / facility management company, and at the same time acquires the means of transportation used by the user from the external server 61a of a company that manages the transportation infrastructure, and documents the behavior, such as "February 21, Friday, 10:30, going out, bus, office building." The behavior analysis device 1 may also digitize the relationship between the going out behavior, destination, and means of transportation, together with time information.

[0089] Here, behavior analysis apparatus 1 may store the information acquired from external servers 61a, 61b, 61c and the quantified analysis results of the relationships of the analysis targets in a server different from storage unit 13, in association with information 31 of the analysis targets.

[0090] The data 21 generated by human behavior acquired by the information acquisition unit 11 may be converted into a human behavior name by the behavior recognition unit 15 and notified to the information acquisition unit 11. The behavior recognition unit 15 is a classifier configured by a learning model such as machine learning (logistic regression, decision tree, support vector machine, random forest, k-nearest neighbor method, etc.) and deep learning (combination of fully connected layer, CNN, RNN, etc.), and associates the data 21 generated by human behavior with the human behavior name based on past measurement data of the user of this embodiment or other users. The behavior recognition unit 15 may be included in the behavior analysis device 1, or may be stored in another server.

[0091] As a result, when inferring human behavior from data 21 generated in accordance with human behavior, the behavioral analysis device 1 is able to make inferences that are not at first glance obvious and therefore difficult for humans to make, based on multiple pieces of information, such as "Based on a comprehensive assessment of the human sensor in the living room, the human sensor in the kitchen, and the kitchen temperature and humidity sensor, it is highly likely that the person is cooking," in addition to obvious inferences such as "Based on a comprehensive assessment of the human sensor in the living room, the human sensor in the kitchen, and the kitchen temperature and humidity sensor, it is highly likely that the person is cooking."

[0092] Next, a description will be given of an output screen of the smart life service displayed on the display unit 14 of the external terminal 71 of the user that is owned by the user or loaned from the business operator. 10A and 10B are display examples of the screen of an app that provides a smart life service.

[0093] Fig. 10A is an example of quantifying the relationship between a live event and a mode of transportation and a time period. In Fig. 10A, the relationship between a live event and a bus, train, or taxi is quantified using boarding and alighting information obtained from an external server 61a of a company that manages the transportation infrastructure. Morning, afternoon, and evening are defined as certain time ranges based on the time when the data was obtained.

[0094] From FIG. 10A, the user or business can understand that the main mode of transportation for users to travel to concerts is by train, and that the most common time period is during the day, with a small possibility of traveling in the morning. In addition, based on the quantification of the relationship and publicly available train congestion forecast information, the system will recommend smart life advice on how to avoid crowds, such as, "Today is the day of a live performance, but daytime trains are expected to be crowded. How about leaving early and enjoying lunch?"

[0095] Figure 10B is an example of quantifying the relationship between the time of year and live performance. The time of year can be any division such as spring, summer, autumn, winter, rainy season, early autumn, late autumn, early winter, etc. The time of year is defined as a certain month range based on the date when the data was acquired.

[0096] From FIG. 10B, the user or business can understand that the most likely times for users to attend live shows are spring, early fall, and late fall, and that other times are less likely. In addition, based on the quantification results of the relationship and event schedule information obtained from the event company's external server 61c, smart life advice such as "The live concert that you usually attend in the fall will be held in the summer this year. Bring cooling products with you and be careful not to get heatstroke" may be given. EXAMPLES

[0097] The following describes a case where the behavior analysis device 1 of the embodiment is applied to implement other smart life services. Fig. 11 is a system configuration diagram of a smart life service to which the behavior analysis device 1 is applied. In the following explanation, explanations of contents that overlap with the system configuration diagrams in Fig. 4 and Fig. 9 will be omitted, and only different parts will be explained.

[0098] The information acquisition unit 11 of the behavior analysis device 1 of a business providing a smart life service acquires data 21 generated in association with human behavior in the home 73 of a user of the smart life service and data from an external server 61. The data from the external server 61 may be acquired with the approval of the organization collecting the data or the individual from whom the data is collected, and may also include information on people other than the user of the service.

[0099] Based on data from multiple external servers 61, the behavior analysis device 1 chronologically documents behavior including multiple pieces of information, such as "February 24th, Monday, 9:30, out, bus, 200 yen, office building lobby", "February 24th, Monday, 14:00, out, office building meeting room A", "February 24th, Monday, 20:00, out, office building meeting room B", "February 24th, Monday, 22:30, out, taxi, 3000 yen", or "February 24th, Monday, 23:00, relaxing". Then, the behavior analysis device 1 (the calculation unit 12) quantifies the relationship between the user's various behaviors and the office building.

[0100] Next, the output screen of the smart life service of this embodiment displayed on the display unit 14 of the external terminal 71 will be described with reference to Figs. 12A and 12B.

[0101] 12A and 12B are examples of quantification of the relationship between the user's behavior and the office building and the associated means of transportation. The user can specify the policy for the recommendation contents such as smart life advice displayed on the display unit 14 of the external terminal 71. Specifically, the user selects the recommendation policy from "saving mode", "healthy mode", "study mode", "family mode", "volunteer mode", etc. on the interface 72 (see FIG. 11) of the external terminal 71, and the recommendation content determination unit 16 (see FIG. 11) receives the selected recommendation policy, determines the recommendation contents based on the output of the calculation unit 12, and displays them on the display unit 14.

[0102] 13 is an example of a screen for selecting a recommendation type on the interface 72 of the external terminal 71. Here, the "Savings Mode" is a mode that aims to reduce or minimize the costs incurred in actions related to the office building and makes recommendations to achieve this. Similarly, the "Healthy Mode" prioritizes the user's diet, nutritional balance, and health, the "Study Mode" prioritizes work efficiency improvement and ability development, the "Family Mode" prioritizes the home environment, and the "Volunteer Mode" prioritizes contributions to others and society outside of work.

[0103] In Figure 13, when the user selects the "Savings Mode," the behavior analysis device 1 displays a "Savings Mode" recommendation, such as "I travel by taxi late at night. By speeding up my travel time and using trains and buses, I could save about 5,000 yen each month," based on the information that quantifies the relationship between the user's behavior and the method of travel to the office building, shown in a polygonal chart (radar chart), as shown in Figure 12A.

[0104] In FIG. 13, when the user selects the "Study Mode," the behavior analysis device 1 displays a "Study Mode" recommendation of "Most of the usage time is late at night. Since the average top 10 performers tend to be morning types, it may be effective to improve work efficiency by coming to work and coming home earlier." based on the information that quantifies the relationship between user behavior and office building usage time shown in a polygonal chart (radar chart) as shown in FIG. 12B.

[0105] In Figure 13, when the user selects "Healthy Mode," the behavioral analysis device 1 displays a "Healthy Mode" recommendation, such as "You often stay in the office building even during your lunch break. Why not have a lunch with lots of vegetables outside and take a walk? Walking or cycling to the nearest station is also effective," based on the information that quantifies the relationship between the user's behavior and the method of transportation to the office building, as shown in the polygonal chart (radar chart) of Figure 12A, and the information that quantifies the relationship between the user's behavior and the time spent in the office building, as shown in the polygonal chart (radar chart) of Figure 12B.

[0106] In Figure 13, when the user selects "Family mode," the behavior analysis device 1 displays a "Family mode" recommendation such as "If you come home on time on Thursdays, you can spend more time with your family. Try adjusting your workload to other days," based on the information that quantifies the relationship between the user's behavior and the office building usage time shown in the polygonal chart (radar chart) in Figure 12B, and the daily life behavior information of the user's family.

[0107] In Figure 13, when the user selects "Volunteer Mode," the behavioral analysis device 1 displays a recommendation for "Volunteer Mode" such as "Blood donations are scheduled during the hours the office building is open. Do you have your blood donation card and blood donation notebook ready?" based on the information that quantifies the relationship between the user's behavior and the office building's usage hours, as shown in the polygonal chart (radar chart) in Figure 12B, and the information obtained from the event company's external server 61c.

[0108] 13, the behavior analysis device 1 allows the user to input a 10-point rating of the recommendation and optionally add comments such as requests through an interface 72, thereby enabling feedback of the evaluation of the processing of the recommendation content determination unit 16. Furthermore, the user's information may be changed and updated through a separate screen (not shown).

[0109] The present invention is not limited to the above-mentioned examples, and includes various modified examples. The above-mentioned examples have been described in detail to easily explain the present invention, and are not necessarily limited to those having all of the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. [Explanation of symbols]

[0110] 1 Behavior analysis device 11 Information acquisition department 12 Arithmetic section 121 Writing Department 122 Word Segmentation 123 Word Quantification Department 124 Similarity calculation unit 13 Storage section 14 Display section

Claims

1. an information acquisition unit that acquires information including at least sensor data generated in association with human behavior and converts the acquired information into words; a calculation unit that quantifies a similarity between a plurality of words associated with human behavior that have been converted into words by the information acquisition unit, and calculates a time series change in the relationship between the words; a display unit that displays the relationship between words calculated by the calculation unit or a time series change in the relationship between words; A behavioral analysis device comprising:

2. 2. The behavioral analysis apparatus according to claim 1, The information acquisition unit is notified of a name of a human action associated with the sensor data by a learning model of the action recognition unit. Behavior analysis device.

3. 2. The behavioral analysis apparatus according to claim 1, The calculation unit, a text creation unit that creates a document by listing the words converted into words by the information acquisition unit in a specific order; a word segmentation unit that segments the sentences created by the sentence creation unit into words for each sentence; a word quantification unit that receives words for each sentence divided by the word division unit, analyzes words present around a certain word in data consisting of at least one sentence, and quantifies the words in a vector format so that the more frequently words occur together, the greater the similarity; a similarity calculation unit that digitizes the relationship between the words digitized in the vector format by the word digitization unit using a similarity between the vectors; A behavioral analysis device comprising:

4. The behavioral analysis device according to claim 3, the similarity calculation unit calculates a cosine similarity as a vector similarity; The display unit displays the similarity of the vectors in a bar graph or a polygon chart. Behavior analysis device.

5. The behavioral analysis device according to claim 3, the similarity calculation unit calculates a probability of occurrence of a vector as a similarity of the vector from a distance of the vector; The display unit displays the similarity of the vectors in a bar graph or a polygon chart. Behavior analysis device.

6. The behavioral analysis device according to claim 3, The display unit displays the recommendation based on the relationship between the actions. Behavior analysis device.

7. The behavioral analysis device according to claim 6, The display unit displays the recommendation based on the relationship between the actions in accordance with the selected recommendation type. Behavior analysis device.

8. 2. The behavioral analysis apparatus according to claim 1, The above calculation calculates the time series changes of vectors between words by dividing the period into months, days, and seasons. Behavior analysis device.

9. An information acquisition unit acquires information including at least sensor data generated in association with human behavior, and converts the acquired information into words; a calculation unit that quantifies a similarity as a relationship between a plurality of words associated with human behavior that have been converted into words by the information acquisition unit, and calculates a time series change in the relationship between the words; a display unit displaying the relationship between words or a time series change in the relationship between words calculated by the calculation unit; A behavioral analysis method including:

Images