Activity management device, activity management system, and activity management method

Abstract

To provide activity management means for determining an activity record showing an activity related to a user with high accuracy while reducing a burden on the user even at an arbitrary location.SOLUTION: An activity management apparatus is provided with an activity prediction model training unit for training an activity prediction model that predicts activities for each time zone related to a first user based on first characteristic information obtained about the first user; an activity determination unit for generating an activity record that indicates a candidate for activities for each time zone related to a second user based on second characteristic information obtained about the second user using the activity prediction model trained by the activity prediction model training unit; and an activity record correction unit for presenting the activity record to the second user and generating a final activity record for determining activities for each time zone related to the second user based on a correction input received from the second user.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an activity management device, an activity management system, and an activity management method. [Background technology]

[0002] In recent years, the environment surrounding humanity has undergone significant changes, including the occurrence of natural disasters due to climate change and the spread of infectious diseases, which are causing major changes in living conditions around the world. At the same time, opportunities for the use of artificial intelligence (AI) are increasing, and the development and spread of fifth-generation mobile communication systems is also progressing. As the living environment surrounding humanity changes in this way, the infrastructure that supports society is using advanced technology to collect large amounts of data, and this big data is used to visualize social activity and behavioral information, leading to a transition to a data-driven industrial structure.

[0003] In this environment, interest in health is increasing more than ever before, and there is a demand to shift to the data-driven structure described above when it comes to health management. Traditionally, healthcare has revolved around "passive healthcare," where people get sick and receive treatment only after their symptoms start to worsen. However, in the future, it is expected that the weight of "proactive healthcare," which looks ahead to health management, and "preventive healthcare," which aims to prevent illness, will increase as healthy people collect and digitize their own data at home to track their health status. Furthermore, it is even possible that "predictive healthcare," which predicts future illnesses and provides health warnings, will emerge in the future.

[0004] In order to realize the above-mentioned "proactive medicine," "preventive medicine," and "predictive medicine," it is important to understand people's health conditions and lifestyle habits on a daily basis. Conventionally, a known means of understanding people's health conditions is to analyze information obtained from a wearable sensor device and estimate the activities (eating, sleeping, exercise) of the user, such as a subject wearing the sensor device.

[0005] However, in reality, the accuracy of the sensor device worn by the user is limited, and there is a problem that the information acquired from the sensor device alone cannot reliably estimate the activity related to the user. Several proposals have already been made to address this problem.

[0006] As one means for improving the accuracy of estimating user-related activities, for example, Japanese Patent Laid-Open Publication No. 2016-157196 (Patent Document 1) discloses a technology that, "When applying semi-supervised learning to behavior recognition, erroneous behavior labels are corrected based on characteristic behaviors for each stay location attribute, thereby generating training data for highly reliable behavior recognition. The training data generation means acquires behavior data, behavior characteristics, and map data, classifies the acquired behavior data by stay location, identifies stay location attributes for each stay location from the acquired map data, and obtains characteristic behaviors from the acquired behavior characteristics. Next, for each stay location, the most frequently recognized behavior for each user is obtained from the acquired behavior data, and the characteristic behavior for each stay location is compared with the most frequently recognized behavior for each user. If there is a mismatch, the label of the most frequently recognized behavior is corrected with the label of the characteristic behavior. Then, training data is generated by extracting user IDs, behavior labels, and features from the behavior data including the corrected labels for each user." [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-157196 Summary of the Invention [Problem to be solved by the invention]

[0008] The above-mentioned Patent Document 1 describes a means for improving the accuracy of behavior identification and generating highly reliable labeled data by correcting erroneous behavior labels based on characteristic behaviors for each user's stay location attribute. In other words, Patent Document 1 describes a method for improving the accuracy of behavior identification by correcting erroneous behavior labels based on characteristic behaviors for each user's stay location attribute. In other words, when the recognition accuracy of a user's activity is insufficient, a representative behavior associated with the user's stay location is determined as the user's activity. This method is said to improve the behavior recognition accuracy of the classifier even when the recognition accuracy of the classifier for user behavior is low.

[0009] However, in the method described in Patent Document 1, characteristic behaviors for each location attribute used to correct incorrect behavior labels are determined based on votes from other users, so recognition accuracy is limited in places where characteristic behaviors have not been determined in advance by voting, or in places such as the home where users perform various activities and it is difficult to identify one representative behavior. Another prior proposal involves presenting a questionnaire to users and encouraging them to record their activities, but the accuracy of this method is limited by the user's memory, and presenting the questionnaire frequently places a heavy burden on the user.

[0010] Therefore, an object of the present disclosure is to provide an activity management means that determines an activity record that shows activities related to a user with high accuracy, even in any location, while reducing the burden on the user. [Means for solving the problem]

[0011] In order to solve the above-mentioned problems, one representative activity management device of the present invention includes: an activity prediction model training unit that trains an activity prediction model that predicts activities for each time period related to a first user based on first characteristic information acquired about the first user; an activity determination unit that uses the activity prediction model trained by the activity prediction model training unit to generate an activity record that indicates candidate activities for each time period related to the second user based on second characteristic information acquired about the second user; and an activity record correction unit that presents the activity record to the second user and generates a final activity record that confirms the activities for each time period related to the second user based on correction input received from the second user. [Effects of the Invention]

[0012] According to the present disclosure, it is possible to provide an activity management means that determines an activity record that shows activities related to a user with high accuracy, even in any location, while reducing the burden on the user. Problems, configurations, and effects other than those described above will become apparent from the following description of the preferred embodiment of the invention. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating a computer system for implementing an embodiment of the present disclosure. [Figure 2] FIG. 2 is a diagram illustrating an example of a configuration of an activity management system according to an embodiment of the present disclosure. [Figure 3] FIG. 3 is a diagram illustrating an example of the flow of the activity prediction model training process according to the embodiment of the present disclosure. [Figure 4] FIG. 4 is a diagram illustrating an example of the flow of an activity record generation process according to an embodiment of the present disclosure. [Figure 5] FIG. 5 is a diagram illustrating an example of the flow of an activity record correction process according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is a diagram illustrating an example of characteristic information according to an embodiment of the present disclosure. [Figure 7]FIG. 7 is a diagram illustrating an example of a user interface that presents an activity record to a user according to an embodiment of the present disclosure. [Figure 8] FIG. 8 is a diagram illustrating an example of a correction input for correcting an activity record shown in a user interface according to an embodiment of the present disclosure. [Figure 9] FIG. 9 is a diagram illustrating an example of a user's actual activity, activity diary information recorded by the user, and a final activity record according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are designated by the same reference numerals. Furthermore, although terms such as "first," "second," and "third" may be used to describe various elements or components in this disclosure, it will be understood that these elements or components should not be limited by these terms. These terms are used only to distinguish one element or component from another. Thus, a first element or component discussed below could also be referred to as a second element or component without departing from the teachings of the inventive concept.

[0015] Next, referring to FIG. 1, a computer system 100 for implementing embodiments of the present disclosure will be described. The mechanisms and devices of various embodiments disclosed herein may be applied to any suitable computing system. The main components of the computer system 100 include one or more processors 102, memory 104, a terminal interface 112, a storage interface 113, an I / O (input / output) device interface 114, and a network interface 115. These components may be interconnected via a memory bus 106, an I / O bus 108, a bus interface unit 109, and an I / O bus interface unit 110.

[0016] Computer system 100 may include one or more general-purpose programmable central processing units (CPUs) 102A and 102B, collectively referred to as processors 102. In some embodiments, computer system 100 may include multiple processors, while in other embodiments, computer system 100 may be a single CPU system. Each processor 102 executes instructions stored in memory 104 and may include an on-board cache.

[0017] In one embodiment, memory 104 may include random-access semiconductor memory, storage devices, or storage media (either volatile or non-volatile) for storing data and programs. Memory 104 may store all or part of the programs, modules, and data structures that implement the functions described herein. For example, memory 104 may store activity management application 150. In one embodiment, activity management application 150 may include instructions or descriptions that execute the functions described below on processor 102.

[0018] In some embodiments, activity management application 150 may be implemented in hardware via semiconductor devices, chips, logic gates, circuits, circuit cards, and / or other physical hardware devices instead of or in addition to a processor-based system. In some embodiments, activity management application 150 may include data other than instructions or descriptions. In some embodiments, cameras, sensors, or other data input devices (not shown) may be provided to communicate directly with bus interface unit 109, processor 102, or other hardware of computer system 100.

[0019] Computer system 100 may include a bus interface unit 109 that facilitates communication between processor 102, memory 104, display system 124, and I / O bus interface unit 110. I / O bus interface unit 110 may couple to an I / O bus 108 for transferring data to and from various I / O units. I / O bus interface unit 110 may communicate via I / O bus 108 with multiple I / O interface units 112, 113, 114, and 115, also known as I / O processors (IOPs) or I / O adapters (IOAs).

[0020] Display system 124 may include a display controller, a display memory, or both. The display controller may provide video, audio, or both data to display device 126. Computer system 100 may also include one or more sensors or other devices configured to collect data and provide the data to processor 102.

[0021] For example, computer system 100 may include biometric sensors that collect heart rate data, stress level data, etc., environmental sensors that collect humidity data, temperature data, pressure data, etc., and motion sensors that collect acceleration data, movement data, etc. Other types of sensors may also be used. Display system 124 may be connected to a display device 126, such as a standalone display screen, a television, a tablet, or a handheld device.

[0022] The I / O interface unit provides functionality for communicating with various storage or I / O devices. For example, the terminal interface unit 112 may be attached to user I / O devices 116, such as user output devices such as a video display, a television with speakers, and user input devices such as a keyboard, a mouse, a keypad, a touchpad, a trackball, buttons, a light pen, or other pointing device. A user may use a user interface to enter input data or instructions into the user I / O devices 116 and the computer system 100, and receive output data from the computer system 100, by operating the user input devices. The user interface may be displayed on a display, played through speakers, or printed via a printer via the user I / O devices 116, for example.

[0023] Storage interface 113 allows attachment of one or more disk drives or direct access storage device 117 (typically a magnetic disk drive storage device, but may also be an array of disk drives or other storage devices configured to appear as a single disk drive). In some embodiments, storage device 117 may be implemented as any secondary storage device. The contents of memory 104 may be stored in storage device 117 and retrieved as needed from storage device 117. I / O device interface 114 may provide an interface to other I / O devices, such as printers, fax machines, etc. Network interface 115 may provide a communications path that allows computer system 100 and other devices to communicate with each other. This communications path may be, for example, network 130.

[0024] In some embodiments, computer system 100 may be a device that receives requests from other computer systems (clients) without a direct user interface, such as a multi-user mainframe computer system, a single-user system, or a server computer. In other embodiments, computer system 100 may be a desktop computer, a portable computer, a laptop, a tablet computer, a pocket computer, a telephone, a smartphone, or any other suitable electronic device.

[0025] Next, an activity management system according to an embodiment of the present disclosure will be described with reference to FIG.

[0026] 2 is a diagram illustrating an example of a configuration of an activity management system 200 according to an embodiment of the present disclosure. As shown in FIG. 2, the activity management system 200 includes a first information management unit 210, a communication network 225, a second information management unit 220, an activity management device 230, and a user terminal 240. In the activity management system 200, the first information management unit 210, the second information management unit 220, the activity management device 230, and the user terminal 240 may be connected to each other via the communication network 225.

[0027] The first information management unit 210 is a functional unit that acquires first characteristic information and activity diary information from the first user 205. In one embodiment, the first information management unit 210 is a functional unit that aggregates information acquired by a sensor device (not shown in FIG. 2 ) worn by the first user 205 and information input by the first user 205, and stores the information in the first characteristic information DB 212 and the activity diary information DB 214. In one embodiment, the first information management unit 210 may be a terminal (smartphone, smartwatch, tablet, or PC) used by the first user 205. Furthermore, the first user 205 may be one or more, but from the viewpoint of obtaining high-quality information, it is desirable to have as many first users 205 as possible. Furthermore, when there are multiple first users 205, they may be divided into groups based on, for example, gender, age, etc.

[0028] The first characteristic information DB 212 in the first information management unit 210 is a database for storing the first characteristic information acquired by the first information management unit 210. In the present disclosure, "characteristic information" refers to information acquired to determine an activity related to a user, and may include, for example, motion information measuring the user's movement (e.g., acceleration information, gyro information, etc.), biometric information related to the user's body (e.g., heart rate, etc.), and user location information.

[0029] The activity diary information DB 214 in the first information management unit 210 is a database for storing activity diary information acquired by the first information management unit 210. In the present disclosure, the "activity diary information" may be information recorded by the first user 205 himself / herself to indicate activities for each time period related to the first user 205. As will be described later, this activity diary information is used to estimate activities corresponding to the user state in the activity prediction model training process.

[0030] In addition, in this disclosure, "activity" refers to an action or activity that characterizes a user's behavior, and may include any matter, such as "exercise," "work," "watch a movie," "eat," etc. Furthermore, the expression "user-related activity" includes both an activity that a specific user actively performs on their own and an activity that a user passively experiences. As described below, the first characteristic information stored in the first characteristic information DB 212 and the activity diary information stored in the activity diary information DB 214 are information that can be accessed from the activity management device 230 via the communication network 225.

[0031] Second information management unit 220 is a functional unit that acquires second characteristic information from second user 215. In one embodiment, second information management unit 220 is a functional unit that aggregates information acquired by a sensor device (not shown in FIG. 2) worn by second user 215 and stores the information in second characteristic information DB 222. In one embodiment, second information management unit 220 may be a terminal (smartphone, smartwatch, tablet, or personal computer) used by second user 215.

[0032] Furthermore, second characteristic information DB222 in second information management unit 220 is a database for storing second characteristic information acquired by second information management unit 220. As will be described later, the second characteristic information stored in second characteristic information DB222 is information accessible from activity management device 230 via communication network 225.

[0033] The communication network 225 may include, for example, a local area network (LAN), a wide area network (WAN), a satellite network, a cable network, a WiFi network, or any combination thereof. The first characteristic information, the activity diary information, and the second characteristic information may be transmitted and received in the activity management system 200 via the communication network 225.

[0034] The activity management device 230 is a device for generating an activity record showing activities related to a user for each time period. As shown in Fig. 2, the activity management device 230 includes an activity prediction model training unit 232, an activity determination unit 234, an activity record correction unit 236, an activity prediction model 237, and a second activity prediction model 238. The various functional units included in the activity management device 230 may be software modules constituting the activity management application 150 shown in FIG. 1, or may be independent dedicated hardware devices. The functional units may be implemented in the same computing environment or in distributed computing environments.

[0035] The activity prediction model training unit 232 is a functional unit for training the activity prediction model 237 that predicts the user's activity for each time period and the second activity prediction model 238. More specifically, the activity prediction model training unit 232 may acquire first characteristic information acquired about the first user 205 from the first characteristic information DB 212 and acquire activity diary information from the activity diary information DB 214, and then train the activity prediction model 237 that predicts the activity for each time period related to the first user 205 based on the first characteristic information and the activity diary information.

[0036] The activity prediction model training unit 232 may use any existing prediction method, such as a Classification Model, a Clustering Model, a Forecast Model, an Outliers Model, a Time Series Model, a Random Forest, a Generalized Linear Model, a Gradient Boosted Model, K-Means, Analysis of Variance, Regression, or a Neural Network.

[0037] Furthermore, the activity prediction model training unit 232 may train a second activity prediction model 238 that has further improved prediction accuracy compared to the activity prediction model 237, based on the final activity record described below. The processing related to the activity prediction model training unit 232 will be described later with reference to FIG. 3, and therefore will not be described here.

[0038] The activity determination unit 234 is a functional unit that generates an activity record indicating activities for each time period related to the user, using the activity prediction model 237 (or the second activity prediction model 238) trained by the activity prediction model training unit 232. More specifically, the activity determination unit 234 may generate an activity record indicating candidates for activities for each time period related to the second user 215 (hereinafter also referred to as "activity candidates"), based on the second characteristic information acquired about the second user 215, using the activity prediction model 237 trained by the activity prediction model training unit 232 based on the first characteristic information and activity diary information acquired about the first user 205, for example.

[0039] In this disclosure, the term "time period" refers to a period between certain times, and the start and end times may be arbitrary. In some embodiments, these time periods may start and end at the start and end times of a particular user activity (i.e., the range of the time period is defined by the user's particular activity). Also, the activity log may show the user's activity over a day or over several hours. In some embodiments, the time range of the activity shown in the activity log may be selected by the user. The processing performed by the activity determining unit 234 will be described later with reference to FIG. 4, and therefore will not be described here.

[0040] The activity record correction unit 236 is a functional unit that presents the activity record generated by the activity determination unit 234 to the second user 215 and generates a final activity record that confirms the activities for each time period related to the second user 215, based on correction input received from the second user 215. In an embodiment, the activity record correction unit 236 may present the activity record to the second user 215 via a user terminal 240, which will be described later. The processing related to the activity record correcting unit 236 will be described later with reference to FIG. 5, and therefore will not be described here.

[0041] The user terminal 240 is a terminal used by the second user 215. The user terminal 240 may include an interface unit 242 that presents information to the second user 215 and accepts information input. In an embodiment, the user terminal 240 may present the activity record generated by the activity determination unit 234 to the second user 215 via a user interface provided by the interface unit 242, and accept correction input to correct the content of the activity record.

[0042] As described above, the activity management system 200 can provide an activity management means for determining an activity record that shows activities related to a user with high accuracy, even in any location, while reducing the burden on the user.

[0043] Next, an activity prediction model training process according to an embodiment of the present disclosure will be described with reference to FIG.

[0044] 3 is a diagram illustrating an example of the flow of activity prediction model training processing 300 according to an embodiment of the present disclosure. The activity prediction model training processing 300 is processing for training an activity prediction model that predicts an activity of a given user for each time period based on characteristic information of the user, and is performed by the activity prediction model training unit 232 described above.

[0045] First, in step S305, the activity prediction model training unit 232 acquires first characteristic information from the first characteristic information DB 212 of the first information management unit 210. As described above, this first characteristic information is information acquired to determine an activity related to the first user 205, and may include, for example, motion information measuring the movement of the first user 205 (e.g., acceleration information, gyro information, etc.), biological information related to the body of the first user 205 (e.g., heart rate, etc.), and user position information.

[0046] Next, in step S310, the activity prediction model training unit 232 calculates a threshold (first threshold) for distinguishing different user states associated with the first user 205 from the first characteristic information acquired in step S305. In the present disclosure, the expression "user state" refers to information indicating the user's posture or movement, and may include categories such as "lying down," "sitting," "standing," and "moving." Furthermore, the threshold for distinguishing the user state includes acceleration values, gyro values, and heart rate values ​​that define different user states such as "lying down," "sitting," "standing," and "moving." In an embodiment, the activity prediction model training unit 232 may calculate an individual threshold for distinguishing each user state.

[0047] In one embodiment, the activity prediction model training unit 232 may calculate a threshold for distinguishing between different user states using a function including the acceleration in the x-direction, y-direction, and z-direction of the first user indicated in the characteristic information, the gyro values ​​in the x-direction, y-direction, and z-direction, and the heart rate, as shown in the following Equation 1.

number

[0048] Next, in step S315, the activity prediction model training unit 232 uses the threshold calculated in step S310 to determine the user state of the first user 205 for each time period from the first characteristic information, thereby generating first time-slot state information indicating the user state for each time period. In one embodiment, the activity prediction model training unit 232 may identify, for each time period in the first characteristic information, a threshold satisfied by the acceleration value, gyro value, and heart rate value corresponding to that time period from among multiple thresholds calculated for each user state, and may set the user state corresponding to the identified threshold as the user state for that time period. As an example, the activity prediction model training unit 232 may generate first time-slot state information such as "2021 / 10 / 11, 00:13 to 07:47 - lying down; 2021 / 10 / 11, 07:48 to 08:30 - moving; 2021 / 10 / 11, 08:31 to 09:00 - sitting."

[0049] Next, in step S320, the activity prediction model training unit 232 acquires activity diary information of the first user 205 from the activity diary information DB 214 of the first information management unit 210. As described above, this activity diary information is information recorded by the first user 205 himself / herself to indicate activities for each time period related to the first user 205. As an example, this activity diary information may indicate "2021 / 10 / 11, 00:13 to 07:47 - Sleep; 2021 / 10 / 11, 07:48 to 08:30 - Getting ready; 2021 / 10 / 11, 08:31 to 09:00 - Breakfast."

[0050] Next, in step S325, the activity prediction model training unit 232 generates first time-zone activity information indicating a user state and activity for each time zone related to the first user 205, based on the first time-zone status information generated in step S315 and the activity diary information acquired in step S320. More specifically, the activity prediction model training unit 232 may generate the first time-zone activity information by associating (mapping) the user state indicated in the first time-zone status information with an activity corresponding to the same time zone in the activity diary information. For example, the first time period status information indicates "2021 / 10 / 11, 00:13~07:47 - lying down; 2021 / 10 / 11, 07:48~08:30 - moving; 2021 / 10 / 11, 08:31~09:00 - sitting", and the activity diary information indicates "2021 / 10 / 11, 00:13~07:47 - sleeping; 2021 / 10 / 11, 07:48~08:30 - In the case where the activity information indicates "Getting ready; 2021 / 10 / 11, 08:31-09:00 - Breakfast," the activity prediction model training unit 232 may generate "2021 / 10 / 11, 00:13-07:47 - Lying down / Sleeping; 2021 / 10 / 11, 07:48-08:30 - Moving / Getting ready; 2021 / 10 / 11, 08:31-09:00 - Sitting / Breakfast" as the first activity information for each time period.

[0051] Next, in step S330, the activity prediction model training unit 232 calculates the elapsed time and frequency for each user state for each activity based on the first time-slot-specific activity information generated in step S325. More specifically, the activity prediction model training unit 232 calculates the cumulative elapsed time T lay , the cumulative elapsed time T spent by the first user in the user state “sitting” sit , the cumulative elapsed time T spent by the first user in the user state “standing” stand , the cumulative elapsed time T that the first user spent in the user state “moving” move , the number of times the first user has transitioned to the user state "lying down" F lay , the number of times the first user has transitioned to the user state "sitting" F sit , the number of times the first user transitioned to the user state "standing" F stand , and the number of times the first user has transitioned to the user state “moving” F move is calculated from the time information of the first time period activity information and the number of times each user state is changed.

[0052] Next, in step S335, the activity prediction model training unit 232 generates, for each activity, an activity function that characterizes the activity, based on the elapsed time and frequency of each user state calculated for each activity in step S330, and the heart rate and location of the first user indicated in the first characteristic information. As an example, the activity prediction model training unit 232 may generate an activity function as shown in Equation 2. In this way, for each activity, the relationship between each user state corresponding to the activity, the frequency of the user state, the heart rate, and the location can be expressed.

number

[0053] Next, in step S340, the activity prediction model training unit 232 determines whether or not there are identical activity functions corresponding to multiple activities among the activity functions generated in step S335. If there are no identical activity functions corresponding to multiple activities, the process proceeds to step S345. If there are identical activity functions corresponding to multiple activities, the process proceeds to step S350.

[0054] If it is determined that the same activity function corresponding to a plurality of activities does not exist, then in step S345, the activity prediction model training unit 232 finalizes the activity function generated in step S335.

[0055] If it is determined that the same activity function corresponding to multiple activities exists, in step S350, the activity prediction model training unit 232 calculates the occurrence probability of each activity based on the occurrence frequency of each activity in the activity diary information acquired in step S320. Among the activities related to the first user 205 shown in the activity diary information, the more frequently the activity occurs, the higher the occurrence probability. Here, the process of calculating the occurrence probability based on the occurrence frequency of the activity may be performed using, for example, an existing statistical analysis method.

[0056] Thereafter, the activity prediction model training unit 232 assigns the activity functions corresponding to the multiple activities to the activity with the highest calculated occurrence probability, and then finalizes the generated activity function. As an example, if a particular activity function corresponds to two activities, "watching movies" and "reading books," and "watching movies" has a "70%" probability of occurrence, and "reading books" has a "30%" probability of occurrence, then the activity function is associated with the activity "watching movies." As a result, when there are multiple possible activities for one user state, for example, "sitting," the activity that occurs most frequently in the activity diary information of the user is preferentially estimated.

[0057] Next, in step S340, the activity prediction model training unit 232 saves the activity function determined in step S345 or step S350 as an activity prediction model (for example, the activity prediction model 237 shown in FIG. 2). In an embodiment, the activity prediction model training unit 232 may save the determined activity function in association with characteristics of the first user 205, such as gender and age.

[0058] According to the activity prediction model training process 300 described above, it is possible to train an activity prediction model that predicts the activities of a given user for each time period based on the user's characteristic information. Furthermore, as will be described later, by using the activity prediction model trained in this way, it is possible to generate an activity record that accurately shows activities related to the user, even in any location, while reducing the burden on the user.

[0059] Next, an activity record generation process according to an embodiment of the present disclosure will be described with reference to FIG.

[0060] 4 is a diagram illustrating an example of the flow of an activity record generation process 400 according to an embodiment of the present disclosure. The activity record generation process 400 is a process for generating an activity record indicating activities for each time period related to the second user 215 based on second characteristic information acquired from the second user 215, using the activity prediction model trained by the activity prediction model training process 300 described above, and is executed by the activity determination unit 234 described above.

[0061] First, in step S405, the activity determination unit 234 acquires second characteristic information from the second characteristic information DB 222 of the second information management unit 220. As described above, this second characteristic information is information acquired to determine an activity related to the second user 215, and may include, for example, motion information (e.g., acceleration information, gyro information, etc.) that measures the movement of the second user 215, biological information related to the body of the second user (e.g., heart rate, etc.), and position information of the second user 215.

[0062] Next, in step S410, the activity determination unit 234 calculates a threshold (second threshold) for distinguishing different user states related to the second user 215 from the second characteristic information acquired in step S405. Here, the method for calculating the threshold for distinguishing the user state from the second characteristic information is substantially similar to step S310 in the activity prediction model training process 300 shown in Fig. 3, and therefore description thereof will be omitted here.

[0063] Next, in step S415, the activity determination unit 234 generates second time-slot state information indicating the user state for each time slot from the second characteristic information using the threshold calculated in step S410. Here, the method for generating the second time-slot state information is substantially the same as step S315 in the activity prediction model training process 300 shown in Fig. 3, and therefore a description thereof will be omitted here.

[0064] Next, in step S420, the activity determination unit 234 acquires an activity function of the activity prediction model 237 trained by the above-described activity prediction model training process 300. In an embodiment, the activity determination unit 234 may determine characteristics (gender, age, etc.) of the second user 215 from which the second characteristic information has been acquired, and acquire an activity function corresponding to the determined characteristics from the activity prediction model 237. A user who shares characteristics such as gender and age with the second user 215 is likely to have similar lifestyle habits and behaviors to the second user 215, so the accuracy of activity records can be improved by using an activity function generated based on characteristic information obtained from information about such users.

[0065] Next, in step S425, the activity determination unit 234 generates an activity record using the second time-slot state information generated in step S415 and the activity function acquired in step S420. More specifically, the activity determination unit 234 estimates activities related to the second user 215 for each time slot by substituting the second time-slot state information indicating the user state for each time slot and the heart rate and location of the second user 215 included in the second characteristic information into the activity function acquired in step S420. Thereafter, by aggregating the activities estimated for each time slot, an activity record indicating candidate activities for each time slot related to the second user 215 can be generated.

[0066] Here, "potential activities for each time period" refers to activities that the second user 215 may have performed in a certain time period. However, since the potential activities here are merely results predicted by the activity prediction model 237, they may differ from the actual activities of the second user 215.

[0067] Next, in step S430, the activity determination unit 234 determines whether or not there are identical time periods corresponding to multiple activity candidates in the activity record generated in step S415. If there are no identical time periods corresponding to multiple activity candidates, the process proceeds to step S435, and if there are identical time periods corresponding to multiple activity candidates, the process proceeds to step S440.

[0068] If it is determined that there is no identical time period corresponding to multiple activity candidates, then in step S435, the activity determination unit 234 finalizes the activity record generated in step S425.

[0069] If it is determined that the same time period corresponds to multiple activity candidates, in step S440, the activity determination unit 234 calculates the occurrence probability of each activity candidate based on the occurrence frequency of each activity in the activity diary information acquired in step S320 in the activity prediction model training process 300 shown in Fig. 3. The occurrence probability of each activity candidate calculated here may be included in the activity record presented to the second user 215.

[0070] As a result, if multiple activities (work, eating, watching TV, reading) can occur for one user state, such as "sitting," a higher occurrence probability is assigned to the activity that occurs more frequently in the user's activity diary information. The second user 215, who then checks the activity record, can refer to the calculated occurrence probability and select an actual activity from among the multiple candidate activities displayed in the activity record. In this way, even if multiple activities can occur for one user state, the user's activity can be determined with high accuracy. Correction input by the second user 215 will be described later, and therefore will not be described here. In one embodiment, the activity determination unit 234 may extract activity diary information of a user who shares characteristics such as gender and age with the second user from the activity diary information acquired in step S320, and calculate the probability of an activity occurring based on the extracted activity diary information.

[0071] Next, in step S445, the activity determination unit 234 saves the activity record finalized in step S435 or step S440 in a predetermined storage area. Thereafter, the activity record may be presented to the second user and modified by the second user, as shown in activity record modification processing 500 described below.

[0072] According to the activity record generation process 400 described above, an activity record indicating candidate activities for each time period related to the second user 215 can be generated based on the second characteristic information acquired from the user, using the activity prediction model trained by the activity prediction model training process 300 described above.

[0073] Next, an activity record correction process according to an embodiment of the present disclosure will be described with reference to FIG.

[0074] 5 is a diagram showing an example of the flow of an activity record correction process 500 according to an embodiment of the present disclosure. Fig. 5 shows a process for generating a final activity record that more accurately shows the user's activities by presenting the activity record generated by the above-described activity record generation process 400 to the user and correcting the activity record based on correction input from the user, and is performed by the above-described activity record correction unit 236.

[0075] First, in step S505, the activity record correction unit 236 presents the activity record generated by the above-described activity record generation process 400 to the second user 215. More specifically, the activity record correction unit 236 retrieves the activity record generated by the activity record generation process 400 from the storage area, then transmits it to the user terminal 240 via the communication network 225, and displays it on the interface unit 242 of the user terminal 240. The user interface for displaying the activity record will be described with reference to FIGS. 7 and 8, and therefore will not be described here.

[0076] Next, in step S510, the activity record correction unit 236 accepts an input indicating whether or not the activity record presented to the second user 215 in step S505 has any deficiencies. As described above, this activity record is information indicating candidate activities predicted by the activity prediction model 237 based on the second characteristic information acquired from the second user 215, and therefore may differ from the actual activities of the second user 215. Therefore, if the activity record presented to the second user 215 in step S505 has any deficiencies (i.e., if the candidate activities or the activity times indicated in the activity record differ from the actual activities of the second user 215), the second user 215 can correct the activity record after inputting an indication to that effect to the interface unit 242 of the user terminal 240. The input indicating that the activity record has any deficiencies may be input to the interface unit 242 of the user terminal 240 and transmitted to the activity record correction unit 236 via the communication network 225.

[0077] If there is a user input indicating that the activity record presented to the second user 215 in step S505 is incomplete, the process proceeds to step S515. On the other hand, if there is no user input indicating that the activity record presented to the second user 215 in step S505 is incomplete (i.e., the activity record is not incomplete and accurately shows the activities of the second user 215), the process proceeds to step S520.

[0078] In step S515, the activity record correction unit 236 receives a correction input from the second user 215 for correcting the activity record presented to the second user 215 in step S505. This correction input for correcting the activity record may be, for example, an input for changing the activity candidate displayed for a specific time period in the activity record, or for changing the start time or end time of a specific activity. In addition, this correction input may be input to the interface unit 242 of the user terminal 240 and transmitted to the activity record correction unit 236 via the communication network 225. As an example, the second user 215 may provide a correction input that changes the suggested activity "eating" to "watching TV." After the correction input is accepted, the process proceeds to step S520.

[0079] Next, in step S520, the activity record correction unit 236 generates a final activity record. This final activity record is an activity record that accurately shows the actual activities of the second user 215. Here, if the activity record correction unit 236 receives correction input from the second user 215 in step S515, it may generate the final activity record by reflecting the changes of the correction input in the activity record presented to the second user 215 in step S505. Furthermore, if there is no user input indicating that the activity record presented to the second user 215 in step S510 is incomplete, the activity record correction unit 236 may confirm the content of the activity record presented to the second user 215 in step S510 and use it as the final activity record.

[0080] Next, in step S525, the activity prediction model may train a second activity prediction model (for example, second activity prediction model 238 shown in FIG. 2 ) using the final activity record generated in step S520 as learning data. Like activity prediction model 237, second activity prediction model 238 is a prediction model for generating an activity record indicating hourly activities related to a user based on the user's characteristic information. However, since second activity prediction model 238 is trained using the final activity record that accurately indicates the user's activities and the times of the activities as learning data, it is a prediction model with improved prediction accuracy compared to activity prediction model 237.

[0081] The above-described activity record correction process 500 can generate a final activity record that accurately indicates the user's activities. Furthermore, by training a prediction model using the final activity record generated in this way as learning data, a second activity prediction model that accurately indicates the hourly activities of any user based on the user's characteristic information can be obtained.

[0082] Next, with reference to FIG. 6, the characteristic information according to the embodiment of the present disclosure will be described.

[0083] 6 is a diagram illustrating an example of characteristic information 600 according to an embodiment of the present disclosure. As described above, this characteristic information 600 is information acquired to determine an activity related to a user, and may include, for example, motion information measuring the user's movement (e.g., acceleration information, gyro information, etc.), biometric information related to the user's body (e.g., heart rate, etc.), and user location information. Furthermore, the characteristic information 600 illustrated in FIG. 6 may be characteristic information acquired from any user, such as the first user 205 or the second user 215.

[0084] More specifically, as shown in FIG. 6, the characteristic information 600 may indicate, for each time 605, the user's acceleration information 610 in the x-, y-, and z-directions, gyro information 615 in the x-, y-, and z-directions, heart rate 620, and position 625. In Figure 6, a specific area (such as a bedroom) in a predetermined location (such as a home) is shown as an example of the user's location 625, but the present disclosure is not limited to this, and the user's location 625 may also be expressed in latitude and longitude.

[0085] As described above, the activity related to the user can be determined according to the user characteristic information 600 shown in FIG.

[0086] Next, with reference to FIGS. 7 and 8, an example of a case where an activity record according to an embodiment of the present disclosure is presented to a user and corrected based on correction input from the user to generate a final activity record will be described.

[0087] 7 is a diagram illustrating an example of a user interface 700 that presents an activity record to a user according to an embodiment of the present disclosure. As illustrated in FIG. 7, the user interface 700 includes a user ID 705 that identifies the user, a date 710 that indicates the date of the activity record, an activity record 715 that indicates activity candidates for each time period associated with the user, and an activity modification button 720 that modifies the activity candidates in the activity record 715.

[0088] The activity record 715 here is generated, for example, by the activity record generation process 400 shown in FIG. 4, and is information indicating candidate activities for each time period related to the user. More specifically, the activity record 715 includes time 716, user states 717 for each time 716, and candidate activities 718 for each time 716. As shown in FIG. 7, the user states 717 indicate different design styles (patterns, colors, etc.) for different user states. The design styles corresponding to each user state are indicated in symbol information 719.

[0089] The activity record 715 also shows candidate activities 718 for each time period 716. For example, as shown in FIG. 7 , for each time period within the time period 716, the user interface 700 may display candidate activities 718 predicted for that time period, such as "sleep," "walk," "work," "housework," and "meal." Each candidate activity 718 may also be associated with a probability value. This probability value may be, for example, the occurrence probability of each activity calculated in step S440 of the activity record generation process 400 shown in FIG. 4 . As shown in FIG. 7 , in one embodiment, candidate activities 718 with high probability values ​​may be displayed in a design style or position that emphasizes them.

[0090] Using the activity correction button 720, a user viewing the user interface 700 may input corrections to modify the candidate activities 718 for each time period. Details of this correction input will be described later with reference to FIG. 8. In addition, in one embodiment, if there are no errors in the candidate activities 718, the user may confirm the candidate activities 718 by using the activity correction button 720 and inputting an input indicating that there are no errors.

[0091] 8 is a diagram illustrating an example of a correction input for correcting an activity record shown on the user interface 700 according to an embodiment of the present disclosure. As described above, a user viewing the user interface 700 may use the activity correction button 720 to confirm the activity candidates 718 that are free of defects and to input a correction input to correct the activity candidates 718 that are defective.

[0092] More specifically, if the activity candidate 718 corresponding to a particular time period is correct (i.e., matches the user's actual activity), a user viewing the user interface 700 may confirm the activity candidate using the activity correction button 720. For example, if the activity candidate "Sleep" corresponding to the time period "06:12:20" is correct, the user may confirm the activity candidate as "Sleep" using the activity correction button 720.

[0093] Furthermore, if multiple candidate activities 718 corresponding to a specific time period are displayed and a correct activity that matches the user's actual activity is included among these candidate activities 718, the user checking the user interface 700 may confirm the candidate activity by selecting the correct candidate activity 718 using the activity correction button 720. For example, if two candidate activities 718, "walk" and "housework," are displayed for the time period "06:12:20 to 08:07:10," and the user's actual activity during this time period was "walk," the user may confirm the candidate activity as "walk" using the activity correction button 720. As shown in FIG. 8, the activity candidates 718 that have been confirmed by the user may be displayed in a design style (color or pattern) that is different from the activity candidates 718 that have not been confirmed.

[0094] On the other hand, if there is an error in the activity candidates 718 displayed for a particular time period (e.g., none of the displayed activity candidates 718 match the user's actual activities, or the start or end times of the activity candidates 718 are incorrect), the user viewing the user interface 700 may use the activity correction button 720 to input the correct activity and time. As an example, if three activity candidates 718, "Work," "Watching TV," and "Video Games," are displayed for the time period "08:07:10 to 10:47:00," but the user actually performed two different activities, "Watching TV" and "Work," during that time period, the user may use the activity correction button 720 to input a correction 820 indicating the actual activity and the correct time for that activity, such as "08:07 to 08:45: Watching TV; 08:45 to 10:47: Work."

[0095] 7 and 8 can be accessed by the user at any time via the interface unit 242 of the user terminal 240, and therefore, input of corrections to correct the activity record may be performed at a time and frequency that is convenient for the user. For example, in one embodiment, the user may access the user interface 700 once a day (for example, before going to bed) to check the activity record for that day and correct it as necessary.

[0096] According to the user interface 700 of the embodiment of the present disclosure described above, a user can input corrections to modify the activity record, thereby generating a final activity record that accurately indicates the user's actual activities and the time of those activities. Furthermore, in the activity record of the embodiment of the present disclosure, the user's activity options are narrowed down to activity candidates predicted based on the user's characteristic information, which reduces the burden on the user compared to, for example, a method of filling out a questionnaire about activities.

[0097] Next, with reference to FIG. 9, a description will be given of a user's actual activity, activity diary information recorded by the user, and a final activity record according to an embodiment of the present disclosure.

[0098] 9 is a diagram illustrating an example of a user's actual activity 910, activity diary information 920 recorded by the user, and a final activity record 930 according to an embodiment of the present disclosure. As described above, in order to understand people's health conditions and lifestyle habits, it is necessary to accurately determine a person's activity. Conventionally, there have been proposals to present a questionnaire to a user and encourage the user to record their own activity, but accuracy is limited by the user's memory, and there may be a discrepancy between the user's actual activity and the activity diary created by the user as a response to the questionnaire. In FIG. 9, each activity in the user's actual activity 910, activity diary information 920 recorded by the user, and final activity record 930 according to the embodiment of the present disclosure is represented by a pattern indicated by symbol information 905.

[0099] For example, as shown in FIG. 9, when comparing a user's actual activities 910 with activity diary information 920 recorded by the user himself, the activity diary information 920 records the user's general activities, but does not record activities performed in a short period of time (for example, housework performed in between work, watching television in between housework, etc.).

[0100] On the other hand, according to the activity management means of an embodiment of the present disclosure, an activity record showing possible activities of the user is generated based on characteristic information obtained from the user, and then this activity record is presented to the user and modified based on correction input from the user, thereby creating a final activity record 930 that accurately shows the user's activities.

[0101] Comparing the final activity record 930 with the user's actual activity 910 reveals that not only the user's general activities but also short-term activities (such as housework between work or watching TV between housework) are accurately recorded. By training a prediction model using the final activity record generated in this way as learning data, it is possible to obtain an activity prediction model that accurately indicates the activities and time periods of any user based on the user's characteristic information.

[0102] The activity management means described above generates an activity record showing candidate activities for a user for each time period based on the user's characteristic information. The activity record is then presented to the user, and the user can modify the activity record based on the user's input, thereby creating a final activity record that accurately shows the user's activities. In this activity record, the user's activity options are narrowed down to candidate activities predicted based on the user's characteristic information. This reduces the burden on the user, as opposed to, for example, methods such as filling out a questionnaire. Furthermore, because the candidate activities are modified by the user themselves, the user's activities can be recorded with higher accuracy than, for example, conventional methods that estimate user activities based on the user's location attributes. This makes it possible to accurately determine user activities even when activities are performed in a short period of time or when multiple activities may occur for a single user state. In this way, it is possible to provide an activity management means that determines an activity record that shows activities related to a user with high accuracy, even in any location, while reducing the burden on the user.

[0103] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0104] 200 Activity Management System 205 First User 210 First Information Management Department 212 First characteristic information DB 214 Activity diary information DB 215 Second User 220 Second Information Management Department 222 Second characteristic information DB 225 Communication Network 230 Activity management device 232 Activity Prediction Model Training Department 234 Activity Judgment Department 236 Activity Record Correction Department 237 Activity Prediction Model 238 Second Activity Prediction Model 240 user terminals 242 Interface section

Claims

1. An activity management device, comprising: a processor and a memory, The memory includes: an activity prediction model training unit that trains an activity prediction model that predicts an activity for each time period related to a first user based on first characteristic information acquired about the first user; an activity determination unit that generates an activity record indicating candidate activities for each time period related to a second user based on second characteristic information acquired about the second user, using the activity prediction model trained by the activity prediction model training unit; and an activity record correction unit that presents the activity record to the second user and generates a final activity record that determines activities for each time period related to the second user based on correction input received from the second user; and processing instructions for causing the processor to function as The activity prediction model training unit In addition to the first characteristic information, activity diary information is acquired that is created by the first user and indicates activities for each time period related to the first user; generating first time-slot state information indicating a user state of the first user for each time slot from the first characteristic information; generating first activity information for each time slot indicating the activity of the first user for each time slot by associating the user status indicated in the first status information for each time slot with the activity indicated in the activity diary information; generating an activity function that characterizes each of the activities indicated in the activity diary information based on the first time-zone activity information, and saving the generated activity function as the activity prediction model; An activity management device comprising:

2. The first characteristic information and the second characteristic information are Including time information, acceleration information, gyro information, heart rate information, and location information, 2. The activity management device according to claim 1, wherein:

3. The activity prediction model training unit calculating a first threshold for determining a plurality of user states associated with the first user based on the first characteristic information; generating first time-zone state information indicating a user state of the first user for each time zone from the first characteristic information based on the calculated first threshold value; 3. The activity management device according to claim 2.

4. The user state is: Including lying down, sitting, standing and moving 4. The activity management device according to claim 3.

5. In the generated activity functions, when a first activity function corresponds to a first activity and a second activity, The activity prediction model training unit calculating an occurrence probability of the first activity based on an occurrence frequency of the first activity from the activity diary information; calculating an occurrence probability of the second activity based on an occurrence frequency of the second activity from the activity diary information; assigning the first activity function to the first activity if the probability of occurrence of the first activity is higher than the probability of occurrence of the second activity; 4. The activity management device according to claim 3.

6. The activity determination unit calculating a second threshold for determining a plurality of user states associated with the second user based on the second characteristic information; generating second time-zone state information indicating a user state of the second user for each time zone from the second characteristic information based on the calculated second threshold; generating the activity record based on the second time slot state information and the activity function of the activity prediction model; 4. The activity management device according to claim 3.

7. The activity prediction model training unit training a second activity prediction model based on the final activity record; 2. The activity management device according to claim 1, wherein:

8. a sensor device for acquiring characteristic information about a user; an information management unit that aggregates and stores the characteristic information acquired by the sensor device; an activity management device that generates an activity record showing activities for each time period related to the user based on the characteristic information stored in the information management unit; An activity management system connected to a user terminal that manages the activity record via a communication network, The activity management device includes: a processor and a memory, The memory includes: an activity prediction model training unit that trains an activity prediction model that predicts an activity for each time period related to a first user based on first characteristic information acquired about the first user; an activity determination unit that generates an activity record indicating candidate activities for each time period related to a second user based on second characteristic information acquired about the second user, using the activity prediction model trained by the activity prediction model training unit; and an activity record correction unit that presents the activity record to the second user via the user terminal and generates a final activity record that determines the activities for each time period related to the second user based on correction input received from the second user; and processing instructions for causing the processor to function as The activity prediction model training unit In addition to the first characteristic information, activity diary information is acquired that is created by the first user and indicates activities for each time period related to the first user; generating first time-slot state information indicating a user state of the first user for each time slot from the first characteristic information; generating first activity information for each time slot indicating the activity of the first user for each time slot by associating the user status indicated in the first status information for each time slot with the activity indicated in the activity diary information; generating an activity function that characterizes each of the activities indicated in the activity diary information based on the first time-zone activity information, and saving the generated activity function as the activity prediction model; An activity management system characterized by:

9. a memory for storing processing instructions; a processor, The processing instructions stored in the memory include: acquiring first characteristic information for a first user, the first characteristic information indicating acceleration information, gyro information, heart rate information, and location information of the first user over time; acquiring activity diary information created by the first user and indicating activities by time period related to the first user; calculating a first threshold for distinguishing between a plurality of user states associated with the first user based on the first characteristic information; generating first time-zone state information indicating a user state of the first user for each time zone from the first characteristic information based on the calculated first threshold; generating first time-slot activity information indicating the activity of the first user for each time slot by associating the user status indicated in the first time-slot status information with the activity indicated in the activity diary information; generating an activity function that characterizes each of the activities indicated in the activity diary information based on the first time-zone activity information, and storing the generated activity function as an activity prediction model; acquiring second characteristic information for a second user, the second characteristic information indicating acceleration information, gyro information, heart rate information, and location information of the second user over time; calculating a second threshold for distinguishing between a plurality of user states associated with the second user based on the second characteristic information; generating second time-zone state information indicating a user state of the second user for each time zone from the second characteristic information based on the calculated second threshold; generating an activity record indicating potential activities for each time period associated with the second user based on the second time period state information and the activity function of the activity prediction model; presenting the activity log to the second user; generating a final activity record that establishes time-slot-specific activity associated with the second user based on the revision input received from the second user; and training a second activity prediction model based on the final activity record.

Images