Care assistance system, device, method, and program

Abstract

Provided is a care assistance system provided with: a sensor information acquisition unit that acquires time-series sensor information detected by one or more sensors provided in a living space of a care subject; a feature quantity generation unit that generates a feature quantity on the basis of the time-series sensor information; a factor analysis unit that performs factor analysis on the basis of the feature amount using a predetermined factor; a change over time determination unit that determines a change over time of the factor on the basis of the factor analysis result; and a prediction information generation unit that generates prediction information relating to the state transition of the object to be cared on the basis of the change over time.

Description

Technical Field

[0001] This invention relates to systems, and more particularly to systems for assisting in the care of the elderly and others. Background Technology

[0002] With the advent of an aging society, the demand for systems that assist in the care of the elderly and those being cared for from remote locations is increasing, and various care assistance systems have been proposed in the past. For example, Patent Document 1 discloses a system that monitors the living conditions of the person being cared for by means of detection values ​​from sensors installed in the living space.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2016-173732 Summary of the Invention

[0006] However, in many previous systems (e.g., Patent Document 1), the current state of the person being cared for is determined by a sensor, and a prescribed notification is given to the caregiver when the state meets the prescribed conditions.

[0007] However, in such a structure, because it is impossible to predict what will happen, caregivers need to constantly maintain the status of the care support system, resulting in a heavy caregiving burden.

[0008] The present invention was made in view of the above-mentioned technical background, and its object is to provide a system that can reduce the caregiving burden of caregivers.

[0009] The aforementioned technical challenges can be addressed through caregiving assistance systems with the following structures.

[0010] That is, the care assistance system of the present invention comprises: a sensor information acquisition unit for acquiring time-series sensor information detected by one or more sensors installed in the living space of the caregiver; a feature quantity generation unit for generating feature quantities based on the time-series sensor information; a factor analysis unit for performing factor analysis based on the feature quantities using predetermined factors; a time-varying change determination unit for determining the time-varying change of the factors based on the factor analysis results; and a prediction information generation unit for generating prediction information related to the state transition of the caregiver based on the time-varying change.

[0011] Based on this structure, predictive information related to the state transitions of the caregiver can be generated by considering changes in factors within a background feature quantity. This feature quantity is derived from time-series sensor information detected by sensors placed in the caregiver's living space. Caregivers can then make predictions about the caregiver's state based on this predictive information, thus reducing their caregiving burden.

[0012] According to the present invention, the caregiving burden on caregivers can be reduced. Attached Figure Description

[0013] Figure 1 This is a diagram of the overall structure of the care support system.

[0014] Figure 2 This is a hardware structure diagram of an information processing device.

[0015] Figure 3 This is a hardware structure diagram of the server device.

[0016] Figure 4 This is an illustrative diagram showing an example of installing a sensor device in a refrigerator located in a kitchen.

[0017] Figure 5 This is an illustrative diagram showing an example of installing a sensor device in a bedroom.

[0018] Figure 6 This is an illustrative diagram showing an example of installing a sensor device on a corridor handrail.

[0019] Figure 7 This is an explanatory diagram showing an example of installing a sensor device on an armchair placed in a Western-style room.

[0020] Figure 8 These are illustrative diagrams showing other examples of installing sensor devices in a bedroom.

[0021] Figure 9 This is an illustrative diagram showing an example of a movable door in a room with a sensor device installed.

[0022] Figure 10 This is a functional block diagram of the server device that performs the learning process.

[0023] Figure 11 This is a functional block diagram of the server device used for assisting caregiving actions.

[0024] Figure 12 This is an example of a screen displayed on an information processing device during the process of associating sensor devices.

[0025] Figure 13 It is an explanatory diagram showing the relationship between the room and the state of the person being cared for.

[0026] Figure 14 It is a typical flowchart related to learning and processing.

[0027] Figure 15 This is a detailed flowchart of the preprocessing process (Part 1).

[0028] Figure 16 It is a typical flowchart related to caregiving procedures.

[0029] Figure 17 This is a detailed flowchart of the preprocessing (Part 2).

[0030] Figure 18 This is an explanatory diagram showing an example of activity level.

[0031] Figure 19 This is an explanatory diagram showing other examples related to activity levels.

[0032] Figure 20 This is a detailed flowchart of factor analysis.

[0033] Figure 21 This is an explanatory diagram related to combinations of factor transfers.

[0034] (Explanation of reference numerals in the attached image)

[0035] 10: Sensor device; 20: Information processing device; 30: Server device; 100: Care assistance system. Detailed Implementation

[0036] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0037] (1. First embodiment)

[0038] As a first embodiment, an example of applying the present invention to a care assistance system 100 for the elderly, those being cared for, etc., will be described. Furthermore, in this embodiment, for convenience, the person providing care using the care assistance system 100 will be referred to as the caregiver, and the person being cared for by the care assistance system 100 will be referred to as the care recipient.

[0039] (1.1 Structure of the caregiving assistance system)

[0040] Figure 1This is an overall structural diagram of the care assistance system 100 according to this embodiment. As shown in the diagram, the care assistance system 100 includes multiple sensor devices 10 (sensor device 1 (10-1), sensor device 2 (10-2), ..., sensor device N (10-N)), an information processing device 20, and a server device 30. These devices are interconnected via a network such as the Internet. Furthermore, the structure of this network is illustrative. Additionally, the server device 30 can be implemented in the cloud.

[0041] As described below, the sensor device 10 is configured corresponding to each room in the living space of the person being cared for. The sensor device 10 includes sensors for sensing the person being cared for, a storage unit for programs such as ROM and RAM, a storage unit for various data, a control unit such as a CPU, and a communication unit including a communication unit for transmitting and receiving information with external devices. Information detected by the sensors in the sensor device 10 is transmitted to other devices, such as a server device 30, via a network.

[0042] Figure 2 This is a hardware structure diagram of the information processing device 20 according to this embodiment. As can be seen from the diagram, the information processing device 20 includes a storage unit 21, a control unit 22, a communication unit 23, an operation input unit 25, an audio output unit 26, a display output unit 27, and an I / O unit 28, which are connected to each other via a bus or the like.

[0043] Storage unit 21 is a storage unit such as ROM / RAM, hard disk, flash memory, etc., storing various programs or data that perform the actions described later. Control unit 22 is a control device such as a CPU, executing programs to achieve the various actions described later. Communication unit 23 is a communication unit for sending and receiving information with external devices via a network. Operation input unit 25 provides input information detected by input devices such as a mouse, keyboard, and touch panel to control unit 22. Audio output unit 26 outputs sound to connected output devices such as speakers under the control of control unit 22. Display output unit 27 outputs image information to display output devices such as displays under the control of control unit 22. I / O unit 28 performs input / output processing with external devices.

[0044] The information processing device 20 transmits and receives information with the sensor device 20 or the server device 30. By operating the information processing device 20, caregivers and system users can perform various settings processes described later. In addition, the information processing device 20 can provide caregivers with various information prompts described later.

[0045] Figure 3This is a hardware structure diagram of the server device 30 according to this embodiment. As can be seen from the diagram, the server device 30 includes a storage unit 31, a control unit 32, a communication unit 33, an operation input unit 35, an audio output unit 36, a display output unit 37, and an I / O unit 38, which are connected to each other via a bus or the like.

[0046] Storage unit 31 is a storage unit such as ROM / RAM, hard disk, flash memory, etc., storing various programs or data that execute the actions described later. Control unit 32 is a control device such as CPU or GPU, executing programs to achieve the various actions described later. Communication unit 33 is a communication unit for sending and receiving information with external devices via a network. Operation input unit 35 provides input information detected by input devices such as mouse, keyboard, and touch panel to control unit 32. Audio output unit 36 ​​outputs sound to connected output devices such as speakers under the control of control unit 32. Display output unit 37 outputs image information to display output devices such as displays under the control of control unit 32. I / O unit 38 performs input / output processing with external devices.

[0047] The server device 30 transmits and receives information with the sensor device 10 or the information processing device 20, and performs various processes described later.

[0048] The sensor device 10 is installed in various locations within the living space of the person being cared for. For example, the floor, walls, handrails, appliances, or furniture in the room.

[0049] Figure 4 This is an explanatory diagram showing an example of installing the sensor device 10 in a refrigerator 11 located in a kitchen. As can be seen from the diagram, the refrigerator 11 has a two-layer structure, with two double doors 111 on the upper layer and a drawer-type storage compartment 113 on the lower layer. Figure (A) shows the overall structure of the refrigerator when the doors are closed. Figure (B) is an explanatory diagram of the opening and closing of the upper double doors 111. Figure (C) is an explanatory diagram of the opening and closing of the lower storage compartment 113.

[0050] As can be seen from Figure (A), each door of the double door 111 is equipped with an acceleration sensor 110 as a sensor device 10. In addition, the lower storage compartment 113 is also equipped with an acceleration sensor (not shown) as a sensor device 10.

[0051] As shown in Figure (B), each door of the double door 111 is equipped with an accelerometer 110. The opening and closing of the door can be detected based on the time-series signal obtained from the accelerometer 110. Furthermore, if a container such as a beverage is stored inside the door, the door becomes heavier. As shown in the left and right diagrams of Figure (B), this change in weight can be detected as a change in the detection value in the accelerometer 110; therefore, the presence or absence of a container located inside the door and its contents can also be indirectly detected.

[0052] As shown in Figure (C), the storage container 113 is equipped with an accelerometer (not shown). Based on the time-series signal obtained from this accelerometer, the opening and closing of the storage container 113 can be detected. Furthermore, if containers such as beverages are stored in the storage container 113, the weight of the storage container 113 increases. As shown in the left and right diagrams of Figure (C), this change in weight can be detected as a change in the detection value of the accelerometer, thus indirectly detecting the presence or absence of containers and the amount of contents within the storage container 113.

[0053] Figure 5 Figure (A) is an explanatory diagram showing an example of installing the sensor device 10 in a bedroom. Figure (B) is an explanatory diagram showing an example of the configuration structure, and Figure (A) is an explanatory diagram regarding an example of sound collection. As can be seen from Figure (A), around the bed 122 in the bedroom, a pair of microphones, namely microphone 121(L) and microphone 121(R), are arranged as sensors for the sensor device 10. These microphones collect sounds generated by the person being cared for while lying on the bed 122 in a time sequence. For example, as shown in Figure (B), the breathing sounds and coughs of the person being cared for can be detected based on the sound pressure waveform obtained through the sound collection. In addition, various known signal processing methods, such as sound collection based on the target area (around the bed 122) and noise removal other than the target sound, can be performed during detection.

[0054] Figure 6 This is an explanatory diagram showing an example of installing the sensor device 10 on a corridor handrail. As shown in Figure (A), the sensor device 10, mounted on the rod-shaped handrail 131, includes a deformation sensor 132. Loads, torques, etc., applied to the handrail can be detected based on the time-series signal obtained from this deformation sensor 132. By detecting these, the state of an object can be detected, such as whether there is any inconvenience to the hand, waist, or legs.

[0055] Alternatively, deformation sensors can be installed on multiple handrails. In Figure (B), three handrails 131 (131-1, 131-2, 131-3) are arranged consecutively, and deformation sensors 132 (132-1, 132-2, 132-3) are installed on each handrail 131. Based on this structure with multiple sensors, not only can load and torque be detected by each deformation sensor 132, but the movement speed of the person being cared for while moving by gripping the handrail can also be calculated by comprehensively analyzing the time series signals obtained from each deformation sensor 132.

[0056] Figure 7 This is an explanatory diagram showing an example of installing the sensor device 10 in an armchair placed in a Western-style room. As shown in Figure (A), a pair of deformation sensors 151 (151(L), 151(R)) are installed on the left and right armrests of the armchair 15 as sensors of the sensor device 10. Various information can be detected based on time-series signals such as load, vibration, and deformation detected by the pair of deformation sensors 151.

[0057] Figure (B) shows an example of a signal detected by a pair of deformation sensors 151. Based on such signals, the dominant hand, leg strength, posture, breathing, etc. of the caregiver can be inferred from the detected load magnitude, timing, left-right difference, etc.

[0058] Figure (C) shows another example of the signal detected by a pair of deformation sensors 151. The state of the person being cared for, such as their breathing, can be inferred from such periodic signals.

[0059] Figure 8 This is an explanatory diagram showing another example of installing the sensor device 10 in a bedroom. In this example, deformation sensors 163, which are sensors of the sensor device 10, are respectively arranged at the bottom of the four legs of the bed 162. For example, the state of the person being cared for on the bed 162 can be estimated based on the time-series signals obtained from these deformation sensors 163. In addition, a distance sensor 161, which is a sensor of the sensor device 10, is arranged at a predetermined distance from the bed 162. For example, the movement of the person being cared for can be detected based on the time-series signals obtained from these distance sensors 161.

[0060] Figure 9This is an explanatory diagram showing an example of a movable door in a room with the sensor device 10 installed. As can be seen from the diagram, the door 171 is configured to open and close by rotating the handle and pushing or pulling. Accelerometers 172 and 173 are installed on the door 171 and the handle. The presence or absence of door opening and closing, the opening and closing speed, etc., can be detected based on the time-series signal obtained from the accelerometer 172. Furthermore, the movement of the room, the frequency of actions performed in the room, etc., can also be calculated using these parameters.

[0061] The above description illustrates an example of sensor installation in sensor device 10, but these are merely illustrative examples, and the present invention is not limited to this structure. Therefore, any sensor can be used to detect the living condition of the person being cared for, and the installation location can be modified in various ways.

[0062] Figure 10 This is a functional block diagram of the server device 30 when performing the learning process described later. As can be seen from the diagram, the server device 30 has multiple learning sensor data acquisition units 301, which read and acquire time-series sensor data detected by the sensor device 10 from the storage unit 31 of the server device 30.

[0063] Depending on the type of sensor, the time-series sensor data acquired by the learning sensor data acquisition unit 301 is either transformed into the frequency domain or directly supplied to the feature quantity generation processing unit 305 as time-domain data. That is, a portion of the time-series data acquired by the learning sensor data acquisition unit 301 is provided to the spectrum transformation unit 302 and transformed into the frequency domain, while other time-series data is directly supplied to the feature quantity generation processing unit 305 as a time-domain signal.

[0064] When time-series sensor data is input to the spectrum conversion unit 302, the time-series sensor data is converted into a spectrum diagram in a predetermined unit and output to the preprocessing unit 303. The preprocessing unit 303 performs predetermined preprocessing on the frequency domain signal provided from the spectrum conversion unit 302, and provides the preprocessed signal to the feature quantity generation processing unit 305 for each sensor.

[0065] The feature generation processing unit 305 performs optimal feature generation processing for each sensor based on either the frequency domain signal or the time domain signal. In this embodiment, features for the frequency domain signal can include, for example, MFCC features including Mel-frequency cepstral coefficients, or the output of a predetermined learned model generated through deep learning. Similarly, features for the time domain can include, for example, transition characteristics (including their slope), or the output of a predetermined learned model generated through deep learning. Furthermore, a spectrum can be directly used as the frequency domain signal.

[0066] The output of the feature generation processing unit 305, i.e., the generated features, is provided to the normalization processing unit 306. The normalization processing unit 306 normalizes each feature by scoring it, etc. As a result, each feature can be processed in the same feature space.

[0067] The feature generation processing unit 305 provides the feature values ​​to the learning processing unit 307. Meanwhile, the learning parameter acquisition unit 308 provides the learning processing unit 307 with various parameters required for learning.

[0068] The learning processing unit 307 performs learning processing based on the provided features and parameters. This learning processing is, for example, the training of a clustering model. In this embodiment, DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is used as a clustering algorithm, for example, but other clustering algorithms can also be used. Additionally, other classification algorithms can also be used.

[0069] The learned model generated by the learning processing unit 307 is provided to the definition information providing unit 309. The definition information providing unit 309 performs defined processing on the output according to a prescribed algorithm or user-defined input. This definition processing defines which sensor activity corresponds to the output node of the clustering model based on the output action.

[0070] The storage processing unit 310 performs the process of storing the learned model generated by the learning processing unit 307 and the definition information generated by the definition information providing unit 309 into the storage unit 31.

[0071] Figure 11 This is a functional block diagram of the server device 30 during caregiving assistance actions. As can be seen from this diagram, the structure from the sensor data acquisition unit 321 to the normalization processing unit 326 (sensor data acquisition unit 321, spectrum transformation unit 322, preprocessing unit 323, feature generation processing unit 325, normalization processing unit 326) is largely the same as the structure involved in learning processing (learning sensor data acquisition unit 301, spectrum transformation unit 302, preprocessing unit 303, feature generation processing unit 305, normalization processing unit 306), therefore, detailed descriptions are omitted. Furthermore, the difference lies in that the data processed during the learning phase is learning sensor data, whereas in the example shown in this diagram, it is sensor data newly acquired by the sensor device 10 and provided to the server device 30.

[0072] In the normalization processing unit 326, the normalized feature values ​​are provided to the activity level generation unit 327. Based on the provided feature values, the activity level generation unit 327 calculates the activity level (activity) of each sensor and provides it to the output processing unit 335. The output processing unit 335 processes the activity level to prompt the caregiver. Alternatively, the activity level can be calculated based on the results of clustering processing.

[0073] Furthermore, the normalized feature values ​​in the normalization processing unit 326 are also provided to the clustering processing unit 328. The clustering processing unit 328 performs clustering processing on the normalized feature values ​​input to the learned model generated through the learning process. The clustering processing result is provided to the output processing unit 335. The output processing unit 335 processes the clustering processing result for use in providing caregiver prompts.

[0074] The clustering results generated in the clustering processing unit 328 are provided to the factor analysis processing unit 329. Based on the clustering results, the factor analysis processing unit 329 performs factor analysis on the specified factors and calculates factor loadings, factor scores, and the average of the factor scores as the factor analysis results.

[0075] Based on the results of factor analysis, the state estimation unit 331 estimates the state of the person under care and outputs the estimation results to the output processing unit 335. The output processing unit 335 processes the information provided by the caregiver's prompts.

[0076] Furthermore, the state transition estimation unit 332 processes the estimation of state transitions of the caregiver based on the results of factor analysis and provides the results to the anomaly detection unit 333. The anomaly detection unit 333 performs anomaly detection according to the degree of state transition and provides the results to the output processing unit 335. The output processing unit 335 processes information related to state transitions and anomalies to provide caregivers with relevant information.

[0077] (1.2 Actions of the care assistance system)

[0078] Next, the operation of the care assistance system 100 will be explained. In order for the care assistance system 100 to function properly, the caregiver or user of the care assistance system 100 performs a process of pre-installing sensor devices 10 in the living space and mapping the sensor devices 10 to the layout of the living space on the care assistance system 100. Additionally, the caregiver performs pre-learning processing based on prescribed learning data.

[0079] (1.2.1 Setting and pre-configuring sensors for living spaces)

[0080] First, caregivers and other stakeholders install various sensor devices 10 in various ways within the living space of the person being cared for (see reference). Figures 4-9 Therefore, the actions of the person being cared for can be obtained through the sensors of the sensor device 10.

[0081] Afterwards, the user sets up the room and layout of the living space via the information processing device 20, and then processes the mapping of each installed sensor device 10 to the room and specific location of the living space. The processing result is stored in the server device 30.

[0082] Figure 12 This is an example of a screen displayed on the information processing device 20 during the process of associating sensor device 10 with other sensors. As shown in the figure, when sensor device 10 is installed on the entryway door, the user selects "Entryway" on the screen to associate the recognition information of sensor device 10 installed on the entryway door. Similarly, when sensor device 10 is installed on a vanity, the user selects "Vanity" on the screen to associate the recognition information of sensor device 10 installed on the vanity. Furthermore, when sensor device 10 is installed on a balcony, the user selects "Balcony" on the screen to associate the recognition information of sensor device 10 installed on the balcony.

[0083] Each room is assigned a label related to the status of the person being cared for. In this embodiment, the status labels are associated with three statuses: cleanliness, exercise, and diet. In addition to these statuses, statuses related to sleep can also be considered.

[0084] Figure 13 This diagram illustrates the relationship between the rooms and the state of the person being cared for. As shown, the "Entrance Hall," "Living Room," "Japanese-style Room," and "Corridor" are spaces primarily for "going out," "relaxation," "sleep," and "movement," respectively—all related to human movement; therefore, these rooms are labeled to indicate a state of movement. Similarly, the "Dining Room," "Kitchen," and "Bathroom" are spaces primarily for "eating," "cooking," and "excretion," respectively—all related to eating; therefore, these rooms are labeled to indicate a state of eating. Furthermore, the "Vanity Area," "Bathroom," "Western-style Room," and "Balcony" are spaces for "grooming," "cleaning," "clothing," and "washing / cultivation," respectively—all related to human cleanliness; therefore, these rooms are labeled to indicate a state of cleanliness.

[0085] In this way, by assigning tags related to the state of the person being cared for to each room, the state of the person being cared for can be inferred from the actions of the sensors deployed in each room via these tags. Furthermore, details regarding the prompting process will be described later.

[0086] (1.2.2 Learning Processing)

[0087] Next, the pre-learning process of the clustering model used in the care assistance system 100 will be explained.

[0088] Figure 14 This is a general flowchart related to the learning process performed in the server device 30. As shown in the diagram, when the process begins, the learning sensor data acquisition unit 301 performs a process (S11) to read and acquire time-series sensor data for learning from the storage unit 31. The time-series sensor data for learning can be past data acquired from each sensor.

[0089] After acquisition and processing, the spectrum transformation unit 302 transforms a portion of the learning sensor data into a spectrum diagram (S12). As a result, a portion of the time-series sensor data is transformed into the frequency domain.

[0090] After the spectrum transformation process, the preprocessing unit 303 performs a prescribed preprocessing on the signal that has been transformed into a spectrum diagram (S13).

[0091] Figure 15 This is a detailed flowchart of the preprocessing (S1). As shown in the flowchart, when processing begins, the preprocessing unit 303 performs a process to limit the frequency band of the spectrum within a specified range (S131). Then, the preprocessing unit 303 performs a process to normalize the spectrum of the specified frequency band (S132). The preprocessing unit 303 then performs a process to mask the specified frequencies and powers of the normalized spectrum (S133, S135). After this, the preprocessing ends.

[0092] By performing such preprocessing, only frequency domain signals within an assumed range can be supplied to the feature generation process described later.

[0093] return Figure 14 After preprocessing, the feature generation processing unit 305 performs processing based on the preprocessed frequency domain signal and time domain sensor data to generate optimal feature quantities for each sensor (S15). Various objects can be used as feature quantities. For example, if it is an audio signal acquired from a microphone, MFCC feature quantities including Mel-frequency cepstral coefficients can be used as feature quantities. Alternatively, the prediction results of a learned model generated through deep learning can also be used as feature quantities. Furthermore, if it is a time domain signal, the slope of the transition characteristics can be used as feature quantities.

[0094] After the feature quantity generation process, the normalization processing unit 306 performs normalization processing such as scoring each feature quantity (S16). As a result, each feature quantity can be processed in the same feature quantity space.

[0095] After the feature normalization process, the learning parameter acquisition unit 308 performs the process of acquiring the learning parameters from the storage unit 31 (S17).

[0096] After the parameter is obtained, the learning processing unit 307 performs learning processing based on the normalized feature quantity and parameter, and trains the specified model using a specified algorithm (S18).

[0097] This learning process can be, for example, the process of training a clustering model. In this embodiment, DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is used as the clustering algorithm, but other clustering algorithms can also be used. Additionally, other classification algorithms can also be used. Furthermore, multiple learning processes can be performed by changing initial parameters, etc., to obtain the most suitable learned model as the final learned model structure.

[0098] After the learning process, the definition information providing unit 309 performs a process of providing a prescribed definition to the learned model (S19). The definition information providing unit 309 can provide definition information based on input from the setter, or it can automatically provide definition information based on prescribed conditions. In this embodiment, the definition information is information that defines which sensor activity corresponds to the output node of the clustering model.

[0099] After the definition process, the storage processing unit 310 performs the process of storing the learned model to the storage unit 31 (S21), and the learning process ends.

[0100] Furthermore, if multiple datasets are available for learning, the above learning process can be repeated.

[0101] (1.2.3 Actions of the care system)

[0102] Next, the operation of the caregiver assistance system 100 will be explained.

[0103] Figure 16 This is a general flowchart relating to the operation of the server device 30 during caregiving. As can be seen from the diagram, when the process begins, the sensor data acquisition unit 321 performs a process (S31) to acquire sensor data detected in the sensor device 10 and store it in the storage unit 31.

[0104] After the sensor data is acquired and processed, the spectrum transformation unit 322 transforms a portion of the sensor data into a spectrum (S32). As a result, a portion of the time-series sensor data is transformed into the frequency domain.

[0105] Based on this structure, sensor detection information can be transformed into time-domain or frequency-domain feature quantities, enabling multi-faceted analysis.

[0106] After the spectrum transformation process, the preprocessing unit 323 performs a prescribed preprocessing on the signal that has been transformed into a spectrum diagram (S33).

[0107] Figure 17 This is a detailed flowchart of the preprocessing (S2). As shown in the diagram, when processing begins, the preprocessing unit 323 performs a process to limit the frequency band of the spectrum within a specified range (S331). Then, the preprocessing unit 323 performs a process to normalize the spectrum of the specified frequency band (S332). The preprocessing unit 323 then performs a process to mask the specified frequencies and powers of the normalized spectrum (S333, S335). Afterwards, the preprocessing ends.

[0108] By performing such preprocessing, only frequency domain signals within an assumed range can be supplied to the feature generation process described later.

[0109] return Figure 16 After preprocessing, the feature generation processing unit 325 performs processing based on the preprocessed frequency domain signal and time domain sensor data to generate optimal feature quantities for each sensor (S35). Various objects can be used as feature quantities. For example, if it is an audio signal acquired from a microphone, MFCC feature quantities including Mel-frequency cepstral coefficients can be used as feature quantities. Alternatively, the prediction results of a learned model generated through deep learning can also be used as feature quantities. Furthermore, if it is a time domain signal, the slope of the transition characteristics can be used as feature quantities.

[0110] After the feature quantity generation process, the normalization processing unit 326 performs normalization processing such as scoring each feature quantity (S36). As a result, each feature quantity can be processed in the same feature quantity space.

[0111] After the feature quantities are normalized, the normalized feature quantities are supplied to the activity quantity generation unit 327 and the clustering processing unit 328.

[0112] The activity quantity generation unit 327 performs processing (S37) to generate the activity quantity of each sensor device 10 based on the normalized feature quantity. The activity quantity (or activity) is an indicator that represents the frequency at which information is detected in each sensor. In this embodiment, for example, it is the frequency of occurrence of the normalized feature quantity per unit time.

[0113] After the activity level generation process, the output processing unit 335 performs the output processing of the activity levels related to each sensor (S38). This output processing involves processing caregiver prompts, such as sending prompts to a display connected to the information processing device 20. After this processing, the process ends.

[0114] Figure 18 These are explanatory diagrams showing examples of activity levels on a display. Diagram (A) is a basic example, and diagram (B) is an example based on additional functions.

[0115] In Figure (A), as an example, the magnitude of sensor activity in each room or sensor device 10 on April 1, 2022, is represented by the diameter of the circles in which each room or sensor device 10 is configured. As shown in the figure, the image displayed on the screen of the information processing device 20 includes a top view (or layout) of the living space of the person being cared for, where the sensor devices 10 are configured. As described above, each sensor device 10 is associated with each room through pre-setting. Therefore, by representing the amount of sensor activity in each room with circles, the state of the person being cared for can be monitored.

[0116] Based on this structure, the state of the person being cared for can be estimated by indicating the amount of sensor activity in accordance with the layout of the living space.

[0117] Furthermore, the methods for indicating activity levels are not limited to this. Therefore, various methods can be used to provide indications. For example, it is also possible to indicate the difference between sensor activity levels on a certain date and those on other dates, or to display the relative relationship between sensor activity levels and other time intervals.

[0118] Figure (B) is an explanatory diagram illustrating an example of the relative relationships of sensor activity levels. In this example, the circles representing sensor activity levels on April 1, 2022, and the circle representing sensor activity levels on April 10, 2022, are depicted overlapping on a top view.

[0119] Based on this structure, changes in activity levels can be immediately captured by the difference in the diameter of the circles in each room, thereby enabling a more reliable understanding of the caregiver's condition.

[0120] Furthermore, activity levels do not necessarily need to be displayed along with the room layout. For example, they can be displayed in various other ways.

[0121] Figure 19These are explanatory diagrams illustrating other display examples related to sensor activity levels. Figure (A) is an example of displaying sensor activity levels in each sensor device 10 using a colored 3D bar chart; Figure (B) is an example of displaying sensor activity levels in each sensor device 10 using a colored pie chart; and Figure (C) is an example of displaying sensor activity levels in each sensor device 10 using the size, position, and color of a fish-shaped illustration. Based on Figure (A), the sensor activity levels in each room or sensor device 10 can be understood in three dimensions. Based on Figure (B), the relative relationships of sensor activity levels in each room or sensor device 10 can be understood. Based on Figure (C), the sensor activity levels in each room or sensor device 10 can be understood in an entertaining way.

[0122] return Figure 16 The clustering processing unit 328 uses the learned model generated through the learning process to cluster the normalized feature quantities (S39). In this embodiment, the clustering algorithm is DBSCAN. Furthermore, as described above, the clustering model is defined. Therefore, it is possible to reliably determine which sensor device 10 detected the information based on the clustering processing results.

[0123] Based on this structure, clustering can reduce the impact of noise caused by sensors, the environment, etc.

[0124] After clustering, the output processing unit 335 performs processing to output the clustering results (S40). This output processing involves processing caregiver prompts, such as sending prompts to a display connected to the information processing device 20. After this processing, the process ends.

[0125] In addition, activity generation processing can be performed on the clustering results. That is, the frequency of occurrence per unit time can be calculated for each output that is equivalent to the clustering results.

[0126] After returning to the clustering process, the factor analysis processing unit 329 performs factor analysis on the clustering results (S41).

[0127] Figure 20 This is a detailed flowchart of factor analysis processing. As shown in the diagram, when processing begins, the factor analysis processing unit 329 performs preprocessing (S411) on the data used for factor analysis, i.e., the clustering results of the normalized feature quantities. In this embodiment, preprocessing is the process of classifying the clustering results, such as the output of each output node, into multiple stages from the perspective of detection frequency per unit time and treating them as discrete values. The multiple stages can be, for example, seven stages.

[0128] After preprocessing, the factor analysis processing unit 329 processes the preprocessed clustering results to generate factor loadings for each of the specified factors (S412). In this embodiment, the factors include factors related to the mobility of the caregiver, namely, the exercise factor; factors related to the diet of the caregiver, namely, the diet factor; and factors related to the cleanliness of the caregiver, namely, the cleanliness factor.

[0129] Based on this structure, the condition of the person being cared for can be assessed from the perspectives of cleanliness, exercise, and diet. Therefore, it is possible to appropriately assess the condition of the elderly and other vulnerable individuals.

[0130] In addition, the factors are not limited to these three; they can also include sleep-related factors, namely sleep factors.

[0131] After generating factor loadings, the factor analysis processing unit 329 performs factor score generation processing based on the factor loadings (S413). Furthermore, after generating the factor scores, the factor analysis processing unit 329 calculates the average of the factor scores over a predetermined time interval (S415). The predetermined time interval is, for example, one day. After this average value generation processing, the factor analysis process ends.

[0132] return Figure 16 After factor analysis, the state estimation unit 331 performs a process to estimate the state of the care recipient based on the factor analysis results. In this embodiment, the health status (Ha) is generated using the average factor score of each generated factor and the following evaluation formula, thereby performing a state estimation. Furthermore, in this embodiment, the estimated terms can be replaced with other terms such as predictions.

[0133] [Mathematical Expression 1]

[0134] Ha = α (cleanliness) * β (exercise) * γ (diet) + C

[0135] Where α, β, and γ are defined coefficients, (cleanliness) represents the average factor score of the cleanliness factor, (exercise) represents the average factor score of the exercise factor, and (diet) represents the average factor score of the diet factor. C is a defined constant.

[0136] Furthermore, the evaluation formula is not limited to this form. Therefore, for example, the natural logarithm e can be used as follows.

[0137] [Mathematical Expression 2]

[0138] Ha=αe （清洁） *βe （运动） *γe （饮食） +C

[0139] Furthermore, when there are four factors, the health status can also be generated in the same way. That is, when there are four factors, including the sleep factor, the health status (H) can be assessed according to the following evaluation formula.

[0140] [Mathematical Expression 3]

[0141] Ha = α (cleanliness) * β (exercise) * γ (diet) * δ (sleep) + C

[0142] Where δ is a specified coefficient, and (sleep) is the average factor score of the sleep factor.

[0143] Furthermore, in this case, the natural logarithm e can also be used as follows.

[0144] [Mathematical Expression 4]

[0145] Ha=αe （清洁） *βe （运动） *γe （饮食） *δe （睡眠） +C

[0146] Based on this structure, health status can be inferred from factors that lie in the background of feature quantities obtained from time-series sensor signals.

[0147] After the status estimation process, the output processing unit 335 performs the processing of outputting the generated health status (S43). This output processing involves processing caregiver prompts, such as sending prompts to a display connected to the information processing device 20. After this processing, the process ends.

[0148] After factor analysis, the state transition estimation unit 332 performs a process to estimate the state transition in the time direction based on the results of factor analysis (S45). More specifically, in this embodiment, the state transition estimation unit 332 determines a tendency based on a predetermined combination of factor transitions, namely, a combination of movement factor and cleanliness factor, a combination of diet factor and movement factor, and a combination of cleanliness factor and diet factor, thereby estimating the state transition. Furthermore, in this embodiment, the combination of movement factor and cleanliness factor is referred to as rapid activity, the combination of diet factor and movement factor is referred to as activity, and the combination of cleanliness factor and diet factor is referred to as cognition.

[0149] Based on this structure, predictive information can be generated through multidimensional assessment. Furthermore, it allows for the assessment of the caregiver's condition from the perspectives of cleanliness, exercise, and diet, enabling appropriate evaluation of the condition of the elderly and other vulnerable groups.

[0150] Figure 21These are explanatory diagrams related to combinations of factor transfer. Diagram (A) is a concept map related to rapid activity, diagram (B) is a concept map related to activity, and diagram (C) is a concept map related to cognition.

[0151] As shown in Figure (A), by evaluating the average factor scores related to motility and cleanliness over time, it is possible to grasp the tendency related to rapid activity. Based on the tendency of this vector, future state transitions can be predicted. For example, if factors related to both motility and cleanliness are judged to be increasing, it is determined that there is a tendency for rapid activity to increase in the future.

[0152] As shown in Figure (B), by assessing the average factor scores related to diet and exercise over time, it is possible to grasp the tendency related to activity levels. Based on the tendency of this vector, future state transitions can be predicted. For example, if factors related to both diet and exercise are judged to be increasing, it is determined that there is a tendency for future activity levels to increase.

[0153] As shown in Figure (C), by assessing the average factor scores related to cleanliness and diet over time, it is possible to grasp the cognitive tendencies. Based on the tendency of this vector, future state transitions can be inferred. For example, if factors related to cleanliness and diet are judged to be improving, it is determined that there is a tendency for future cognitive improvement.

[0154] Furthermore, although this embodiment describes combining two factor transitions to predict state transitions, the present invention is not limited to this structure. Therefore, state transitions can also be predicted based on the transition of one factor, or by combining three or more factors.

[0155] return Figure 16 After the state transition estimation process, the anomaly detection unit 333 performs anomaly detection processing (S46) based on whether the degree of change of each factor exceeds a predetermined threshold. For example, in Figure 21 In this system, anomaly detection can be performed based on whether the magnitude of the vector representing the shift in the average factor score exceeds a predetermined threshold. Furthermore, various methods can be used to determine the criteria for this detection. In short, as long as the degree of change in the shift vector can be captured, anomaly detection can be performed based on the vector's difference, slope, etc., or by comprehensively judging these features. Additionally, anomalies can be identified if any one of the three factors (rapid activity, activity, and cognition) is detected, or if two or more of the three factors are found to be abnormal.

[0156] After the anomaly detection process, the output processing unit 335 performs an estimate of the output state transition and a process to determine the presence or absence of an anomaly (S47). The output processing involves processing caregiver prompts, such as sending prompts to the display connected to the information processing device 20. After this processing, the process ends.

[0157] Based on the above structure, predictive information related to the state transitions of the caregiver can be generated by considering changes in factors in the background of a feature quantity, which is obtained from time-series sensor information detected by sensors installed in the caregiver's living space. Caregivers can then make certain predictions about the caregiver's state based on this predictive information, thus reducing the caregiver's workload.

[0158] (2. Variation)

[0159] This invention can be implemented in various variations.

[0160] In the above embodiments, a structure for providing caregivers with predictive information about the state or state transitions of the person being cared for has been described, but the present invention is not limited to such a structure. Therefore, the care assistance system 100 may also include a prescribed prompting component installed in the living space of the person being cared for, through which various prompts are given to the person being cared for, such as prompts to give attention related to their own state or prompts to promote improvement of their state.

[0161] More specifically, after the server device 30 generates the aforementioned prediction information of the state or state transition, it can also generate information for giving attention related to the state or for promoting the improvement of the state, and send this information to the prompting component to prompt the object under care.

[0162] A prompting component is a device placed in the living space of the person being cared for and providing prescribed stimulation to their five senses. Examples include speakers (auditory stimulation), displays (visual stimulation), light sources such as LEDs (visual stimulation), and vibration units (tactile stimulation). Furthermore, the prompting component can be integrated with the sensor device 10. Information used to provide attention related to the state includes, for example, sounds (or functional sounds), images (or moving images), light emission patterns, and vibration patterns indicating the current state of the person being cared for. Information that promotes improvement in the state includes, for example, information that promotes the improvement of the state or symptoms presented by the person being cared for; for example, in the case of loss of time perception, sounds (or functional sounds), images (or moving images), light emission patterns, and vibration patterns used to restore time perception or promote time-related attention.

[0163] In the above embodiment, it is explained that various prompts to the caregiver are ultimately made on a display integrated with the information processing device 20, but the present invention is not limited to this structure. Prompts may also be made not through visual means, but through components that stimulate other senses, such as speakers or vibration devices included in the information processing device 20, providing prompts based on sound or vibration.

[0164] In the above embodiments, it is explained that only the caregiver is notified when an anomaly is detected, but it can also be configured to notify external devices such as servers of medical institutions, directly or indirectly.

[0165] The embodiments of the present invention have been described above. However, the above embodiments are merely some examples of applications of the present invention and do not imply that the technical scope of the present invention is limited to the specific structures of the above embodiments. In addition, the above embodiments can be appropriately combined without causing contradictions.

[0166] This invention can be utilized in industries that manufacture and use network systems.

Claims

1. A caregiver assistance system, comprising: The sensor information acquisition unit acquires time-series sensor information detected by one or more sensors installed in the living space of the person being cared for; The feature generation unit generates feature quantities based on the time-series sensor information; The factor analysis department performs factor analysis based on the aforementioned feature quantities using prescribed factors. The time-varying variation determination unit determines the time-varying variation of the factors based on the factor analysis results; as well as The prediction information generation unit generates prediction information related to the state transition of the care object based on the time-related changes.

2. The caregiving assistance system according to claim 1, wherein, The features include time-domain features and frequency-domain features.

3. The caregiving assistance system according to claim 1, wherein, The factor analysis unit performs clustering on the normalized feature quantities and then performs factor analysis on the clustering results.

4. The caregiving assistance system according to claim 1, wherein, The factors include factors related to the cleanliness of the person being cared for, namely the cleanliness factor; factors related to the mobility of the person being cared for, namely the mobility factor; and factors related to the diet of the person being cared for, namely the diet factor.

5. The caregiving assistance system according to claim 4, wherein, The factors also include factors related to the sleep of the person being cared for, namely sleep factors.

6. The caregiving assistance system according to claim 1, wherein, The factor analysis includes the process of calculating the average of the factor scores over time after the factor scores have been calculated.

7. The caregiving assistance system according to claim 6, wherein, The time variation determination unit determines the time variation of the average value.

8. The caregiving assistance system according to claim 1, wherein, The prediction information generation unit generates the prediction information based on a combination of the time-varying changes of multiple factors.

9. The caregiving assistance system according to claim 8, wherein, The combination includes a combination of factors related to the cleanliness of the caregiver, namely a cleanliness factor, and factors related to the mobility of the caregiver, namely a movement factor; a combination of the movement factor and factors related to the diet of the caregiver, namely a diet factor; and a combination of the diet factor and the cleanliness factor.

10. The care assistance system according to claim 1 further comprises an anomaly detection unit, which performs anomaly detection based on the time-varying changes of the factor.

11. The care assistance system according to claim 1 further comprises a state estimation unit, which estimates the health status of the care recipient based on each of the factors.

12. The care assistance system according to claim 1 further comprises a sensor information prompting unit, which calculates the information detection frequency of each of the sensors based on the feature quantity, and prompts the information detection frequency in accordance with the layout of the living space corresponding to each of the sensors.

13. A caregiver assistive device, comprising: The sensor information acquisition unit acquires time-series sensor information detected by one or more sensors installed in the living space of the person being cared for; The feature generation unit generates feature quantities based on the time-series sensor information; The factor analysis department performs factor analysis based on the aforementioned feature quantities using prescribed factors. The time-varying variation determination unit determines the time-varying variation of the factors based on the factor analysis results; as well as The prediction information generation unit generates prediction information related to the state transition of the care object based on the time-related changes.

14. A caregiving assistance method, comprising: The sensor information acquisition step involves acquiring time-series sensor information detected by one or more sensors installed in the living space of the person being cared for; The feature generation step generates feature quantities based on the time-series sensor information. The factor analysis step involves performing factor analysis based on the specified features using prescribed factors. The step of determining the temporal variation is to determine the temporal variation of the factor based on the factor analysis results; as well as The prediction information generation step generates prediction information related to the state transition of the care object based on the time-related changes.

15. A caregiving assistance program, comprising: The sensor information acquisition step involves acquiring time-series sensor information detected by one or more sensors installed in the living space of the person being cared for; The feature generation step generates feature quantities based on the time-series sensor information. The factor analysis step involves performing factor analysis based on the specified features using prescribed factors. The step of determining the temporal variation is to determine the temporal variation of the factor based on the factor analysis results; as well as The prediction information generation step generates prediction information related to the state transition of the care object based on the time-related changes.

Images