State estimation device, program, state estimation system, and state estimation method
The state estimation system addresses inefficiencies in conventional systems by using a parent model with personal optimization information for user-specific estimation, allowing for accurate and efficient state estimation without retraining, thus overcoming the challenges posed by complex neural network models.
Patent Information
- Application Number
- PCT/JP2023/020042
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-30
- Publication Date
- 2024-12-05
AI Technical Summary
Conventional state estimation systems require relearning and storing multiple models for each user, leading to inefficiencies due to the complexity and size of neural network models, making real-time estimation challenging without retraining.
A state estimation device and method that uses a parent model with added personal optimization information to estimate user states without relearning, utilizing a storage unit for the parent model and an addition unit that incorporates optimization information for accurate user-specific estimation.
Enables efficient and accurate state estimation for each user without the need for retraining models, improving estimation accuracy and reducing processing time by using a parent model with personalized optimization information.
Smart Images

Figure JP2023020042_05122024_PF_FP_ABST
Abstract
Description
State estimation device, program, state estimation system, and state estimation method
[0001] The present disclosure relates to a state estimation device, a program, a state estimation system, and a state estimation method.
[0002] Conventionally, there have been technologies aimed at estimating a user's condition using detected data such as biosignals obtained from wearable devices, etc. However, since biometric information generally varies from person to person, it is difficult to prepare a general-purpose model that can be used by all users in advance.
[0003] For example, the situation estimation device described in Patent Document 1 receives a signal from a wearable device worn by a user, selects a model from pre-trained models based on the received signal, and re-trains the selected model each time training data is acquired. This enables the situation estimation device described in Patent Document 1 to improve the accuracy of estimating the user's situation, and enables each user to estimate their state using a model optimized for them.
[0004] Japanese Patent Application Laid-Open No. 2022-55736
[0005] However, in the conventional technology, the state estimation model is retrained using data obtained from each user, thereby achieving personal optimization of the inference model for each user. This requires retraining the model for each user who uses the system, and it is also necessary to store as many models on the system as there are users.
[0006] In recent years, estimation models using neural networks and other technologies have become more complex, and as a result, the size (capacity) of the models has increased. When such models are adopted in a system, it takes a very long time to retrain the models.
[0007] Therefore, one or more aspects of the present disclosure aim to make it possible to perform appropriate estimation for each user without re-learning a model for estimating a state.
[0008] A state estimation device according to one aspect of the present disclosure is characterized by comprising: an addition unit that adds personal optimization information to a parent model by inputting detection data detected from a person, so that estimation using a parent model for estimating the state of the person is suited to the user; and a state estimation unit that estimates the state of the user by inputting detection data detected from the user to the parent model to which the personal optimization information has been added.
[0009] A program according to one aspect of the present disclosure is characterized in that it causes a computer to function as an addition unit that inputs detection data detected from a person and adds personal optimization information to a parent model to make estimations using the parent model for estimating the state of the person suitable for the user, and a state estimation unit that inputs detection data detected from the user to the parent model to which the personal optimization information has been added, to estimate the state of the user.
[0010] A state estimation system according to one aspect of the present disclosure is characterized by comprising: a memory unit that stores a parent model for estimating a state of a person by inputting detection data detected from the person; an addition unit that adds personal optimization information to the parent model to make estimation using the parent model suitable for the user; and a state estimation unit that estimates the state of the user by inputting the detection data detected from the user to the parent model to which the personal optimization information has been added.
[0011] A state estimation method according to one aspect of the present disclosure is characterized in that, by inputting detection data detected from a person, personal optimization information is added to a parent model for estimating the state of the person so that estimation using the parent model is suited to the user, and the state of the user is estimated by inputting the detection data detected from the user to the parent model to which the personal optimization information has been added.
[0012] According to one or more aspects of the present disclosure, appropriate estimation can be performed for each user without re-learning a model for estimating a state.
[0013] FIG. 1 is a block diagram schematically showing the configuration of a state estimation device according to a first embodiment; FIG. 2 is a schematic diagram for explaining an example of adding personal optimization information to the output of a parent model; (A) and (B) are schematic diagrams for explaining an example of adding personal optimization information to some layers of a parent model; FIG. 3 is a block diagram schematically showing the configuration of a PC; FIG. 4 is a block diagram schematically showing the configuration of a state estimation device according to a second embodiment; and FIG. 5 is a block diagram schematically showing the configuration of a state estimation device according to a third embodiment.
[0014] 1 is a block diagram showing a schematic configuration of a state estimation device 100 according to embodiment 1. The state estimation device 100 includes a model management unit 110, an optimization information management unit 120, an input unit 130, and a detection unit 140.
[0015] The model management unit 110 includes a data storage unit 111 , a model generation unit 112 , and a parent model storage unit 113 .
[0016] The data storage unit 111 stores training data for training a parent model, which will be described later. The state estimation device 100 is a device that estimates a user's state based on detection data detected from the user. Therefore, the training data here includes detection data detected from the person and the person's state estimated from the detection data. In addition to the detection data, the training data may also include clinical data of the person collected at a hospital or the like and the person's state diagnosed from the clinical data.
[0017] The model generation unit 112 uses the training data stored in the data storage unit 111 to learn a parent model, which is a model for estimating the state of a person, based on detection data detected from the person.
[0018] The parent model storage unit 113 is a storage unit that stores the parent model generated by the model generation unit 112. In the above example, the parent model is generated by the model generation unit 112, but the first embodiment is not limited to this example. For example, the parent model may be generated by a device other than the state estimation device 100 and stored in the parent model storage unit 113. In this case, the data storage unit 111 and the model generation unit 112 are unnecessary.
[0019] The optimization information management unit 120 includes a personal optimization information generation unit 121 and a personal optimization information storage unit 122 .
[0020] The personal optimization information generation unit 121 generates personal optimization information for modifying a part or output of the parent model so that estimation using the parent model is suitable for the user of the state estimation device 100. For example, the personal optimization information generation unit 121 may use data (also referred to as personal optimization data) obtained in advance from the user of the state estimation device 100, in which detected data is associated with the state at the time the detected data was detected, to generate personal optimization information for the user so that the state of the user is estimated from the output when the detected data is input into the parent model. Such personal optimization data may be input via the input unit 130 or may be stored in the data storage unit 111.
[0021] The personal optimization information storage unit 122 stores the personal optimization information generated by the personal optimization information generation unit 121. Here, if there are multiple users using the state estimation device 100, the personal optimization information is stored in association with each of the multiple users. For example, the multiple users are assigned user IDs (IDentification) as user identification information that is identification information for identifying each of the multiple users, and the personal optimization information may be associated with the user IDs.
[0022] In the first embodiment, it is assumed that personal optimization information for a user has already been generated and stored in the personal optimization information storage unit 122 before the state estimation device 100 estimates the user's state. In the above example, the personal optimization information is generated by the personal optimization information generation unit 121, but the first embodiment is not limited to this example. For example, the personal optimization information may be generated by a device other than the state estimation device 100 and stored in the personal optimization information storage unit 122. In this case, the personal optimization information generation unit 121 is unnecessary.
[0023] The input unit 130 functions as an input receiving unit that receives input of various data. For example, the input unit 130 receives input of the user ID of the user of the state estimation device 100. The input unit 130 also functions as a detection data receiving unit that receives input of detection data detected from the user of the state estimation device 100. Here, the detection data is preferably, for example, vital data detected by a sensor (not shown). The vital data may be detected by a medical device (not shown) or by a user terminal (not shown) such as a smart watch used by the user. The input user ID and detection data are provided to the detection unit 140.
[0024] The detection unit 140 includes an addition unit 141 and a state estimation unit 142 .
[0025] The addition unit 141 reads out, from the personal optimization information storage unit 122, personal optimization information corresponding to the user ID received from the input unit 130. The addition unit 141 then adds the read-out personal optimization information to the parent model stored in the parent model storage unit 113. The parent model to which the personal optimization information has been added provides estimations suited to the user. Specifically, in a case where the parent model includes multiple sub-models that estimate a person's state from each of multiple types of data included in the detection data, the addition unit 141 adds, to the output of the parent model, personal optimization information that is information for estimating the user's state by weighting and evaluating the outputs of the multiple sub-models. Furthermore, the addition unit 141 may add, to the parent model, information that modifies the output of some layers of the parent model to suit the user as personal optimization information.
[0026] 2 is a schematic diagram for explaining an example of adding personal optimization information to the output of a parent model. For example, as shown in FIG. 2, if the parent model 113a is composed of multiple sub-models, such as a first model, a second model, and so on, which perform estimation for each of multiple types of data included in the detection data, the personal optimization information can be used as information for estimating the user's state from the combined results by weighting and combining the outputs from the multiple sub-models with weights (e.g., w1, w2, w3, and so on) determined for each user.
[0027] 3A and 3B are schematic diagrams illustrating an example of adding personal optimization information to some layers of a parent model. As shown in FIG. 3A, if the normal space CA1 assumed by the parent model differs from the normal space CA2, which is data from the user's detection data when the user's condition is normal, the user's normal biometric information may not be mapped to the normal space assumed by the parent model and may be determined to be abnormal. For this reason, the adding unit 141 inserts a projection function into some layers of the parent model to correct the output of the layer for each user, so that the normal space CA1 assumed by the parent model matches the normal space CA3, which is data from the user's detection data when the user's condition is normal, as shown in FIG. 3B. In such a case, the projection function inserted into some layers of the parent model becomes personal optimization information.
[0028] The state estimation unit 142 estimates the state of the user by inputting detection data detected from the user into a parent model to which personal optimization information has been added. The estimated state of the user is displayed, for example, on a display unit (not shown) or transmitted to a terminal used by the user via a communication unit (not shown). In other words, the estimated state of the user is output from an output unit (not shown).
[0029] The state estimation device 100 described above can be realized by a computer such as a PC 10 shown in Fig. 4. The PC 10 includes an auxiliary storage device 11 such as a hard disk drive (HDD) or a solid state drive (SSD), a memory 12, a processor 13 such as a central processing unit (CPU), a communication interface (I / F) 14 such as a network interface card (NIC), an input interface 15 such as a mouse or keyboard, and a display 16.
[0030] The data storage unit 111, parent model storage unit 113, and personal optimization information storage unit 122 of the state estimation device 100 can be realized by the auxiliary storage device 11 or the memory 12. The model generation unit 112, personal optimization information generation unit 121, addition unit 141, and state estimation unit 142 can be realized by the processor 13 reading a program stored in the auxiliary storage device 11 into the memory 12 and executing the program. The input unit 130 can be realized by the communication I / F 14 or the input I / F 15. Furthermore, an output unit (not shown) can be realized by the communication I / F 14 or the display 16.
[0031] As described above, according to the first embodiment, estimation suitable for a user can be performed using the parent model without re-learning the parent model itself for each user.
[0032] 5 is a block diagram showing a schematic configuration of a state estimation device 200 according to embodiment 2. The state estimation device 200 includes a model management unit 110, an optimization information management unit 120, an input unit 130, a detection unit 140, and an update unit 250.
[0033] The model management unit 110, optimization information management unit 120, input unit 130, and detection unit 140 of the state estimation device 200 according to the second embodiment are similar to the model management unit 110, optimization information management unit 120, input unit 130, and detection unit 140 of the state estimation device 100 according to the first embodiment. However, the input unit 130 also provides the input user ID and detection data to the update unit 250.
[0034] The update unit 250 includes a storage unit 251 and a personal optimization information update unit 252. The storage unit 251 stores the user ID and the detection data from the input unit 130 in association with each other.
[0035] The personal optimization information update unit 252 reads out, from the personal optimization information storage unit 122, personal optimization information associated with the user ID stored in the accumulation unit 251, and updates the personal optimization information using the detection data associated with the user ID. Here, the personal optimization information update unit 252 may update the personal optimization information of the user when the user's condition in the detection data becomes clear. The user's condition in the detection data can be clarified, for example, by inputting the user's condition into the input unit 130 by the user or a third party such as a doctor. In other words, when the user's condition corresponding to the detection data detected from the user is acquired, the personal optimization information update unit 252 updates the personal optimization information so that the acquired condition is estimated from the detection data.
[0036] Specifically, as shown in Figure 2, if the personal optimization information is information for estimating a user's state by weighting and combining outputs from multiple sub-models with weights determined for each user, the personal optimization information update unit 252 updates the weights so that the user's state estimated from the output obtained by inputting the detection data into the parent model becomes the clarified user's state. Also, as described using Figures 3(A) and (B), if the personal optimization information is a projection function inserted into a certain layer of the parent model, the personal optimization information update unit 252 updates the projection function so that the output obtained by inputting the detection data into the parent model becomes the clarified user's state.
[0037] The state estimation device 200 described above can also be realized by a computer such as the PC 10 shown in Fig. 4. For example, the accumulation unit 251 can be realized by the auxiliary storage device 11 or the memory 12. Furthermore, the personal optimization information update unit 252 can be realized by the processor 13 reading a program stored in the auxiliary storage device 11 into the memory 12 and executing the program.
[0038] As described above, according to the second embodiment, the personal optimization information can be updated to be more optimal by using the state estimation device 200. Therefore, by using the state estimation device 200, the estimation accuracy for each user can be improved.
[0039] 6 is a block diagram showing a schematic configuration of a state estimation device 300 according to embodiment 3. The state estimation device 300 includes a model management unit 110, an optimization information management unit 320, an input unit 330, a detection unit 140, an update unit 250, and a cluster identification unit 360.
[0040] The model management unit 110 and the detection unit 140 of the state estimation device 300 according to the third embodiment are similar to the model management unit 110 and the detection unit 140 of the state estimation device 100 according to the first embodiment. Furthermore, the update unit 250 of the state estimation device 300 according to the third embodiment is similar to the update unit 250 of the state estimation device 200 according to the second embodiment.
[0041] The optimization information management unit 320 includes a personal optimization information generation unit 121 , a personal optimization information storage unit 122 , a clustering unit 323 , a cluster optimization information generation unit 324 , and a cluster optimization information storage unit 325 .
[0042] The optimization information management unit 320 includes a personal optimization information generation unit 121 , a personal optimization information storage unit 122 , a clustering unit 323 , a cluster optimization information generation unit 324 , and a cluster optimization information storage unit 325 .
[0043] The personal optimization information generation unit 121 and the personal optimization information storage unit 122 of the optimization information management unit 320 in embodiment 3 are similar to the personal optimization information generation unit 121 and the personal optimization information storage unit 122 of the optimization information management unit 120 in embodiment 1.
[0044] The clustering unit 323 generates clusters, which are groups of similar users, by performing clustering using the teacher data stored in the data storage unit 111. The clustering performed here may be a known method of dividing data into groups (clusters) based on the similarity between the data.
[0045] For example, if the training data includes the user's clinical data, the clustering unit 323 can perform clustering using attributes such as the subject's age, medical history, and lifestyle habits included in the clinical data.
[0046] The clustering unit 323 may also perform clustering using detection data included in the training data. Furthermore, the clustering unit 323 may perform clustering using both the subject's attributes and the detection data. For example, the clustering unit 323 can cluster the feature space after including the subject's attributes in the feature amounts calculated from the detection data.
[0047] The cluster optimization information generation unit 324 generates cluster optimization information for correcting a part or output of the parent model so that the output from the parent model is suitable for the cluster generated by the clustering unit 323. For example, the cluster optimization information generation unit 324 may generate cluster optimization information for each cluster so that the state of a user included in a cluster generated by the clustering unit 323 can be estimated from the output when the detected data of the user is input to the parent model.
[0048] The cluster optimization information storage unit 325 stores the cluster optimization information generated by the cluster optimization information generation unit 324. For example, a cluster ID is assigned to each of the multiple clusters as cluster identification information, which is identification information for identifying each of the multiple clusters, and the cluster optimization information may be associated with the cluster ID. In the above example, the cluster optimization information is generated by the cluster optimization information generation unit 324, but the third embodiment is not limited to this example. For example, the cluster optimization information may be generated by a device other than the state estimation device 300 and stored in the cluster optimization information storage unit 325. In such a case, the clustering unit 323 and the cluster optimization information generation unit 324 are unnecessary.
[0049] The input unit 330 accepts input of various data. For example, the input unit 330 accepts input of a user ID of a user of the state estimation device 300, as in the first embodiment. Furthermore, the input unit 330 accepts input of detection data detected from the user of the state estimation device 300, as in the first embodiment. The input user ID and detection data are provided to the detection unit 140 and the update unit 250.
[0050] In the third embodiment, the input unit 330 also functions as a clustering data input receiving unit that receives input of clustering data, which is data necessary for clustering users of the state estimation device 300. For example, when the cluster optimization information generation unit 324 performs clustering using the attributes of the subjects to generate clusters, attribute data indicating the attributes of the users becomes the clustering data. When the cluster optimization information generation unit 324 performs clustering using the detection data to generate clusters, the detection data of the users becomes the clustering data. When the cluster optimization information generation unit 324 performs clustering using the attributes and detection data of the subjects to generate clusters, the attribute data indicating the attributes of the users and the detection data become the clustering data. The input unit 330 provides the user ID and the clustering data to the cluster identification unit 360.
[0051] The cluster identification unit 360 identifies a cluster of a user indicated by the user ID from the input unit 330 by performing clustering using the clustering data from the input unit 330. The clustering here may be performed using a method similar to the clustering performed by the clustering unit 323. In other words, the cluster identification unit 360 identifies the cluster to which a user belongs from among a plurality of clusters based on at least one of the user's attributes and the detection data detected from the user.
[0052] Then, the cluster identification unit 360 reads out the cluster optimization information associated with the cluster ID of the identified cluster from the cluster optimization information storage unit 325, associates the cluster optimization information with the user ID from the input unit 330, and stores the cluster optimization information as personal optimization information in the personal optimization information storage unit 122. The cluster optimization information stored as personal optimization information is updated by the personal optimization information update unit 252, as in the second embodiment.
[0053] The state estimation device 300 described above can also be realized by a computer such as the PC 10 shown in Fig. 4. For example, the cluster optimization information storage unit 325 can be realized by the auxiliary storage device 11 or the memory 12. Furthermore, the clustering unit 323, the cluster optimization information generation unit 324, and the cluster identification unit 360 can be realized by the processor 13 reading a program stored in the auxiliary storage device 11 into the memory 12 and executing the program.
[0054] As described above, according to the third embodiment, even if personal optimization information of a user of the state estimation device 300 is not stored in the state estimation device 300, estimation is performed using, as an initial value, the cluster optimization information of a cluster into which similar users are classified. Then, by that user using the state estimation device 300, the personal optimization information is updated to become more optimal. Therefore, by using the state estimation device 300, the estimation accuracy for each user is improved.
[0055] In the above-described first to third embodiments, examples have been shown in which processing is performed by one of the state estimation devices 100 to 300, but the first to third embodiments are not limited to such examples. For example, the processing performed by each of the state estimation devices 100 to 300 according to the first to third embodiments may be distributed and performed by a plurality of computers, such as a server and a PC, connected to a network such as the Internet. In other words, the processing performed by each of the state estimation devices 100 to 300 according to the first to third embodiments may be performed by a state estimation system.
[0056] For example, in the first embodiment, the state estimation system may be configured by a server (not shown) including the model management unit 110 and the optimization information management unit 120, and a PC (state estimation device) including the input unit 130 and the detection unit 140. Furthermore, in the second embodiment, the state estimation system may be configured by a server (not shown) including the model management unit 110 and the optimization information management unit 120, and a PC (state estimation device) including the input unit 130, the detection unit 140, and the update unit 250. Furthermore, in the third embodiment, the state estimation system may be configured by a server (not shown) including the model management unit 110 and the optimization information management unit 320, and a PC (state estimation device) including the input unit 330, the detection unit 140, the update unit 250, and the cluster identification unit 360.
[0057] 100, 200, 300 State estimation device, 110 Model management unit, 111 Data storage unit, 112 Model generation unit, 113 Parent model storage unit, 120 Optimization information management unit, 121 Personal optimization information generation unit, 122 Personal optimization information storage unit, 323 Clustering unit, 324 Cluster optimization information generation unit, 325 Cluster optimization information storage unit, 130 Input unit, 140 Detection unit, 141 Addition unit, 142 State estimation unit, 250 Update unit, 251 Accumulation unit, 252 Personal optimization information update unit, 360 Cluster identification unit.
Claims
1. A state estimation device comprising: an addition unit that adds personal optimization information to a parent model by inputting detection data detected from a person, so that estimation using a parent model for estimating the state of the person is suitable for the user; and a state estimation unit that estimates the state of the user by inputting the detection data detected from the user to the parent model to which the personal optimization information has been added.
2. The state estimation device according to claim 1, wherein a plurality of types of data are detected from the person as detection data, the parent model includes a plurality of sub-models that estimate the state of the person from each of the plurality of types of data, and the personal optimization information is information that estimates the state of the user by weighting and evaluating the outputs of the plurality of sub-models.
3. The state estimation device according to claim 1, wherein the personal optimization information is information for modifying the output of a part of layers of the parent model to suit the user.
4. A state estimation device as described in any one of claims 1 to 3, further comprising a personal optimization information update unit that updates the personal optimization information when a state of the user corresponding to detection data detected from the user is acquired, so that the acquired state is estimated from the detection data corresponding to the acquired state.
5. The state estimation device according to any one of claims 1 to 3, further comprising a cluster identification unit that identifies a cluster to which the user belongs among a plurality of clusters from the attributes of the user or detection data detected from the user, wherein the addition unit identifies cluster optimization information corresponding to the identified cluster among a plurality of cluster optimization information that makes estimation using the parent model suitable for people classified into each of the plurality of clusters, and adds the identified cluster optimization information to the parent model as the personal optimization information, and further comprising a personal optimization information update unit that updates the identified cluster optimization information when a state of the user corresponding to detection data detected from the user is acquired, so that the acquired state is estimated from the detection data corresponding to the acquired state.
6. A program causing a computer to function as: an addition unit that adds personal optimization information to a parent model by inputting detection data detected from a person, so that estimation using a parent model for estimating the state of the person is suited to the user; and a state estimation unit that estimates the state of the user by inputting detection data detected from the user to the parent model to which the personal optimization information has been added.
7. A state estimation system comprising: a memory unit that stores a parent model for estimating a state of a person by inputting detection data detected from the person; an addition unit that adds personal optimization information to the parent model so as to make estimation using the parent model suitable for a user; and a state estimation unit that estimates the state of the user by inputting the detection data detected from the user to the parent model to which the personal optimization information has been added.
8. A state estimation method comprising: adding personal optimization information to a parent model for estimating a state of a person by inputting detection data detected from the person, so as to make estimation using the parent model suitable for the user; and estimating the state of the user by inputting the detection data detected from the user to the parent model to which the personal optimization information has been added.
Citation Information
Patent Citations
Inference device, inference method, and inference program
JP2021022079A
Situation estimation apparatus, server apparatus, situation estimation system, and situation estimation method
JP2022055736A
Learning-based image compression setting
US20220053195A1
KR20210055992A
