Information processing system, information processing method, and program

Abstract

To enable more accurate estimation of frailty status. [Solution] An information processing system according to one aspect of the present disclosure includes: a primary analysis unit that acquires feature quantities indicating the state of a subject based on input data obtained from at least one of an image input unit that acquires an image of a subject and an audio input unit that acquires the voice of a subject; a secondary analysis unit that performs a time-series fluctuation analysis of the feature quantities and identifies the fluctuation status of the feature quantities at predetermined time intervals; and a frailty estimation unit that estimates the frailty state of the subject based on the analysis results of the secondary analysis unit.

Description

Technical Field

[0001] The present disclosure relates to an information processing system, an information processing method, and a program.

Background Art

[0002] In recent years, with the progress of an aging society, technologies for analyzing or estimating the health, frailty, etc. of elderly people without the diagnosis of a doctor using images, voices, etc. of the target person have attracted attention. For example, technologies for estimating the health state at a certain point in time based on the voice, face information, etc. of the target person at that point in time, and technologies for detecting fine facial skin movements of the target person at a certain point in time and performing emotion state, biometric authentication, etc. at that point in time have been proposed (see Patent Documents 1 to 4).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0004] <第 Conventional technologies only perform single-shot (so-called snapshot-type) evaluations for estimating the health state (also referred to as "frailty state") of a specific target person at a specific timing. The single-shot measurement values have problems that they easily vary due to slight differences in the physical condition on that day and the measurement environment, and contain a lot of noise. In addition, it is difficult to detect early a progressive state change such as a change in the frailty state or its omen by single-shot evaluation.

[0005] Therefore, the purpose of this disclosure is to provide an information processing system, an information processing method, and a program that enable more appropriate estimation of frailty status. [Means for solving the problem]

[0006] An information processing system according to one aspect of this disclosure includes: a primary analysis unit that acquires feature quantities indicating the state of a subject based on input data obtained from at least one of an image input unit that acquires an image of a subject and an audio input unit that acquires the voice of a subject; a secondary analysis unit that performs a time-series variation analysis of the feature quantities and identifies the variation status of the feature quantities at predetermined time intervals; and a frailty estimation unit that estimates the frailty state of the subject based on the analysis results of the secondary analysis unit.

[0007] An information processing method relating to another aspect of this disclosure is an information processing method executed by an information processing system, comprising: a primary analysis step of acquiring feature quantities indicating the state of a subject based on input data obtained from at least one of an image input unit that acquires an image of a subject and an audio input unit that acquires the voice of a subject; a secondary analysis step of performing a time-series variation analysis of the feature quantities and identifying the variation status of the feature quantities at predetermined time intervals; and a frailty estimation step of estimating the frailty state of the subject based on the analysis results of the secondary analysis step.

[0008] A program according to yet another aspect of this disclosure causes an information processing system to perform a primary analysis step of acquiring feature quantities indicating the state of a subject based on input data obtained from at least one of an image input unit that acquires an image of the subject and an audio input unit that acquires the voice of the subject; a secondary analysis step of performing a time-series variation analysis of the feature quantities and identifying the variation in the feature quantities at predetermined time intervals; and a frailty estimation step of estimating the frailty state of the subject based on the analysis results of the secondary analysis step. [Effects of the Invention]

[0009] According to one aspect of this disclosure, an information processing system, an information processing method, and a program can be provided that enable more appropriate estimation of frailty status. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 shows an example of the overall configuration of an information processing system according to the embodiment. [Figure 2] Figure 2 is a flowchart showing an overview of the processes performed by the information processing system according to the embodiment. [Figure 3] Figure 3 is a block diagram showing an example of the configuration of a terminal according to the embodiment. [Figure 4] Figure 4 is a block diagram showing an example of a server configuration according to the embodiment. [Figure 5] Figure 5 shows an example of a detailed data flow in the information processing system according to the embodiment. [Figure 6] Figure 6 is a diagram illustrating an example of the progression process of frailty according to this embodiment. [Figure 7] Figure 7 is a diagram illustrating an example of data variability analysis and risk assessment according to the embodiment. [Modes for carrying out the invention]

[0011] The information processing system according to this embodiment will be described below with reference to the drawings. In the drawings, identical or similar parts are denoted by the same or similar reference numerals.

[0012] (1) Example of system configuration FIG. 1 is a diagram showing an overall configuration example of an information processing system 1 according to the present embodiment. The information processing system 1 is a system that estimates (which may be paraphrased as analyzes, evaluates, or determines, etc.) the frailty state of a target person. The information processing system 1 includes, for example, a server 100, a terminal 200, and a user interface (I / F) 300 that are communicably connected to each other via a network 5. This configuration can be realized as a cloud computing system, a client-server system, an edge computing system, or the like.

[0013] The server 100 is a computer arranged on the Internet (i.e., on the cloud side), and is constituted by, for example, a cloud server, a web server, an application server, or the like. The server 100 executes various information processes such as estimating the frailty state of the target person based on the information transmitted from the terminal 200. Note that the server 100 may be a single server, or may be a server group (server cluster) in which a plurality of servers cooperate to function.

[0014] The network 5 is a communication network that enables communication between the server 100 and the terminal 200. The network 5 includes, for example, the Internet, a LAN (Local Area Network), a WAN (Wide Area Network), or a mobile phone network (e.g., 5G, LTE), etc.

[0015] The terminal 200 is included in an edge device (group) arranged on the target person (user) side. The edge device exists in the daily environment of the target person, for example, inside a home (such as a washroom, a kitchen, a toilet, a bathroom, a bedroom, etc.), or as a device carried by the target person. The terminal 200 may be, for example, a smartphone, a tablet terminal, a personal computer (PC), a smart mirror, a smart speaker, a wearable device (such as a smartwatch, etc.), or an in-vehicle device, etc. Also, it may be a smart mirror or the like installed in a washroom or the like as disclosed in JP-A-2025-155588 or Japanese Patent No. 7336688.

[0016] The user I / F 300 is a device for inputting and outputting information to and from the target person, and is included in the edge device(s). The user I / F 300 includes, for example, an image input unit 301 and an audio input unit 302. The user I / F 300 may further include an image output unit 303 and an audio output unit 304. The user I / F 300 communicates with the terminal 200 via wire or wirelessly.

[0017] The image input unit 301 is a device for acquiring (imaging) an image (still image or moving image) of the target person. The image input unit 301 is constituted by, for example, a camera (visible light camera, infrared camera, etc.) including an image pickup device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The image input unit 301 may be, for example, a camera mounted on a smartphone, a tablet terminal, a PC, etc., or a camera built in a smart mirror. The image input unit 301 acquires image data regarding the appearance and movement of the target person, such as changes in the arrangement of facial parts (eyes, mouth, etc.), movement of the corners of the mouth and the mouth, movement of the tongue, etc.

[0018] The audio input unit 302 is a device for acquiring (collecting) the audio of the target person. The audio input unit 302 is constituted by, for example, a microphone. The audio input unit 302 may be, for example, a microphone built in a smartphone, a PC, a smart speaker, a smart mirror, etc. The audio input unit 302 acquires audio data regarding the audio of the target person, such as the voice uttered by the target person, the content of the conversation, the timbre, the loudness (volume) of the voice, the voice quality (hoarseness, rattling voice, etc.), the conversation speed, the pitch (fundamental frequency), the jitter (fine fluctuation of the pitch), the rhythm (intonation), the smoothness of speech (e.g., clarity of a specific pronunciation "pataka", etc.).

[0019] The image input unit 301 and the audio input unit 302 function as sensing devices for acquiring the input data in the present embodiment. These sensings may be so-called passive sensing that is executed while the target person performs daily actions (e.g., washing face, having a conversation, eating, etc.) without being particularly conscious.

[0020] The image output unit 303 is a device that displays images (text, graphics, photographs, videos, etc.). The image output unit 303 is composed of, for example, a liquid crystal display (LCD), an organic electro-luminescence (OLED) display, or a projector. In the case of a smart mirror, the display may be integrated into the front or back surface of the mirror.

[0021] The audio output unit 304 is a device that outputs sound (warning sounds, guidance, conversational voices, etc.). The audio output unit 304 is composed of, for example, a speaker. The image output unit 303 and the audio output unit 304 are used to present (notify, provide feedback to) the target person the processing results from the server 100 or terminal 200 (for example, estimated results of frailty status, notification of changes, confirmation (inquiry) of health status, improvement suggestions, etc.).

[0022] The information processing system 1 primarily performs a primary analysis on terminal 200 (edge ​​side) to acquire feature quantities indicating the subject's state based on input data obtained from at least one of the image input unit 301 and the voice input unit 302. Then, a time-series analysis of the acquired feature quantities (secondary analysis) and estimation of the frailty state based on the results may be performed on terminal 200 or server 100 (cloud side). In particular, from the standpoint of protecting privacy, it is desirable to avoid as much as possible the transmission of raw input data (unstructured data) such as image data and voice data from terminal 200 to server 100.

[0023] For example, terminal 200 may perform a primary analysis and send only the features extracted from the input data (structured data or text data) to server 100. Alternatively, terminal 200 may perform a secondary analysis (variability analysis) in addition to the primary analysis and send only the analysis results (for example, an index showing the variability or the degree of deviation from the standard variation) to server 100.

[0024] In this way, by processing sensing data (input data) and personally identifiable information as much as possible within the edge device (terminal 200) and sending only anonymized or statistically processed analysis results to the cloud side (server 100) (so-called edge computing, or hierarchical data processing), it is possible to achieve both privacy protection and advanced analytical processing (for example, machine learning based on a large amount of data, or complex factor analysis). For example, it is conceivable to convert image information into text and send it to the server 100's SLM (Small Language Model) or LLM (Large Language Model) to accumulate insights, or to use blockchain or hash values ​​to ensure data integrity and traceability.

[0025] (2) Operation overview Figure 2 shows the information processing (information processing method) performed by the information processing system 1 according to this embodiment. This processing starts, for example, when sensing of a target person is initiated via the user I / F 300 of terminal 200.

[0026] First, in step S1, terminal 200 performs a primary analysis (primary analysis step).

[0027] For example, the primary analysis unit of terminal 200 (primary analysis unit 281 in Figure 3, described later) acquires feature quantities indicating the subject's state based on input data obtained from at least one of the image input unit 301 and the voice input unit 302. Through this step, objective indicators (feature quantities) indicating the subject's state at any given time are acquired by non-invasive means such as images or audio. This makes it possible to collect data that forms the basis for estimating the frailty state without burdening the subject.

[0028] Furthermore, "feature" refers to data fragments that quantify or qualitatively represent the subject's state (physical, mental, emotional state, etc.), and may be interpreted as "state index," "biometric parameter," "health index," "behavioral characteristics," or "emotion score," etc.

[0029] The features obtained in the primary analysis step (S1) can include a variety of things. For example, features based on image data from the image input unit 301 (face image, facial expression, etc.) can include skin tone, complexion, body temperature, changes in the arrangement of facial features (eyes, mouth, etc.), up and down of the corners of the mouth, mouth movements (how much it opens, speed, etc.), tongue movements, number of blinks, and eye movements. These can serve as indicators of the subject's stress level, fatigue level, or decline in oral function (oral frailty). Furthermore, features based on audio data from the audio input unit 302 can include the frequency spectrum of the voice, pitch (fundamental frequency), jitter (subtle fluctuations in pitch), shimmer (subtle fluctuations in amplitude), volume (loudness of voice), voice quality (hoarseness, raspy voice, etc.), conversation speed, prosody (intonation), and clarity of articulation (e.g., clarity of specific pronunciations). Furthermore, as in the case of Japanese Patent No. 7628354, the frequency of use of frequently occurring words, characteristic words, and positive or negative words (positive / negative word analysis) may be extracted from the conversation content transcribed by speech recognition and used as features. These can serve as indicators of the subject's emotions (joy, anger, sadness, pleasure, etc.), stress level, mental stress, depressive tendencies, or declines in cognitive function or swallowing function.

[0030] Regarding "features," they can be selected from all or some of the data sets mentioned above. Alternatively, in the learning and analysis using AI in the secondary analysis section, parameters that appear to be characteristic factors influencing the subjects' variability can be pre-selected as "features."

[0031] Furthermore, when deep learning (S26) is performed on a large dataset, by referring to "features" based on the subject's attributes (age, gender, medical history, etc.) and selecting "features" appropriate for the subject to perform primary analysis (especially the first time) and secondary analysis, frailty estimation (S19) can be performed with fewer resources, in a shorter time, and with high accuracy. In addition, notification of changes (S20) can be made quickly or in real time.

[0032] As mentioned above, the primary analysis step (S1) is preferably performed within the terminal 200, which is an edge device on the subject's side (e.g., a smartphone or smart mirror), from a privacy protection standpoint. This prevents raw data, such as image and audio data, from being transmitted outside the terminal 200.

[0033] Next, in step S2, terminal 200 or server 100 performs a secondary analysis (secondary analysis step).

[0034] For example, the secondary analysis unit (secondary analysis unit 282 in Figure 3, or secondary analysis unit 131 in Figure 4, described later) performs a time-series variation analysis of the features obtained in step S1 and identifies the variation status of the features at predetermined time intervals. This step allows for the understanding of not only the features at a single point in time, but also the patterns and trends of their temporal changes (variations).

[0035] "Variation analysis" refers to the process of analyzing the temporal changes and variability of data, and may be rephrased as "change analysis," "trend analysis," or "trend analysis." "Variation status" refers to the way data has changed over time, and may be rephrased as "variation pattern," "trend of change," or "degree of variability." Identifying the "variation status" includes, for example, calculating the mean, median, maximum, minimum, variance, standard deviation, coefficient of variation, or the number of times a specific threshold (e.g., LLV / HLV described later) is exceeded, or the integral value (accumulation).

[0036] Frailty does not progress overnight; rather, it is thought to be gradually influenced by small daily changes and the accumulation of stress (see Figure 6). Therefore, by performing a time-series analysis of fluctuations, it becomes possible to detect signs and the degree of progression of frailty more early and accurately. Thus, the secondary analysis step (S2) is the process of accumulating and analyzing the time-series data (historical data) of the features obtained in the primary analysis step (S1).

[0037] This analysis may be performed on terminal 200 or on server 100. For example, processing can be distributed, with relatively short-term fluctuation analysis (e.g., daily fluctuations) performed on terminal 200, and medium- to long-term fluctuation analysis (e.g., weekly fluctuations, monthly fluctuations) or analyses including comparisons with other subjects performed on server 100, which has more computing resources.

[0038] Furthermore, while the "specified time interval" may be a single length (e.g., only "1 day"), it is desirable to include multiple time intervals with different durations (e.g., "daily," "weekly," "monthly," etc.) (see Figure 6). This makes it possible to distinguish between or correlate short-term abrupt changes (e.g., temporary physical discomfort due to lack of sleep) and the accumulation of long-term changes (e.g., mental damage due to accumulated stress or progression of frailty due to decreased behavior).

[0039] Finally, in step S3, server 100 performs frailty estimation (frailty estimation step).

[0040] For example, the frailty estimation unit (frailty estimation unit 132 in Figure 4, described later) estimates the subject's frailty status based on the results of the secondary analysis (variability analysis) in step S2. "Frailty status" refers to a state in which reserve capacity such as physical and cognitive functions declines with age, and vulnerability to stress (invasion) increases. It may also be interpreted as "weak state," "pre-care stage," or "intermediate between health and care needs." This may include multifaceted aspects such as physical frailty, mental / psychological frailty, and social frailty. "Estimating" means, for example, inferring and determining an unknown matter (in this case, the frailty status) based on known data (in this case, the results of the secondary analysis). It may also be interpreted as "determining," "evaluating," "predicting," or "scoring."

[0041] This step not only presents changes in feature quantities but also estimates how much they relate to a more specific health indicator, namely the subject's "frailty status." This makes it easier for the subject and their stakeholders (family, doctors, etc.) to intuitively understand changes in their health status and take necessary measures (e.g., improving lifestyle habits, seeking medical attention).

[0042] The frailty estimation step (S3) may be a process that determines how well the "variability" identified in the secondary analysis step (S2) (e.g., daily, weekly, monthly, and multi-month variation patterns and accumulation) matches a specific pattern indicating a frail state, or how much it increases the risk of frailty. This estimation may be performed, for example, based on a predetermined rule-based logic, or using a machine learning model (e.g., support vector machine, random forest, neural network, deep learning, etc.). In this case, an estimation model may be pre-built using variation data of the features of a large number of subjects and the actual frailty status of those subjects (e.g., a doctor's diagnosis) as training data to learn the correlation between variation patterns and frailty status. Then, by inputting the results of the secondary analysis of the subject under analysis into this estimation model, the subject's current frailty status (e.g., frailty score, or categories such as "healthy," "pre-frail," or "frail") can be estimated (classified or regressed).

[0043] Thus, according to this embodiment, it is possible to perform highly accurate frailty estimation compared to conventional techniques that perform a one-off (so-called snapshot) evaluation of the subject's health status (frailty state) at a specific timing. Furthermore, it becomes possible to detect progressive changes in status, such as changes in frailty, or their precursors at an early stage.

[0044] (3) Example of terminal configuration Figure 3 is a block diagram showing an example configuration of terminal 200 according to this embodiment.

[0045] The terminal 200 includes an image input unit 210, an audio input unit 220, an image output unit 230, an audio output unit 240, a communication unit 250, an operation input unit 260, a storage unit 270, and a processing unit 280.

[0046] The image input unit 210, audio input unit 220, image output unit 230, and audio output unit 240 have the same configuration as the image input unit 301, audio input unit 302, image output unit 303, and audio output unit 304 of the user I / F 300 described above.

[0047] The communication unit 250 is an interface for communicating with the server 100 and other devices. The communication unit 250 is composed of, for example, a wired LAN adapter, a wireless LAN (Wi-Fi) adapter, a short-range wireless communication adapter such as Bluetooth, or a communication module compatible with a mobile phone network (5G, LTE, etc.). The communication unit 250 communicates with the user I / F 300 and with the server 100 via the network 5. The communication unit 250 transmits, for example, the analysis results of the primary analysis unit 281 or secondary analysis unit 282, which will be described later, to the server 100.

[0048] The operation input unit 260 is a device that receives operations from the user (for example, turning the power on / off, changing settings, inputting responses to inquiries, etc.), and is composed of, for example, a touch panel (which may be integrated with the image output unit 230), physical buttons, a keyboard, or a mouse.

[0049] The memory unit 270 is a storage device that stores various types of data and programs. The memory unit 270 is composed of, for example, ROM, RAM, flash memory, HDD (Hard Disk Drive), or SSD (Solid State Drive). The memory unit 270 stores, for example, the OS (Operating System), a program (application) for executing the information processing according to this embodiment, input data (or its raw data) acquired from the image input unit and the audio input unit, analysis results (feature quantities, fluctuation status) from the primary analysis unit 281 and the secondary analysis unit 282, and information received from the server 100 (frailty estimation results, proposed information, etc.). For privacy protection, the input data (raw data) may be stored only in a temporary buffer within the memory unit 270 and immediately erased after the completion of the primary analysis, or it may be stored encrypted.

[0050] The processing unit 280 controls the overall operation of the terminal 200 and performs various information processing. The processing unit 280 is composed of processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an NPU (Neural Processing Unit).

[0051] The processing unit 280 functions, for example, by the processor executing a program. The processing unit 280 includes a primary analysis unit 281. The processing unit 280 may further include a secondary analysis unit 282. These functional blocks may also be software functions realized by the processor executing a program stored in memory.

[0052] The primary analysis unit 281 is a functional unit that executes step S1 (primary analysis step) in Figure 2. For example, the primary analysis unit 281 acquires feature quantities indicating the state of the subject based on input data obtained from at least one of the image input unit 301 and the voice input unit 302 of the user I / F 300. Alternatively, the primary analysis unit 281 may acquire feature quantities indicating the state of the subject based on input data obtained from at least one of the image input unit 210 and the voice input unit 220 of the terminal 200.

[0053] The primary analysis unit 281 may include, for example, an image analysis unit (see S13 in Figure 5) and a voice analysis unit (see S14 in Figure 5). The image analysis unit acquires image features such as skin tone, facial expression, mouth corner movement, and tongue movement based on image data (including video). Image processing techniques and machine learning (e.g., face recognition models, facial expression recognition models) may be used for this. The voice analysis unit acquires voice features such as frequency characteristics, pitch, jitter, voice quality, and articulation based on voice data. Voice signal processing techniques (e.g., frequency analysis, spectral analysis), speech recognition techniques, natural language processing techniques (e.g., frequent word analysis after text conversion, feature word analysis, positive / negative word analysis, etc.), and emotion recognition techniques may be used for this. These processes, especially analyses using machine learning models, can be executed at high speed on the edge side by utilizing the capabilities of the processor (e.g., NPU, GPU, or smartphone AI chip) installed in the terminal 200. This makes it possible to acquire features in near real-time while respecting privacy.

[0054] The secondary analysis unit 282 is a functional unit that performs part or all of step S2 (secondary analysis step) in Figure 2. Based on the time-series data of feature quantities acquired by the primary analysis unit 281 (which can be stored in the storage unit 270), the secondary analysis unit 282 performs variation analysis and identifies the variation status of feature quantities at predetermined time intervals. In the example in Figure 3, the terminal 200 is shown to have both the primary analysis unit 281 and the secondary analysis unit 282, but when the secondary analysis is performed on the server 100, the terminal 200 does not need to have the secondary analysis unit 282.

[0055] In other words, as a first mode of data transmission, the primary analysis unit 281 is installed on the subject's edge device (e.g., terminal 200). The secondary analysis unit 131 and the frailty estimation unit 132 are installed on a server 100 on the internet (see Figure 4). In the first mode, input data is not transmitted from the edge device to the server 100, but the analysis results of the primary analysis unit 281 are transmitted from the edge device to the server 100. This mode can provide a hybrid architecture that realizes privacy by design. Sensitive input data such as raw images and voices of the subject may not be transmitted outside the edge device, and the primary analysis unit 281 may extract features (such as text information and numerical data) within the edge device. Then, only the analysis results processed into a format that makes it difficult to identify individuals are transmitted to the server 100. This can minimize the risk of leakage of sensitive personal information, which has been a major problem from the perspective of privacy protection. At the same time, server 100 can perform advanced secondary analysis 131 and frailty estimation 132 utilizing its computing resources, making it possible to achieve both privacy protection and highly accurate analysis.

[0056] Furthermore, "edge device" refers to a device that is installed, for example, in the user's possession or home and performs data processing at the network edge, and may be interpreted as "terminal," "user terminal," or "on-premise equipment." "Server" 100 refers to a computer that provides services or data via a network, and may be interpreted as "cloud server," "information processing device," or "computer center."

[0057] Furthermore, "analysis results" refers to information (features) extracted from the input data by the primary analysis unit 281, and may be interpreted as "extracted data," "feature vectors," "structured data," etc. The action of "transmitting" refers to sending data to another device via a network, and may be interpreted as "uploaded," "transferred," "communicated," etc.

[0058] Directly transmitting a subject's daily image and audio data to the cloud (server 100) poses a significant privacy concern. To address this challenge, this embodiment employs distributed processing. Specifically, the "primary analysis," which analyzes facial features, expressions, and voice tone, is completed within an edge device (e.g., a smartphone or smart mirror) located near the user. The data transmitted to server 100 is not the raw images or audio itself, but rather "analysis results" (features) processed into a format that makes it difficult to identify individuals, such as "text conversion of image information" or "hash values" obtained by irreversibly transforming the data. This architecture also offers the secondary benefit of reducing communication bandwidth and cloud (server 100) storage (storage unit 120) costs.

[0059] In a second mode of data transmission, the primary analysis unit 281 and the secondary analysis unit 282 are located on the subject's edge device. The frailty estimation unit 132 is located on a server 100 on the internet (see Figure 4). In the second mode, input data is not transmitted from the edge device to the server 100, but the analysis results of the secondary analysis unit 282 are transmitted from the edge device to the server 100. This mode may further expand the processing scope on the edge device side compared to the first mode, thereby further enhancing privacy protection. In addition to primary analysis (feature extraction), the secondary analysis unit 282 within the edge device may also perform secondary analysis (variability analysis) which analyzes the temporal variation of features. As a result, the time-series data of the features themselves are not transmitted outside the edge device, and only the "results" of the variation (for example, indicators such as "short-term daily variation increased to 1.5 times the standard" or "medium-term monthly trend is deteriorating") may be transmitted to the server 100. This further reduces the risk of leakage of personally identifiable information and can achieve a higher level of privacy protection.

[0060] The term "secondary analysis unit" refers to a functional unit that analyzes the temporal changes in the features extracted by the primary analysis unit 281, and may be interpreted as "variability analysis unit," "trend analysis unit," "accumulation analysis unit," etc. The term "analysis results of the secondary analysis unit" refers to the fluctuation status of the features identified by the secondary analysis unit 282, and may be interpreted as "variability index," "trend data," "trend analysis results," etc. The term "frailty estimation unit" refers to a functional unit that estimates the frailty status of the subject based on the analysis results, and may be interpreted as "status determination unit," "weakness evaluation unit," "risk scoring unit," etc.

[0061] A second embodiment may be an architecture that maximizes the level of privacy protection by utilizing the computing power of edge devices (for example, an AI chip installed in a smartphone). In this embodiment, not only the "primary analysis unit" (S13, S14) in the flow of Figure 5, but also some or all of the "secondary analysis unit = variation analysis" (S15, S16, S17, S18) is executed by the secondary analysis unit 282 on the edge device side. In particular, it may be efficient to execute some of the "short-term variation analysis" (S15) and "(individual) standard variation analysis" (S18), which require real-time performance, on the edge device. Furthermore, a configuration is also conceivable in which the "analysis results of the secondary analysis unit" (indicators of the variation status and the textualized description of the changes) are sent to the server 100 only when a specific condition is met, such as when the "variation status" exceeds a preset threshold (for example, LLV / HLV derived from the individual's standard variation), i.e., when there is a sign of variation in the measured value. This is expected to dramatically reduce communication volume and cloud costs. The secondary analysis unit 131 on the server 100 side may also be responsible for further aggregating and analyzing the secondary analysis results transmitted from the edge devices (for example, aggregation across multiple devices or longer-term analysis).

[0062] In the following embodiments, a first mode of data transmission is primarily assumed, but a second mode of data transmission may also be assumed.

[0063] (4) Server configuration example Figure 4 shows an example of the configuration of server 100 according to this embodiment.

[0064] The server 100 includes a communication unit 110, a storage unit 120, and a processing unit 130.

[0065] The communication unit 110 is an interface for communicating with the terminal 200 and other devices via the network 5. The communication unit 110 receives, for example, primary analysis results (e.g., feature quantities) or secondary analysis results (e.g., fluctuation status) from the terminal 200. The communication unit 110 may also transmit processing results from the processing unit 130 (e.g., frailty estimation results, fluctuation notifications, inquiries, suggestion information, etc.) to the terminal 200.

[0066] The memory unit 120 is a large-capacity storage device that stores various types of data and programs. The memory unit 120 can function as a database (DB) that stores, for example, the OS, a program for executing the information processing according to this embodiment, time-series data of primary or secondary analysis results received from the terminal 200 (for each subject or for multiple subjects), processing results by the processing unit 130 (frailty estimation results, factor analysis data, etc.), machine learning models (estimation models, trained parameters, etc.), subject attribute data (personal attribute input, see S24 in Figure 5), behavioral data (vital data, etc., see S23 in Figure 5), etc.

[0067] The processing unit 130 controls the operation of the entire server 100 and performs various information processing. The processing unit 130 is composed of processors such as a CPU, GPU, and NPU. The processing unit 130 functions, for example, by the processor executing a program. The processing unit 130 may include, for example, a secondary analysis unit 131, a frailty estimation unit 132, a standard variation analysis unit 133, a variation notification unit 134, a factor analysis unit 135, a data acquisition unit 136, a verification unit 137, a learning processing unit 138, and a proposal processing unit 139. These functions are realized by the processing unit 130 executing a program stored in the storage unit 120.

[0068] In a first embodiment of the server 100, the frailty estimation unit 132 may estimate the subject's frailty status based on the analysis results of the primary analysis unit 281 and the analysis results of the secondary analysis units 131 and 282. This embodiment can improve the accuracy and immediacy of frailty estimation. This is because the frailty estimation unit 132 considers both the subject's "state at that moment" (primary analysis result, snapshot) and the "trend of changes from the past" (secondary analysis result, vector). For example, it can capture both sudden changes such as "normal fluctuations (secondary analysis result is normal), but a sudden deterioration in complexion today (primary analysis result is abnormal)" and chronic changes such as "daily fluctuations are within an acceptable range (primary analysis result is almost normal), but a long-term deterioration trend is accumulating (secondary analysis result is abnormal)." This makes it possible to distinguish between short-term poor health and long-term progressive changes that are signs of frailty, while achieving a comprehensive estimation of the frailty status.

[0069] In the first embodiment, while the conventional technology is limited to snapshot-type evaluation that "analyzes the health status of a specific subject at a specific time," this embodiment further reinforces the introduction of dynamic "vector" analysis and may combine the two. Single measurement values ​​(primary analysis results) fluctuate due to differences in the subject's physical condition and measurement environment on that day and are prone to noise. On the other hand, by capturing the "trend" over time (secondary analysis results), temporary fluctuations (noise) can be smoothed out, and signs (signals) of irreversible changes in state can be extracted. In this embodiment, the frailty estimation unit 132 of the server 100 may use both of these information as input. For example, as shown in Figure 7, it may comprehensively judge "short-term fluctuations (detection of sudden changes)" (when the primary analysis result exceeds the individual's LLV / HLV) and "long-term fluctuations (accumulative system / detection of frailty tendencies)" (when the secondary analysis result exceeds a threshold on a weekly or monthly basis). This makes it possible to evaluate the subject's condition from multiple angles and estimate with higher reliability whether it is merely a temporary ill health or a sign of progression of frailty.

[0070] In a second embodiment of the server 100, the predetermined time interval for identifying the fluctuation status of feature quantities may include multiple time intervals with different durations. The secondary analysis units 131 and 282 may perform fluctuation analysis for each of these multiple time intervals. This embodiment can address the essential issue of frailty, a "slowly progressing state." In this embodiment, the secondary analysis units 131 and 282 may be provided with analysis functions for different time scales, such as a "short-term fluctuation analysis unit," a "medium-term fluctuation analysis unit," and a "long-term fluctuation analysis unit" (S15, S16, S17 in Figure 5). This makes it possible to distinguish or correlate and analyze short-term fluctuations in physical condition (e.g., daily sleep deprivation), medium-term stress accumulation (e.g., weekly physical damage), and even long-term trends in state changes (e.g., monthly mental damage or several-month-long decline in behavior). This makes it possible to extract with high reliability the precursors (signals) of gradual but irreversible changes that might be missed with a single time scale.

[0071] Furthermore, "predetermined time intervals" refers to, for example, the unit or length of the period subject to the variation analysis, and may be reinterpreted as "analysis timescale," "time window," "aggregation period," etc. "Multiple time intervals with different durations" refers to setting periods of different granularity, such as days, weeks, months, and several months, and may be reinterpreted as "multi-layered time scales," "multi-timeframes," "hierarchical period settings," etc.

[0072] In this embodiment, to address the challenge that frailty and cognitive decline are "progressive" diseases, for example, the system captures temporal "trends" rather than single measurements (snapshots). The progression of frailty is a complex process in which various events can be involved on different time scales. For example, sleep deprivation and autonomic nervous system disorders may appear as short-term fluctuations on a "daily" basis, physical damage on a "weekly" basis, stress accumulation on a "monthly" basis, and long-term fluctuations such as decreased behavior and depression / dementia on a "several-month" basis. As shown in Figure 5, the secondary analysis unit 131 performs fluctuation analysis on these different time scales (short-term (e.g., daily) fluctuation analysis S15, medium-term (e.g., weekly) fluctuation analysis S16, and long-term (e.g., monthly) fluctuation analysis S17). This smooths out temporary fluctuations (noise), captures essential signals indicating the progression of frailty in a multi-layered manner, and can dramatically improve estimation accuracy.

[0073] In a third embodiment of the server 100, the secondary analysis unit 131 may include a standard variation analysis unit 133 that identifies the standard variation of the subject for each of several time intervals. The frailty estimation unit 132 may estimate the deterioration of the frailty state based on whether the variation of the feature in any of the several time intervals exceeds the standard variation. This embodiment makes it possible to estimate frailty in a personalized way, tailored to the constitution and lifestyle of the individual subject, rather than using a uniform standard value. The standard variation analysis unit 133 of the server 100 may learn and identify the subject's unique normal variation pattern (standard variation). Since the normal level of facial color and voice tone, or the range of daily variation, differs from person to person, detecting deviations from this "individual standard" makes it possible to capture meaningful changes (signs of frailty) earlier and more accurately than when using a uniform threshold. This reduces false positives and increases the reliability of the estimation.

[0074] Furthermore, "standard variation" refers to, for example, the normal (healthy) range or pattern of variation of feature quantities for an individual subject, and may be reinterpreted as "baseline variation," "normal pattern," "individual reference value," etc. "Standard variation analysis unit" refers to, for example, a functional unit that identifies (learns) the standard variation of an individual subject, and may be reinterpreted as "baseline setting unit," "individual model learning processing unit," "calibration unit," etc. "Exceeding standard variation" refers to, for example, the variation status of feature quantities deviating from the identified range of standard variation (e.g., statistically acceptable range), and may be reinterpreted as "deviation from baseline or master curve," "outside normal range," "abnormal variation," etc.

[0075] This embodiment may further personalize the time-series fluctuation analysis. The standard fluctuation analysis unit 133 of the server 100 may use the time-series data of the primary or secondary analysis results transmitted from the terminal 200 (stored in the storage unit 120) to identify the range of the individual's unique "acceptable fluctuation (normal)" from data of the subject's past stable period using statistical methods or machine learning (AI). For example, as shown in Figure 7, individual "Lower Limit Value (LLV)" and "Higher Limit Value (HLV)" may be set on the server 100 side for each feature and each time interval (day, week, month) and stored in the storage unit 120. These thresholds are not merely fixed values, but may be adaptively updated according to the subject's condition. The frailty estimation unit 132 monitors whether the latest fluctuation status identified by the secondary analysis units 131,282 exceeds these individual thresholds (standard fluctuations) (for example, the "number of times the weekly or monthly value exceeds LLV or HLV" or the "exceeded in the integral value" as shown in Figure 7). Alternatively, methods such as Bayesian estimation may be used to define standard variation, taking into account not only the threshold exceedance but also the range of variation and probabilistic factors influencing the variation, and then evaluating the deviation from this standard variation.

[0076] In a fourth embodiment of the server 100, a change notification unit 134 may be further provided, which notifies at least one of the subject and their related parties in response to the frailty estimation unit 132 estimating a change in frailty status. In this embodiment, the embodiment may function not merely as an analysis system, but as a practical health management partner that encourages the subject's "awareness" and enables early intervention. The change notification unit 134 may notify the subject themselves, or related parties such as pre-registered family members or their primary care physician, in an appropriate manner of the change (signal) detected by the frailty estimation unit 132 of the server 100. This may enable the subject to become aware of changes in their physical condition and be given an opportunity to review their lifestyle, provide family members living in distant locations with a sense of security through "monitoring," and encourage early intervention by medical professionals.

[0077] "Changes in frailty status" refers to, for example, signs of progression or worsening of frailty, or an increase in risk, as estimated by the frailty estimation unit 132, and may be reinterpreted as "worsening of condition," "increased risk," "changes in health status," etc. However, "changes in frailty status" may also include improvements in frailty status.

[0078] "Related parties" refers to third parties who are permitted to share information about the subject's health status, such as family members, caregivers, friends, primary care physicians, and staff at community comprehensive support centers, and may be interpreted as "family members, etc.," "support staff," or "watchers."

[0079] The term "variation notification unit" refers to a functional unit that notifies of estimated changes, and may be reinterpreted as "notification unit," "alert generation unit," "feedback unit," etc. The action of "notifying" refers to transmitting information and drawing attention, and may be reinterpreted as "notifying," "issuing an alert," "providing feedback," etc.

[0080] This embodiment may also include a function that links analysis results (estimation) to specific actions (notification). For example, an edge device or terminal 200 acting as a user I / F 300 may provide positive feedback such as good complexion, good voice tone, or the color of clothing, or conversely, point out signs of change such as disheveled clothing or poor hygiene. In the flow of Figure 5, after "frailty estimation" in step S19, "change notification" is performed in step S20, which is presented to the target person through the image output (S21) or audio output (S28) of the terminal 200. The change notification unit 134 of the server 100 may be responsible for generating this notification content and transmitting it to the terminal 200 via the communication unit 110. Also, in Figure 7, it is shown that "notification is given to the user" when short-term changes (sudden changes) or long-term changes (accumulation) are detected. The change notification unit 134 may control the content, timing, and recipients of the notification (whether only the individual or also informs relevant parties) according to the urgency and importance of the estimated change (e.g., whether it is "normal" or "dangerous" in Figure 7).

[0081] In a fifth embodiment of the server 100, the server may further include a factor analysis unit 135 that analyzes the factors causing the change in frailty status, in response to the frailty estimation unit 132 estimating a change in frailty status. In this embodiment, the system can provide not only notification that "a change has occurred," but also deeper and more practical information on "why that change occurred." The factor analysis unit 135 of the server 100 may compare the change in frailty status (result) with various potentially related data (e.g., daily fluctuations in feature quantities, changes in lifestyle, stress events, etc.) to analyze the underlying factors of the change. This can enable the subject and related parties to understand not only the superficial symptoms but also the underlying causes (e.g., "due to continued sleep deprivation," "due to increased caregiving stress"), and to take more accurate and fundamental measures (e.g., improving sleep, introducing care support).

[0082] The term "factor" refers to, for example, the cause of a certain outcome (in this case, a change in the state of frailty) or the factors that contributed to it, and may be reinterpreted as "cause," "contributing factor," "related factor," etc. The term "factor analysis unit" refers to, for example, a functional unit that identifies the factors of change, and may be reinterpreted as "cause analysis unit," "contribution analysis unit," "correlation analysis unit," etc. The action of "analyzing the factors of change" refers to, for example, analyzing the relationship between the change and other data and inferring the cause, and may be reinterpreted as "identifying the cause," "evaluating the degree of contribution," "inferring causal relationships," etc.

[0083] According to this embodiment, the system of this embodiment can function as an advanced analysis platform. In the flow chart of Figure 5, step S25 is positioned as "(Personal Data) Learning (DB → Factor Analysis)". The factor analysis unit 135 of the server 100 receives the result (change) of the estimation (S19) by the frailty estimation unit 132 and uses various data stored in the memory unit 120 (primary analysis results, secondary analysis results, behavioral data and attribute data described later, and verification results) as input to explore the factors. For this analysis, personal attributes may also be taken into account, and factor analysis may be performed using neural networks, causal AI, cocorrelation analysis, etc. For example, learning complex causal relationships in which multiple factors are intertwined, such as "caregiving stress leads to worsening of complexion through sleep deprivation," and "sensitivity analysis such as which fluctuating factors are large" (S18) using Bayesian estimation can also be considered as part of the function of the factor analysis unit 135.

[0084] In a sixth embodiment of the server 100, the server 100 further includes a data acquisition unit 136 that acquires at least one of measurement data relating to the subject's behavior and attribute data relating to the subject's attributes, and the factor analysis unit 135 may analyze the factors of the change in the frail state based on the data acquired by the data acquisition unit 136. This embodiment can significantly improve the accuracy and reliability of factor analysis. Image and audio data (primary analysis) alone may reveal the subject's "state" (result), but there may be limitations in identifying its "background" (cause). The data acquisition unit 136 of the server 100 acquires "measurement data relating to behavior," such as sleep data from a smartwatch and usage data obtained from toilets and bathrooms, as well as "attribute data relating to attributes," such as age, gender, and medical history, and provides these to the factor analysis unit 135. This allows for a more accurate factor analysis by combining behavioral data such as "sleep data has worsened" with background information (obtained in the verification described later) such as "care has recently started" in response to a change in state such as "pale complexion."

[0085] Furthermore, "measurement data related to behavior" refers to data that shows the subject's behavior and physiological state in their daily life, and may be reinterpreted as "life log data," "vital data," "activity level data," etc. "Attribute data related to attributes" refers to basic personal information and background information of the subject, and may be reinterpreted as "personal attributes," "demographic information," "profile data," etc. "Data acquisition unit" 136 refers to a functional unit that acquires supplementary data other than images and sounds from external sources (other IoT devices or databases), and may be reinterpreted as "external sensor linkage unit," "data linkage unit," "information collection unit," etc. The action of "acquiring" refers to receiving data from an external source and inputting it into the system, and may be reinterpreted as "collecting," "receiving," "linking," etc.

[0086] In this embodiment, this embodiment may be designed not merely as a "frailty estimation device," but as a "central hub (platform)" that integrates a wide variety of health and lifestyle data within the home, as described in Japanese Patent Application Publication No. 2025-155588. For example, behavioral data could include sleep data obtained from a smartwatch, activity levels (lack of exercise), usage status obtained from toilets and bathrooms (usage status of flushing, etc., decrease in frequency / duration of toilet use, decrease in frequency / duration of bathroom use), increased nocturnal enuresis, and stool scans. Attribute data could include age, gender, medical history, and family structure (e.g., elderly person living alone). The data acquisition unit 136 of the server 100 works in conjunction with these various IoT devices and external databases to acquire data and store it in the storage unit 120. The factor analysis unit 135 can further improve the accuracy of frailty estimation by taking into account this multifaceted information.

[0087] In a seventh embodiment of the server 100, the server 100 further includes a verification unit 137 that performs inquiry processing to the subject to confirm the subject's health status in response to the frailty estimation unit 132 estimating a change in the frailty state, and the factor analysis unit 135 may analyze the factors of the change in the frailty state based on the verification results of the verification unit 137. This embodiment can realize a "two-way feedback loop" between the system and the user. Rather than the system unilaterally presenting analysis results, the verification unit 137 of the server 100 may also make proactive questions such as, "We have observed these changes, how are you feeling?" When the subject responds to this (e.g., "Come to think of it, I wonder if it's because I've started taking care of my husband"), the system can link objective sensing data (results) with the user's subjective life background (causes). As a result, the factor analysis unit 135 can perform factor analysis by taking into account important information (such as life events) that cannot be obtained from sensing alone, and the accuracy of the analysis can be dramatically improved.

[0088] Furthermore, "inquiry processing" refers to interactive processing that presents questions to the subject and prompts them to answer, and may be reinterpreted as "user confirmation," "questioning," "hearing," etc. "Verification unit" refers to a functional unit that confirms the validity of the estimation results and background factors with the subject, and may be reinterpreted as "question generation unit," "dialogue control unit," "response input unit," etc.

[0089] This embodiment may also include a function that elevates the system of this embodiment from a mere monitoring tool to a partner that interacts with the user and learns together. Following the change notification (S20), "(user confirmation) verification" is performed in step S22. The verification unit 137 of the server 100 generates questions such as "Q1: 'We have observed these changes, how are you feeling?'" or "Q2: 'If you have any idea what it might be, please let us know.'" and transmits them to the terminal 200 via the communication unit 110, where they are presented to the subject via image output (S21) or audio output (S28). The subject's response (e.g., "A2: 'Come to think of it, maybe it's because I've started taking care of my husband.'") is received via the voice input unit or operation input unit of the terminal 200, transmitted to the server 100 as "verification results," and stored in the storage unit 120. The factor analysis unit 135 integrates and analyzes these subjective verification results with objective data obtained through sensing (such as behavioral and attribute data) (S25), making it possible to learn deeper causal relationships, such as "caregiving stress leads to a worsening of complexion through sleep deprivation."

[0090] In an eighth aspect of server 100, the server may further include a learning processing unit 138 that learns from the analysis data of the factor analysis unit 135 for each of multiple subjects, including the subject, using machine learning, and a suggestion processing unit 139 that presents the subject with suggested information to improve the subject's frailty status based on the learning results of the learning processing unit 138. In this aspect, this embodiment may evolve into a platform that learns from data of a large number of users and provides personalized health management. The learning processing unit 138 of server 100 can acquire patterns that were not visible at the individual level, new risk factors, or universal insights (general models) regarding the effectiveness of interventions (suggestions) by using machine learning (deep learning) on ​​anonymized factor analysis data (state changes, factors, background, intervention results, etc.) collected from multiple subjects. The suggestion processing unit 139 may utilize these learning results to generate and present "suggestion information" (e.g., supplement recommendations, dietary suggestions, exercise advice) optimized for the situation (attributes, state, factors) of each individual subject.

[0091] The "analysis data of the factor analysis unit" refers to data representing the entire analysis process (input data, intermediate products, analysis results) by the factor analysis unit 135, and may be interpreted as "case dataset," "training data," "analysis log," etc. The "learning processing unit" refers to a functional unit that builds and updates machine learning models from accumulated data, and may be interpreted as "machine learning engine," "AI model construction unit," "general-purpose model learning processing unit," etc. The action of "learning" refers to extracting patterns and rules from data and improving the model, and may be interpreted as "training the model," "acquiring knowledge," "optimizing," etc. The "proposal processing unit" refers to a functional unit that generates advice for the subject based on the learning results, and may be interpreted as "advice generation unit," "recommendation engine," "intervention planner," etc.

[0092] This embodiment demonstrates a clear advantage: it allows for the integrated learning of anonymized data from numerous users on a cloud server, enabling simulations to be performed on new target individuals. In the flow chart of Figure 5, the learning processing unit 138 of server 100 is responsible for step S26 "(Large-scale) data group learning," and the proposal processing unit 139 is responsible for step S27 "Customer proposal." In the inventor's materials, a specific example of this "proposal" is the recommendation of health foods ("Recommended health foods," "Data shows that Mr. A's complexion has improved by about 10% since starting the health foods"). In the future, this platform has the potential to develop into an ecosystem in which various third-party companies participate, such as food companies offering health foods and meal suggestions, pharmaceutical companies recommending supplements, and insurance companies conducting risk assessments.

[0093] In the ninth aspect of server 100, the learning processing unit 138 may acquire information indicating the change in the subject's frailty state in response to the proposal result of the proposal processing unit 139, and may further learn from the acquired data. This aspect allows for the establishment of a continuous PDCA (Plan-Do-Check-Action) cycle in which the system evolves on its own. After the subject executes (Do) the "proposal" (intervention, Plan) made by the proposal processing unit 139 of server 100, the system continues sensing (primary analysis and secondary analysis) and monitors and acquires the subsequent "change in frailty state" (intervention effect, Check). The learning processing unit 138 "further learns" (Action) from this "intervention result" data, thereby accumulating knowledge in the memory unit 120 that "this proposal was effective / ineffective for this subject." As a result, the learning model and proposal logic are continuously improved, becoming smarter the more they are used, and the accuracy (intervention effect) of the proposals made by the proposal processing unit 139 increases.

[0094] Furthermore, "information showing changes in the proposed results" refers to, for example, time-series data of the subject's characteristics and frailty status after the proposal (intervention) has been made, and may be reinterpreted as "intervention effect data," "feedback data," "A / B test results," etc. The action of "acquiring" refers to, for example, collecting data on changes in state after the intervention and inputting it into the learning process, and may be reinterpreted as "collecting," "observing," "receiving feedback," etc. The action of "further learning" refers to, for example, updating and improving an existing model using new data (intervention results), and may be reinterpreted as "retraining the model," "continuous learning," "reinforcement learning," etc.

[0095] This embodiment provides an advanced platform that goes beyond mere one-way suggestions (e.g., Japanese Patent 7349759, Japanese Patent Application Publication No. 2025-155588, etc.) and extends to "effectiveness verification." For example, after terminal 200 recommends (suggests) a health food or supplement (S27 in Figure 5), the subject consumes it, and subsequently, facial color data (primary analysis) and subjective experience (verification results) are continuously acquired. The learning processing unit 138 of server 100 may "link supplement / facial color / physical condition / subjective experience and store it in personal data" if the subject agrees or is convinced of the effects of the health food or supplement based on subjective experience, and "feed back the evidence of the relationship that there was an effect based on subjective experience to the cloud for retraining." As a result, the suggestion processing unit 139 will be able to generate more accurate suggestions in the future (e.g., "61% of people the same age as Person A say this supplement is effective...data change indicating that Person A's physical condition is good").

[0096] (5) Example of data flow Figure 5 shows an example of a detailed data flow in the information processing system 1 according to this embodiment. This flow is a more concrete representation of the overview flow (S1-S3) in Figure 2, and shows in detail the coordination between each step and the flow of data.

[0097] First, the process begins with a primary analysis on the subject's terminal 200. In step S11, the image input unit 301 acquires an image (still image or video) of the subject, and in step S12, the voice input unit 302 acquires the subject's voice (conversation, monologue, etc.). This input data is collected naturally during everyday activities.

[0098] Next, in step S13, the image analysis unit (part of the primary analysis unit 281) performs analysis based on the image data. This analysis not only measures body temperature and complexion, but also uses deep learning models such as convolutional neural networks (CNNs) to detect the positions of facial landmarks (eyes, nose, mouth, etc.) and tracks changes in their relative arrangement (e.g., frown lines, drooping of the corners of the eyes), movement of the corners of the mouth, speed of mouth opening and closing, and tongue movement patterns over time. This allows for the estimation of stress levels from subtle changes in facial expression and the extraction of features related to signs of oral frailty.

[0099] In parallel, in step S14, the speech analysis unit (part of the primary analysis unit 281) performs a multifaceted analysis based on the speech data. First, speech recognition technology is used to convert the speech into text data, and natural language processing (NLP) technology (e.g., RNN or Transformer-based models) is applied. This analyzes the frequency and context of frequently used words, unique expressions (characteristic words), and emotional words (positive / negative words) in conversation, and obtains linguistic features related to mental stress, depressive tendencies, or changes in cognitive function (upper part of S14 in Figure 5). At the same time, acoustic analysis is also performed on the speech data itself. Frequency analysis and spectral analysis using Fourier transforms are performed to quantify voice tone (pitch), voice fluctuations (jitter, shimmer), volume, voice quality (e.g., degree of hoarseness), speech rate, rhythm, and intonation (prosody). These acoustic features may reflect the subject's emotional state (excitement, depression, etc.), stress level, and even the state of their vocal organs, such as articulation and tongue movement (Figure 5, S14 lower panel).

[0100] The diverse features extracted in these primary analyses (S13, S14) are converted into meaningful numerical or categorical data (structured data). From a privacy (security) perspective, it is important to complete these primary analysis processes within terminal 200 as much as possible. Only the extracted feature data is passed on to subsequent processing, and the original image and audio data (raw data) are promptly discarded within the terminal or temporarily stored in strictly controlled local storage.

[0101] Next, a secondary analysis (variability analysis and accumulation analysis) is performed on terminal 200 or server 100. The secondary analysis unit (282 or 131) performs variation analysis on multiple different time scales based on the time-series data of features obtained in the primary analysis (e.g., data from the past few weeks to several months). In step S15, a short-term variation analysis is performed. This aims to capture daily variations in features (e.g., differences in facial color between morning and evening, changes in conversation speed during the day) and sudden changes (abrupt changes). In step S16, a medium-term variation analysis is performed, analyzing weekly average values, trends (increase / decrease), and periodicity. This allows for the understanding of patterns such as fatigue accumulating towards the weekend or tendencies for increased activity on specific days of the week. In step S17, a long-term variation analysis is performed to evaluate longer-term trends in change and the influence of seasonality. Furthermore, in step S18, an individual standard variation analysis is performed. This is a process that statistically learns and identifies the subject's unique normal (healthy) variation pattern (baseline). For example, the mean and standard deviation of each feature are calculated from data from a stable period in the past, and individual normal ranges (e.g., mean ± 2 standard deviations) are set based on these. Furthermore, methods such as Bayesian estimation are used to model the probabilistic distribution of variability, taking into account not only the range but also the likelihood of variability (variability). In addition, sensitivity analysis of variability factors is performed to determine which variability of one feature is more likely to affect the variability of another feature, thereby gaining a deeper understanding of individual characteristics.

[0102] The execution location for these secondary analyses (S15-S18) is flexibly determined according to system requirements. For example, short-term fluctuation analysis (S15), which requires real-time processing, and individual baseline setting (part of S18) are performed on the terminal 200 side (secondary analysis unit 282) to minimize response delays. On the other hand, long-term analysis (S16, S17) using large amounts of historical data, sensitivity analysis (part of S18) requiring complex statistical models, or comparative analysis with other user data are performed by transferring data to the server 100 side (secondary analysis unit 131), which has abundant computing power and storage capacity. This division of processing reduces the load on edge devices while simultaneously utilizing the advanced analytical capabilities of the cloud.

[0103] Next, frailty estimation and subsequent coordination processing are mainly performed on server 100. In step S19, the frailty estimation unit 132 comprehensively evaluates multiple analysis results obtained by the secondary analysis unit (results from S15 to S18, i.e., daily, weekly, and monthly fluctuations, and the degree of deviation from individual criteria, etc.) and performs frailty estimation. This estimation uses rule-based judgment (e.g., "If the activity level is below the baseline for three consecutive weeks in weekly fluctuations, and a decreasing trend in conversation speed is observed in monthly fluctuations, it is judged as pre-frail") or pre-trained machine learning models (e.g., regression models or classification models that take multiple fluctuation indicators as input and output frailty stages and risk scores). At this time, the robustness and accuracy of the estimation are improved by combining fluctuation information captured on multiple feature items (e.g., complexion and articulation) and multiple time scales (e.g., daily abrupt changes and monthly accumulation).

[0104] In step S20, the change notification unit 134 processes a notification if the frailty estimation unit 132 estimates a change in the frailty state (especially signs of deterioration or increased risk). This notification information is transmitted to the subject's terminal 200 via the communication unit 110 and presented to the subject in step S21 via the image output unit 303 (e.g., message display on a smart mirror screen) or in step S28 via the voice output unit 304 (e.g., a message using synthesized voice). The content of the notification is adjusted according to the subject's condition and privacy settings (e.g., minor changes are notified only to the subject, while significant risks are notified to family members as well).

[0105] In step S22, the verification unit 137, in conjunction with the notification (S20), conducts a (user confirmation) verification (inquiry) with the subject. This is an interactive process to confirm whether the estimated changes match the subject's subjective symptoms and actual situation, and to explore the factors behind the changes. For example, questions such as "Q1: You seem a bit tired lately, how are you feeling?" and "Q2: Is there anything that comes to mind?" are presented through the screen (S21) or voice (S28) of the terminal 200. The subject responds through the voice input unit 302 (voice response) or operation input unit (such as answering with a choice on the touch panel). The content of the response (e.g., "A2: Come to think of it, I've recently started taking care of my husband...") is sent to the server 100 as a verification result and becomes important information for the subsequent factor analysis (S25).

[0106] In steps S23 and S24, the data acquisition unit 136 collects supplementary data other than images and audio in order to perform more precise factor analysis (S25) and machine learning (S26). In step S23, measurement data related to the subject's daily activities is acquired as behavioral input (e.g., steps, heart rate, sleep duration from wearable devices; frequency of toilet use, bathing time, water intake from in-home IoT sensors; smartphone usage logs, etc.). In step S24, basic attribute information of the subject is acquired as personal attribute input (e.g., age, gender, medical history, medication information, family structure, participation in social activities, etc.). This data is acquired through linkage with other systems or pre-entered by the user.

[0107] In step S25, the factor analysis unit 135 conducts an in-depth analysis to uncover the root causes of the change in frailty status (estimated in S19). This analysis integrates all available information as input, including primary analysis results (S13, S14), secondary analysis results (S15-S18), user verification results (S22), supplementary behavioral data (S23), and attribute data (S24). The factor analysis unit 135 analyzes the complex correlations and causal relationships between these multidimensional data using statistical methods (cocorrelation analysis, regression analysis, etc.) and more advanced machine learning techniques (causal inference models, Bayesian networks, deep learning-based feature importance calculation, etc.). This generates and evaluates hypotheses of specific causal chains, such as "the life event 'caring for a husband' obtained from the verification results is related to the 'increase in mental stress indicators' detected in the voice analysis and the 'decrease in sleep duration' confirmed in the behavioral data, ultimately leading to a worsening of the frailty status." The analysis results are stored as personal data in a database (storage unit 120).

[0108] In step S26, the learning processing unit 138 aggregates data collected from a large number of subjects (such as the individual factor analysis datasets obtained in step S25), processes it with privacy in mind (anonymization, statistical analysis, etc.), and then performs large-scale machine learning (especially deep learning). This allows for the modeling and discovery of general trends at the group level, hidden patterns, new risk factors, or the effects of specific interventions (proposals) that cannot be found through individual case analysis alone ((large-scale) data learning). Through this learning, the accuracy of the frailty estimation model and the factor analysis model themselves is continuously improved.

[0109] In step S27, the suggestion processing unit 139 generates personalized customer suggestions (suggestion information) to improve the frailty status, tailored to the current situation of the subject (estimated frailty status, analyzed factors, personal attributes and preferences, etc.), based on the learning results (refined model and acquired insights) from step S26. The suggestions cover a wide range of topics, including advice on improving lifestyle habits (e.g., "It seems you've been suffering from sleep deprivation. Why not try some relaxation techniques before bed?"), exercise recommendations, nutritional guidance, stress coping strategies, or information encouraging consultation with local support services or specialists. The generated suggestion information is transmitted to the subject's terminal 200 via the communication unit 110 and presented clearly through the screen (S21) and audio (S28).

[0110] This series of processes forms a closed loop. That is, the subject's response to the proposal (S27) (e.g., whether or not they performed the proposed action) and subsequent changes in state (which are continuously monitored again through sensing and analysis in S11-S19) become the subject of the next verification (S22) and factor analysis (S25), and the results are then fed back as learning (S26) data. In this way, the effectiveness of the proposal (intervention) is verified based on real data, and the results are reflected in updating the model and optimizing the next proposal, thereby realizing a continuous PDCA cycle in which the entire system self-improves.

[0111] (6) Specific examples Figures 6 and 7 illustrate specific operational examples of this embodiment, particularly the relationship between the frailty progression process, variability analysis, and risk assessment.

[0112] Figure 6 illustrates a conceptual model showing that frailty is not a single event, but rather a gradual progression (vertical axis) over time (horizontal axis) involving a variety of interconnected factors. It also shows that there are individual differences in the degree of frailty progression (sensitivity), and that different progression patterns (types) may exist depending on the individual. This provides a basis for setting multiple time intervals in secondary analysis.

[0113] On the shortest timescale, "daily," relatively common factors such as "sleep deprivation" and "lack of daily exercise" can cause "autonomic nervous system imbalances." This can be observed as short-term fluctuations in features obtained from the primary analysis (S15 in Figure 5), such as a temporary worsening of complexion in the morning, hoarseness, or a slight increase in negative words during conversation. Changes at this stage are often considered to be within the range of reversible physical discomfort that can be recovered with sufficient rest, etc. (see Type C in Figure 6).

[0114] However, if these daily ailments persist or recur frequently without being resolved, the effects begin to appear on a "weekly" timescale, potentially manifesting as "physical damage" such as accumulated fatigue and weakened immunity. This can be captured as medium-term variations in the features, such as weekly fluctuations (S16 in Figure 5), for example, the continuation of an average poor complexion throughout the week, a persistent decrease in conversation speed, or "changes in conversation" (e.g., an increase in the number of times one stumbles over words), "worsening of oral / skin condition," or "decreased tongue movement / swallowing," as shown in the example in Figure 5.

[0115] Furthermore, if these physical ailments or external stressors (e.g., life events such as "caring for a husband" as suggested in A2 of Figure 5) persist on a "monthly" basis without being resolved, the "stress accumulation" worsens, increasing the risk of developing into "psychological damage" (see upper-middle region of Figure 6). This can be observed as long-term variations in features, such as monthly fluctuations (S17 in Figure 5), such as a persistently high frequency of negative word use throughout the month, a lack of facial expression (a decrease in "changes in facial expression" in Figure 5), a darkening of voice tone ("changes in voice tone" in Figure 5), or a change in color choices in clothing (e.g., choosing only dark colors, see Figure 5) (see Type B in Figure 6).

[0116] Furthermore, if these physical and mental damages accumulate and progress over "several months," the individual's motivation and activity levels decline, and more visible changes may appear, such as "decreased activity" (e.g., a significant decrease in the frequency of going out, avoidance of social interaction) and "poor hygiene" (e.g., decreased frequency of bathing and changing clothes, indifference to personal grooming) (see upper area of ​​Figure 6). Examples include "not looking in the mirror," "decreased frequency / duration of brushing teeth," "messy hair," "misbuttoning clothes," "wearing the same clothes repeatedly," "repeating the same conversations," and "dirty clothes." If this condition persists, there is a risk of ultimately reaching more serious and irreversible conditions such as "depression" and "dementia," i.e., a highly advanced stage of frailty. Elderly people living alone, in particular, who have little social support, tend to fall into this negative spiral (see Type A of Figure 6). Furthermore, regarding the criteria for the progression of frailty on the vertical axis of Figure 6, in addition to the changes and frequency of changes mentioned above, it is also possible to set criteria using the diagnostic criteria for frailty (J-CHS criteria) or the 25 items of "Ministry of Labour Notification No. 197 of 2015 (Basic Checklist Notification)," which conducts a multifaceted assessment of physical, mental, and social aspects.

[0117] In this embodiment, the secondary analysis unit (282 or 131) may take into account the multi-stage and multi-timescale nature of frailty progression and perform fluctuation analysis over multiple time intervals, such as short-term (S15), medium-term (S16), long-term (S17), and even several months. This makes it possible to evaluate more precisely and from multiple perspectives which stage of the frailty progression process the subject is currently closest to, on which time scale changes are most pronounced, and which factors (e.g., short-term stress or long-term decline in behavior) are accumulating. Furthermore, as shown in Figure 6, assuming that different sensitivities to frailty progression exist among subjects (e.g., Type A, Type B, Type C), it becomes possible to predict and estimate future progression trends (predictive analysis) at an earlier stage (e.g., weekly or monthly) by analyzing past data and trends.

[0118] Figure 7 shows a concrete example of how to analyze the fluctuations in data obtained from secondary analysis and connect them to risk assessment (part of frailty estimation) and subsequent actions (notification of fluctuations, data transfer, and proposal of countermeasures). Here, an approach is shown in which the analysis targets are managed as a step-by-step flow according to the nature of the fluctuations, such as "crisis analysis (large fluctuations)," "trend analysis (small / normal fluctuations)," and "predictive analysis."

[0119] "Trend analysis (small / normal fluctuations)" primarily focuses on "understanding the variability of short-term (e.g., daily) fluctuations." For this purpose, the standard fluctuation analysis unit 133 (corresponding to S18 in Figure 5) first uses AI (such as a machine learning model) to learn and set a range of "acceptable fluctuations (normal)" specific to each individual, i.e., a statistical baseline, from the subject's past short-term fluctuation (S15) data. This baseline (standard fluctuation) can be defined, for example, as a pair of higher limit values ​​(HLV) and lower limit values ​​(LLV) for each feature. It is desirable that these thresholds are not merely fixed values, but are adaptively updated in response to changes in the subject's state.

[0120] In the "crisis analysis (major fluctuation)" phase, if the latest features acquired daily (primary analysis results) deviate significantly from the baseline (LLV / HLV) set in the "trend analysis" (major fluctuation), or if they show an abnormal value that exceeds the past observation range, this is detected as a "sudden change." Based on this detection, the fluctuation notification unit 134 promptly issues a "sudden change notification" (alert) to the target user (see the wavy line flow in Figure 7). This type of processing, which requires real-time processing, is most efficiently performed on the terminal 200 side.

[0121] On the other hand, longer-term analysis results, such as "short-term daily fluctuations" identified in the "trend analysis," as well as medium-term fluctuations (S16) and long-term fluctuations (S17), are transferred and stored at the data storage location (for example, the cloud on server 100) (see the arrow from "personal terminal" to "cloud" in Figure 7). On the server 100 side, this stored data, along with "data from many subjects," is used to "assess accumulation trends such as stress" and "understand trends in frailty" as shown in Figure 6.

[0122] Furthermore, as part of "predictive analysis," the server 100 uses the captured "trends and predictions of the target individuals" to issue notifications via the fluctuation notification unit 134 (e.g., "It seems you've been a little tired these past few weeks") and to make "countermeasure suggestions" via the suggestion processing unit 139 (see the solid line flow in Figure 7). In this way, normal data (small / normal fluctuations) are handled in the flow from "trend analysis" to "predictive analysis" (solid line section) and linked to countermeasure suggestions. On the other hand, data with high urgency (large fluctuations) are handled in the "crisis analysis" flow (wavy line section) and linked to immediate notifications (notification of sudden changes).

[0123] This hierarchical processing and data management approach makes it possible to achieve a good balance between multiple objectives: suppressing resource load and network traffic on the terminal side, meeting privacy protection requirements, and ensuring that significant changes in health conditions are not overlooked and addressed early.

[0124] (7) Other embodiments Although embodiments have been described in detail above, these embodiments are not limited to the examples described above.

[0125] For example, in the above embodiment, a smartphone and a smart mirror were given as examples of edge devices, but any other device capable of sensing the image or voice of a subject (e.g., a tablet, PC, smart speaker, wearable camera, monitoring camera, robot, in-car infotainment system, etc.) can also be used as terminal 200.

[0126] Furthermore, the features obtained in the primary analysis are not limited to the examples above (complexion, facial expression, voice quality, articulation, etc.). For example, in image analysis (S13), the color, stains, wrinkles, or incorrect buttoning of the subject's clothing may be detected and used as features. Additionally, the frequency and duration of the subject looking in the mirror, the number and duration of brushing teeth, and the condition of their hair (whether it is messy or not) may be used as features through image recognition. It is also possible to refer to evaluations of the relationship between dementia BPSD and clothing, and assessments of IADL for the elderly. Similarly, in voice analysis (S14), the content of the conversation (transcribed) may be used as a feature to determine whether the same conversation is being repeated, or the frequency of use of demonstrative pronouns such as "that" and "it" (which may suggest a decline in cognitive function). It is also possible to refer to dementia assessments such as the Hasegawa Dementia Scale (HDS-R) and MMSE, as well as the Frailty Treatment Guidelines (supervised by the Japan Geriatrics Society).

[0127] Furthermore, the behavioral data (S23) acquired by the data acquisition unit 136 may include a wider variety of life log data, such as the usage status (number of uses, duration) of the water in a washroom where a smart mirror is installed, as in Japanese Patent Application Publication No. 2025-155588, or the frequency / duration of toilet use, the frequency / duration of bathroom use, nocturia (increased number of times of bedwetting), the condition of stool (stool scan) obtained from other IoT sensors, or sleep data (sleep deprivation), activity level data (lack of exercise) obtained from smartwatches, etc. These data are closely related to the progression process of frailty shown in Figure 6 and the various changes shown in Figure 5, and it is expected that the estimation accuracy and the quality of the suggestions (S27) can be improved by using them in factor analysis (S25) and learning (S26).

[0128] Furthermore, although the above embodiment mainly describes an example where the server 100 resides on the cloud, some or all of the functions of the server 100 may be deployed on-premises (for example, a home server in a household or a local server in a hospital). From the standpoint of privacy protection, as suggested in Figure 7, it is more desirable to convert (text, anonymize, structure) the changes in images and sounds (primary analysis results) into a format that does not identify individuals (for example, numerical data such as "complexion is 10% darker than normal" or "speech speed is 5% slower than normal," or text data such as "slightly tired expression" or "slightly unclear articulation") and send it to the server 100. This allows the server 100 to perform statistical analysis, learning, and estimation without handling raw image and sound data at all, by effectively utilizing, for example, AI generation after text conversion, thereby enabling not only privacy and security but also high-speed and sequential processing such as real-time. Furthermore, by using S25 (personal data) learning, reliable detection and rapid notification become possible without false positives in times of danger or sudden changes. Furthermore, by using S26's data swarm learning, it becomes possible to immediately provide suggestions for effective and useful countermeasures in the aforementioned dangerous or sudden changes in condition, even if the customer (including family members and caregivers) lacks specialized knowledge.

[0129] In the embodiment described above, at least a portion of the processing performed by the server 100 may be modified to be performed on the terminal 200 side. In this case, at least a portion of each part included in the processing unit 130 may be moved to the terminal 200, or at least a portion of the DB included in the storage unit 120 may be moved to the terminal 200.

[0130] The operation flow and operation examples in the above-described embodiments do not necessarily have to be executed chronologically in the order shown in the flowchart. For example, the steps in the operation may be executed in a different order than that shown in the flowchart, or they may be executed in parallel. Also, some of the steps in the operation may be deleted, or further steps may be added to the process.

[0131] A program may be provided that causes a computer (information processing device) to perform the operations according to the above embodiment. The program may be recorded on a computer-readable medium. Using a computer-readable medium, it is possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient storage medium. The non-transient storage medium is not particularly limited, but may be a storage medium such as a CD-ROM or DVD-ROM.

[0132] The functions realized by the information processing system according to this embodiment may be implemented in a circuit or processing circuitry, including a general-purpose processor, an application-specific processor, an integrated circuit, an ASIC (Application Specific Integrated Circuit), a CPU (a Central Processing Unit), conventional circuits, and / or a combination thereof, which are programmed to realize the described functions. A processor, including transistors and other circuits, is considered a circuit or processing circuitry. A processor may be a programmed processor that executes a program stored in memory. In this specification, a circuit, a unit, or a means is hardware programmed to realize or execute the described functions. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to realize or execute the described functions. If the hardware is a processor which is considered a type of circuit, then the circuit, a means, or a unit is a combination of hardware and software used to constitute such hardware and / or processor.

[0133] As used herein, the terms “based on” and “according to” do not mean “based solely on” or “according solely to” unless otherwise specified. “Based on” means both “based solely on” and “based at least partially on.” Similarly, “according to” means both “based solely on” and “according at least partially to.” Furthermore, the terms “include,” “comprise,” and their variations do not mean to include only the listed items, but may include only the listed items, or may include additional items in addition to the listed items. Also, the term “or” as used herein is not intended to mean exclusive OR. Where articles are added by translation, such as a, an, and the in English, these articles are considered plural unless the context clearly indicates otherwise.

[0134] Although the embodiments have been described in detail above with reference to the drawings, the specific configuration is not limited to those described above, and various design changes can be made without departing from the gist of the invention. [Explanation of Symbols]

[0135] 1… Information processing system 5…Network 100... Server 110... Communications Department 120...Storage section 130... Processing Unit 131…Secondary analysis department 132...Frailty Estimation Unit 133...Standard Variation Analysis Department 134...Fluctuation Notification Section 135…Factor Analysis Department 136...Data acquisition unit 137…Verification Department 138...Learning Processing Unit 139... Proposal Processing Section 200… Terminal 210...Image input section 220...Voice input section 230...Image output unit 240...Audio output section 250... Communications Department 260... Operation input section 270...Storage section 280... Processing Unit 281…Primary Analysis Department 282…Secondary Analysis Department 300...User Interface 301...Image input section 302...Voice input section 303...Image output unit 304...Audio output section

Claims

1. A primary analysis unit obtains feature quantities indicating the state of the subject based on input data obtained from at least one of an image input unit that obtains an image of the subject and an audio input unit that obtains the voice of the subject, A secondary analysis unit performs a time-series analysis of the aforementioned feature quantities and identifies the fluctuation status of the feature quantities in each of several time intervals with different time lengths. The system includes a frailty estimation unit that, based on a combination of the fluctuations in each of the aforementioned multiple time intervals, distinguishes between temporary fluctuations in the subject's condition and progressive changes in the condition over the medium or long term, and estimates the subject's frailty state. Information processing system.

2. The primary analysis unit is provided on the edge device on the subject's side, The secondary analysis unit and the frailty estimation unit are provided on a server on the Internet. The input data is not transmitted from the edge device to the server, and the analysis results of the primary analysis unit are transmitted from the edge device to the server. The information processing system according to claim 1.

3. The primary analysis unit and the secondary analysis unit are provided on the edge device on the subject's side. The frailty estimation unit is located on a server on the Internet. The input data is not transmitted from the edge device to the server, and the analysis results from the secondary analysis unit are transmitted from the edge device to the server. The information processing system according to claim 1.

4. The analysis results from the primary analysis unit and / or the secondary analysis unit are transmitted as text from the edge device to the server. The information processing system according to claim 3.

5. The frailty estimation unit estimates the frailty status of the subject based on the analysis results of the primary analysis unit and the analysis results of the secondary analysis unit. The information processing system according to claim 1.

6. The secondary analysis unit includes a standard variation analysis unit that identifies the standard variation of the subject for each of the plurality of time intervals, The frailty estimation unit estimates the deterioration of the frailty state based on whether the fluctuation of the feature in any of the plurality of time intervals exceeds the standard fluctuation. The information processing system according to claim 1.

7. The system further includes a change notification unit that notifies at least one of the subject and their related parties in response to the frailty estimation unit estimating a change in the frailty state. The information processing system according to claim 1.

8. The system further includes a factor analysis unit that analyzes the factors causing the change in the frailty state, based on the frailty estimation unit's estimation of the change in the frailty state. The information processing system according to claim 1.

9. The system further includes a data acquisition unit that acquires at least one of the following: measurement data relating to the subject's behavior and attribute data relating to the subject's attributes. The factor analysis unit analyzes the factors causing the change in the frailty state based on at least one of the measurement data and attribute data acquired by the data acquisition unit. The information processing system according to claim 8.

10. The system further includes a verification unit that, in response to the frailty estimation unit estimating a change in the frailty state, performs an inquiry process with the subject to confirm the subject's health status. The factor analysis unit analyzes the factors causing the change in the frailty state based on the verification results from the verification unit. The information processing system according to claim 8.

11. A learning processing unit that learns from the analysis data of the factor analysis unit for each of the multiple subjects, including the aforementioned subject, using machine learning, The system further includes a suggestion processing unit that, based on the learning results of the learning processing unit, presents the subject with suggested information to improve the subject's frailty condition. The information processing system according to any one of claims 8 to 10.

12. The learning processing unit acquires information indicating the change in the subject's frailty status in relation to the proposal results of the proposal processing unit, and further learns from the acquired data. The information processing system according to claim 11.

13. The learning processing unit learns using generative AI. The information processing system according to claim 11.

14. An information processing method performed by an information processing system, A primary analysis step to obtain feature quantities indicating the state of the subject based on input data obtained from at least one of an image input unit that acquires an image of the subject and an audio input unit that acquires the voice of the subject, A secondary analysis step involves performing a time-series variation analysis of the aforementioned feature quantities to identify the variation status of the feature quantities in each of several time intervals with different time lengths, The frailty estimation step involves estimating the frailty status of the subject by distinguishing between temporary changes in the subject's condition and progressive changes in the condition over the medium or long term, based on a combination of the aforementioned fluctuations in each of the aforementioned multiple time intervals. Information processing methods.

15. In the information processing system, A primary analysis step to obtain feature quantities indicating the state of the subject based on input data obtained from at least one of an image input unit that acquires an image of the subject and an audio input unit that acquires the voice of the subject, A secondary analysis step involves performing a time-series variation analysis of the aforementioned feature quantities to identify the variation status of the feature quantities in each of several time intervals with different time lengths, The system performs a frailty estimation step, which involves estimating the frailty status of the subject by distinguishing between temporary changes in the subject's condition and progressive changes in the condition over the medium or long term, based on a combination of the aforementioned fluctuations in each of the aforementioned multiple time intervals. program.

Images