Posture estimation display system

Abstract

The present application can accurately estimate a patient's posture for a long time and provide a UI that can visually recognize measurement data for a long time. A posture estimation display system includes a first sensor worn by a user and measuring air pressure, a second sensor worn by the user or placed in a space where the user is and measuring air pressure, a posture estimation unit that acquires an air pressure difference between the air pressure of the first sensor and the air pressure of the second sensor and estimates the posture of the user based on the air pressure difference, and a presentation unit that presents a proportion of each of different postures of the user estimated from the air pressure difference acquired from the first and second sensors in a predetermined period.

Description

Technical Field

[0001] This invention relates to an attitude estimation display system. Background Technology

[0002] Patent document 1 discloses an action state monitoring system, which includes: an acquisition unit that acquires sensing information from a sensor installed on a target part; an installation direction detection unit that detects the installation direction of the sensor; and a control processing unit that establishes a corresponding association with the installation direction of the sensor and outputs information associated with the sensing information.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2022-034450 Summary of the Invention However, during the relatively long periods of hospitalization for rehabilitation training, when accelerometers are worn on the body, their orientation can sometimes change unexpectedly if the patient moves. In such cases, accurate posture estimation cannot be performed over extended periods. Therefore, it is necessary to accurately estimate the patient's posture over long periods and present the measurement data using an optimal user interface (UI) that is easily visually recognizable to healthcare professionals.

[0004] The present invention was made to solve this problem, and its purpose is to provide a posture estimation display system, etc., which can accurately estimate the patient's posture over a long period of time and provide prompts with an optimal UI that can visually recognize long-term measurement data.

[0005] An embodiment of the present invention relates to a posture estimation and display system comprising: a first sensor worn by a user and measuring air pressure; a second sensor worn by the user or placed stationary in the space where the user is located and measuring air pressure; a posture estimation unit that acquires the air pressure difference between the air pressure of the first sensor and the air pressure of the second sensor, and estimates the user's posture based on the air pressure difference; and a prompting unit that prompts the proportion of different postures of the user estimated based on the air pressure difference acquired from the first sensor and the second sensor during a specified period.

[0006] Therefore, it is possible to accurately estimate the patient's posture over a long period of time, and to provide prompts with an optimal UI that can visually recognize long-term measurement data.

[0007] The prompting unit can, for each segment divided within the specified period, indicate the proportion of different postures according to their categories. In this case, the display order of the proportions of the different posture categories in each segment is the same, so that it is possible to visually identify how the proportions of the different posture categories shift through the consecutive segments.

[0008] Therefore, the user interface can improve visual recognizability by using time as a hierarchical structure (e.g., 24 hours and 1 hour). Furthermore, it can visually discern how the proportions of different posture categories shift over time.

[0009] The prompting unit can correspond to the plurality of intermediate segments that divide the specified period, and prompt at least one of the weather information and air pressure information of the surrounding area.

[0010] This allows us to verify the reliability of the data and make appropriate corrections.

[0011] Invention Effects According to the present invention, a posture estimation display system or the like can be provided, which can accurately estimate a patient's posture over a long period of time and can provide prompts with an optimal UI that can visually recognize long-term measurement data. Attached Figure Description

[0012] Figure 1 This is a block diagram illustrating the structure of the attitude estimation display system.

[0013] Figure 2 This diagram illustrates two typical configuration modes of the first and second sensors used in the attitude estimation display system.

[0014] Figure 3 This is a diagram illustrating the method for estimating the posture of a patient using the first and second sensors.

[0015] Figure 4 This is a diagram illustrating an example of a user interface for a pose estimation display system.

[0016] Figure 5 This is a diagram illustrating an example of a user interface for a pose estimation display system.

[0017] Figure 6 This is a diagram illustrating an example of a user interface for a pose estimation display system. Detailed Implementation

[0018] The present invention will now be described through embodiments thereof, but the invention as described in the claims is not limited to these embodiments. Furthermore, not all structures described in the embodiments are necessarily necessary means to solve the problem. For clarity, the following descriptions and drawings have been appropriately omitted and simplified. In the drawings, the same symbols are used to label the same parts, and repeated descriptions have been omitted as necessary.

[0019] In the following embodiments, for convenience, they are described in multiple sections or embodiments as necessary. Unless otherwise stated, they are not unrelated to each other, and one is a variation, application example, detailed description, or supplementary description of another. Furthermore, in the following embodiments, when referring to the quantity of elements (including number, value, quantity, range, etc.), the quantity is not limited to that specific quantity, except as otherwise stated or where it is clearly limited to a specific quantity in principle. It can be more than or less than that specific quantity.

[0020] Furthermore, in the following embodiments, unless otherwise stated or clearly necessary in principle, the constituent elements (including operation steps, etc.) are not necessarily essential. Similarly, in the following embodiments, when referring to the shape, positional relationship, etc., of constituent elements, etc., it substantially includes those that are similar or close in shape, except as otherwise stated or clearly not considered in principle. The same applies to the quantities (including number, value, quantity, and range) mentioned above.

[0021] Figure 1 This is a block diagram illustrating the structure of a posture estimation and display system. The posture estimation and display system 1 estimates the user's posture over a specified period and displays the estimated posture and its progression in a manner easily recognizable to medical personnel or others. The user may be, for example, a hospitalized patient undergoing rehabilitation training, but is not limited to this. The specified period may be, for example, a specific period of the patient's hospitalization (including training and hospital stay), but is not limited to this. The specified period may be, for example, various periods of more than 2 hours, more than 3 hours, more than 8 hours, more than 1 day, or more than 3 days. Even patients with the same impairment and requiring rehabilitation training who undergo the same training may experience significant differences in their recovery levels. Therefore, the posture estimation and display system can also be used during hospital stays outside of training periods to visualize and analyze changes in the patient's posture or activity levels.

[0022] The attitude estimation display system 1 includes at least a first sensor 2 and a second sensor 3, an attitude estimation device 10, and a display 4. The first sensor 2 may be a barometric pressure sensor worn on the user's body to measure air pressure, or may include the barometric pressure sensor. The first sensor 2 may be a wearable device or may have a mounting part (e.g., a belt, strap, etc.) for installation at any location on the body. In some embodiments, the first sensor 2 may include the function of measuring three-axis acceleration and angular velocity. Therefore, the angles of each axis can also be calculated. Furthermore, the angles with adjacent sensors can also be measured. The display is not limited to a computer, but may be a display device of various electronic devices such as tablet computers and smartphones.

[0023] The second sensor 3 is a wearable device worn on the user's body that measures air pressure. Alternatively, the second sensor 3 can be stationary in the user's space to measure air pressure. For example, the second sensor 3 can be placed on the floor of the user's space, or on a table or chair. "Stationary placement" can mean the sensor is fixed in place and does not move, or it can mean that it is isolated and cannot be moved by anyone. The second sensor 3 measures a reference air pressure used to estimate the user's posture, and therefore can also be called a reference air pressure sensor. In some embodiments, the second sensor 3 may include the function of measuring three-axis acceleration and angular velocity. This allows the calculation of the angles of each axis. Furthermore, it allows the measurement of the angles with adjacent sensors.

[0024] Both the first sensor 2 and the second sensor 3 are capable of measuring air pressure. Therefore, for example, when a patient wears the device and has undergone training, even if the orientation of the first sensor 2 or the second sensor 3 changes unexpectedly, the posture can be accurately estimated based on the air pressure difference without relying on the orientation of the sensors.

[0025] The first sensor 2 and the second sensor 3 are configured to communicate bidirectionally with the attitude estimation device 10 via a wired or wireless network. The first sensor 2 and the second sensor 3 are typically configured to communicate bidirectionally with the attitude estimation device 10 via short-range wireless communication technologies such as Bluetooth (registered trademark) or Near Field Communication (NFC).

[0026] The attitude estimation device 10 is a computer equipped with a processor and memory. The control unit of the attitude estimation device 10 has a functional arithmetic unit that executes each of the subdivided processes by reading a stored program. Specifically, the control unit includes an attitude estimation unit 11 and a prompting unit 12. The attitude estimation unit 11 acquires the pressure difference between the air pressure of the first sensor 2 and the air pressure of the second sensor 3, and estimates the user's attitude based on the pressure difference. The prompting unit 12 (via display 4) prompts the cumulative time or cumulative percentage of each of the user's different attitudes estimated from the pressure difference acquired from the first sensor 2 and the second sensor 3 within a specified period. The display 4 is connected to the attitude estimation device 10 via a wired or wireless network. These functions can be utilized in various electronic devices by installing an application.

[0027] Figure 2This diagram illustrates two typical configuration modes of the first and second sensors used in the attitude estimation display system. In configuration mode (1), the first sensor 2 is mounted on the back of the user U, and the second sensor 3a is mounted on the shin of the user U's left foot. This allows the air pressure near the back of the first sensor 2 to be measured, and the air pressure near the shin of the user U to be measured by the second sensor 3a. Thus, the pressure difference between the first sensor 2 and the second sensor 3a can be obtained, and the user's attitude can be estimated based on this pressure difference.

[0028] Furthermore, this configuration mode is only an example, and various variations can be considered. The first and second sensors can be positioned at different heights when the user is standing. For example, the first sensor 2 can be located on any part of the upper body, such as the chest, stomach, upper arm, shoulder, or neck. The first sensor 2 can also be located on the torso. Furthermore, the second sensor 3a, which measures the reference air pressure, can be located on any part of the lower body, such as the shin or calf of the right foot, ankle, instep, or toes. The second sensor 3a can also be located on the thigh or calf.

[0029] In configuration mode (2), the position of the first sensor 2 is the same as in configuration mode (1), but the second sensor 3b is positioned in a location that is not a body part. The second sensor 3b can be built into an external terminal 30. The external terminal can be any terminal such as a smartphone, tablet, or laptop. The second sensor 3b is placed statically in the user's space to measure the reference air pressure. The second sensor 3b can be placed on a table, chair, floor, or other surface in the user's space.

[0030] Furthermore, in Figure 2 In the configuration modes (1) and (2) shown, a first sensor and a second sensor are provided, but... Figure 2 Other areas indicated by the dashed lines may also be equipped with additional sensors. In some embodiments, the first sensor 2, the second sensor 3, and the additional sensors may include functions for measuring the acceleration and angular velocity of the three axes, thereby enabling the calculation of the angles of each axis. Furthermore, it is also possible to measure the angles formed with adjacent sensors.

[0031] Figure 3 This diagram illustrates the method for estimating the patient's posture using both the first and second sensors. Figure 3 The upper part shows typical posture examples that can be determined based on pressure difference (PD), namely, standing, sitting, and lying down, but the invention is not limited to these. Figure 3 As shown, the pressure difference with respect to sensors 1 and 2 is greatest when standing. The pressure difference with respect to sensors 1 and 2 is smallest when lying down. The pressure difference in the sitting position is intermediate between standing and lying down. Figure 3 The figure below shows the shift in air pressure difference with the first and second sensors. Thus, the posture estimation device 10 can determine whether the patient's posture is standing, sitting, or lying down based on the air pressure difference with the first and second sensors within a specified period.

[0032] Figures 4-6 These are diagrams illustrating various examples of user interfaces for attitude estimation display systems.

[0033] These user interfaces are designed to provide an easy overview of long-term measurement data, to allow for detailed information at any point in time, and to enable users to grasp the results in terms of individual time units.

[0034] exist Figure 4 In the lower region 41, the overall measurement data over a long period of time is shown. Figure 4 Area 411. In this example, regarding the whole (representing 24 hours, or 1 day), the cumulative time and proportion are displayed according to different postures such as walking, standing, sitting, and lying down. Figure 5 In the lower region 41, the specified period for long-term measurement data (in Figure 5 In area 412 (16:00-16:59), the cumulative time and percentage are displayed according to different postures such as walking, standing, sitting, and lying down. That is, users can select either the "overall" or "specified period" of long-term measurement data via input devices (keyboard, touch panel, mouse, etc.). Figure 4 and Figure 5 The image shows a bar chart representing the cumulative time and proportions according to different orientations. Figure 6 In the lower region 41, a circular diagram (region 413) is shown to represent the cumulative time and proportion according to different postures. However, the present invention is not limited to these, and the cumulative time and proportion can also be represented for different postures by various other preferred methods.

[0035] exist Figure 4 Above region 41 is region 42, used to confirm detailed information at any given time. Region 42, for each segment (e.g., 1 hour) of a defined period (e.g., 1 day, January 16, 2025), displays a histogram showing the proportion of the user's posture according to different posture categories (e.g., walking, standing, sitting, and lying down). This is achieved across multiple consecutive segments (…). Figure 4 In region 427 (3 hours from 2 PM to 4 PM), the display order of the proportions of the different posture categories is the same in each segment (every hour from 2 PM to 4 PM) so that the proportions of the different posture categories can be visually discerned as they shift (i.e., in...). Figure 4 In this case, the order from top to bottom is walking, standing, sitting, and lying down.

[0036] Thus, the user interface involved in this invention becomes a hierarchical structure based on time (e.g., 24 hours and 1 hour), which can improve visual recognizability.

[0037] In some implementations, weather information for each time period can be displayed below area 427. Figure 5 (Region 428). (Sequence of sunny, cloudy, and rainy). That is, it can provide at least one of the following weather and air pressure information for the surrounding area corresponding to each of the multiple mid-segments (1 hour each from 2 PM to 4 PM) that divide the specified period: Weather information for the surrounding area can be obtained from external information such as weather information media or news. Air pressure information for the surrounding area can be obtained from external information such as weather information media or news, rather than from the first and second sensors.

[0038] The weather and air pressure information for the surrounding area is intended to inform users of the impact of attitude estimation based on air pressure differences. In particular, in configuration mode (2), if the user moves relative to the external terminal used to measure the reference air pressure in the altitude direction (e.g., at another level), accurate attitude estimation based on the air pressure difference may sometimes be impossible. Furthermore, it may be difficult to determine whether the reference air pressure changes due to this movement or due to changes in the external environment (weather). Therefore, the air pressure data measured by the external terminal 30 may need to be saved to verify the reliability of the data and to make appropriate corrections. Therefore, the user interface according to this invention displays the weather and air pressure information for the surrounding area.

[0039] exist Figure 4 The area above region 42 shows region 43, which represents further details at any given time. The proportions of the selected portion in region 42 (in this example, 1 hour from 16:00) are arranged in 60 increments per minute (for a total of 60 minutes).

[0040] In some implementations, as shown in region 43, the proportion of a user's posture can be displayed in the form of a histogram for each smaller segment (e.g., 1 minute) that is further subdivided into a medium segment (e.g., 1 hour). Across multiple consecutive smaller segments, the manner in which the proportions of each posture category shift can be visually identified, with the proportions of the different posture categories displayed in the same order across each smaller segment (i.e., in...). Figure 4 In this case, the order from top to bottom is walking, standing, sitting, and lying down.

[0041] Regarding the time-series measurement data shown in area 43, a list of portions within a specified period (e.g., 1 hour) can be displayed via an input device. The list can, for example, indicate the start time, the estimated posture category, and the duration. For instance, the list could show “16:00:00 Sitting (00:03:12), 16:03:12 Standing (00:00:10), 16:03:22 Sitting (00:00:01), 16:03:23 Lying down (00:05:56)”...“16:50:00 Sitting (00:03:12), 16:53:12 Standing (00:00:10), 16:53:22 Sitting (00:00:01), 16:53:23 Lying down (00:05:56)”. The starting point of the list can be selected by moving the progress bar 431.

[0042] Furthermore, the list can record weather information or air pressure information for the surrounding area. In some embodiments, at least one of the weather information and air pressure information for the surrounding area can be displayed corresponding to the plurality of small segments obtained by dividing the middle segment.

[0043] In other implementations, such as Figure 5 As shown, the occurrence of standing up and sitting down can be indicated by markings (e.g., triangles) on at least a portion of region 432 representing time-series measurement data. The number or frequency of standing up and sitting down can also be indicated by the presence or absence of markings (e.g., triangles). Abnormal standing up or sitting down can be represented by color differences. Indications regarding the occurrence of standing up and sitting down can be displayed outside region 43 using triangles or arrows to indicate the time of occurrence.

[0044] exist Figure 4 The upper side of region 43 shows region 44, which represents time-series measurement data for a specified period (e.g., 30 seconds) starting from any position specified by the progress bar 431 of region 43. In region 44, the horizontal axis represents time, and the progression from left to right—standing, sitting, walking, sitting, walking, sitting, standing, lying down, etc.—can be identifiable by shading or color differences. Furthermore, the progress bar 431 of region 43 can be moved arbitrarily horizontally via an input device, and correspondingly, the start of the time-series measurement data in region 44 can also be changed.

[0045] exist Figure 4 The left side of region 44 can display a video representing the state of walking.

[0046] exist Figure 4 The upper part is provided with an indicator Figure 2 Region 46 of either of the two configuration modes shown. Figure 4The dashed area 461 indicates that the measured air pressure data is in... Figure 2 The data obtained from the configuration mode (1). In particular, in Figure 4 In this configuration, a back sensor is used as the first sensor (2), and a calf sensor is used as the second sensor (3). This indicates that no external terminal is being utilized.

[0047] On the other hand, Figure 5 In region 46, the dashed area 462 indicates that the air pressure data was measured using... Figure 2 The configuration mode (2) is obtained. That is, it means that the back sensor is used as the first sensor 2 and the sensor built into the external terminal is used as the second sensor 3.

[0048] In addition, whether the measured air pressure data is acquired in configuration mode (1) or configuration mode (2) can be represented by various methods that can be understood by those skilled in the art (e.g., colored when the sensor is active and gray when the sensor is inactive).

[0049] As described above, especially in configuration mode (2), the reference pressure may change if the user moves relative to the external terminal used to measure the reference pressure in the altitude direction (e.g., to another floor). Therefore, the reliability of the measurement data using the pressure sensor can be communicated via the UI.

[0050] exist Figure 4 In this configuration, a back sensor is used as the first sensor 2, and a lower limb sensor is used as the second sensor 3. Therefore, a region 47 is provided to the right of the region 46 representing the configuration mode (1) to represent the air pressure of the back sensor and the air pressure of the lower limb sensor.

[0051] exist Figure 5 In this configuration, a back sensor is used as the first sensor 2, and a sensor built into an external terminal is used as the second sensor 3. Therefore, a region 47 is provided to the right of the region 46 representing the configuration mode (2) to represent the air pressure of the back sensor and the ambient air pressure of the external terminal.

[0052] exist Figure 6 In the diagram, air pressure information is shown in region 414 to the right of the circular graph (region 413) representing the cumulative time and scale according to different postures in region 41. Figure 6The barometric pressure information displays the air pressure of the area surrounding the external terminal, the air pressure of the reference sensor (lower limb sensor), and the air pressure of the worn sensor (back sensor). A full-body humanoid icon is also displayed to indicate the wearing positions of the first sensor (2) and the second sensor (3). This can be represented in color when using the external terminal and in gray when not using the external terminal. Weather information for the region at the time the measurement data was acquired can also be displayed to the right of the barometric pressure information. This barometric pressure or weather information is useful for verifying the reliability of data obtained using barometric pressure sensors.

[0053] When the first sensor 2 or the second sensor 3 worn by the user not only functions as a barometric pressure sensor but also measures three-axis acceleration, velocity, and angular velocity, specific patient behaviors (e.g., standing up, sitting down, or abnormal movements) can be detected. In this case, Figure 6 A labeled area 432 is provided outside area 43 to indicate when a specific behavior (e.g., standing up, sitting down, or abnormal movement) occurred. As described above, area 43 represents how the proportions of different posture categories shift, but the occurrence time of a specific behavior is indicated from outside area 43 by labels (e.g., arrows, triangles) in a time sequence. This improves the visual recognizability of long-term measurement data and enables analysis of the occurrence of specific behaviors that should be focused on during rehabilitation training.

[0054] exist Figure 6 Outside region 42, markers (e.g., arrows, triangles) and color tones can be used to indicate the frequency or number of occurrences of a specific behavior (e.g., standing up or unusual actions). For example, white can indicate that a specific behavior occurs 1 to 4 times, yellow can indicate 5 to 9 times, and red can indicate more than 10 times. This allows for analysis of when and how many times a specific behavior occurred.

[0055] exist Figures 4-6 In the diagram, region 43 indicates how the proportions of each of the different pose categories shift, but it can also only indicate the category of the pose with the highest proportion, or only the top few in descending order of proportion (e.g., 3).

[0056] As explained above, the user interface involved in this invention is optimized to make it easy for medical personnel to visually identify long-term measurement data and to easily analyze the patient's rehabilitation training status.

[0057] In some embodiments, a posture estimation and display method is provided. In this method, air pressure is acquired from a first sensor worn by the user, and air pressure is acquired from a second sensor worn by the user or stationary in the space where the user is located. The pressure difference between the air pressure of the first sensor and the air pressure of the second sensor is acquired. The user's posture is estimated based on the pressure difference, and the cumulative time or cumulative proportion of each of the different postures of the user estimated from the pressure difference acquired from the first and second sensors over a specified period is displayed. The posture estimation and display method can be incorporated into some or all of the features of the posture estimation and display system described above.

[0058] In some embodiments, a program is provided to enable a computer to execute the above-described attitude estimation and display method. The program can incorporate some or all of the features of the above-described attitude estimation and display system and program.

[0059] In the examples above, the program can use various types of non-transitory computer-readable media to store and supply to the computer. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., floppy disks, magnetic tapes, hard disk drives) and optical-magnetic recording media (e.g., optical discs). Examples of non-transitory computer-readable media include CD-ROM, CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM). Other examples of non-transitory computer-readable media include Programmable ROM (PROM), Erasable PROM (EPROM), Flash ROM, and Random Access Memory (RAM). Furthermore, programs can be supplied to a computer via various types of transient computer-readable media. Examples of transient computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transient computer-readable media can provide programs to a computer via wired or wireless communication paths such as wires and optical fibers.

[0060] Symbol Explanation 1-Attitude estimation display system, 2-First sensor, 3-Second sensor, 3a-Reference side barometric pressure sensor, 3b-Reference side barometric pressure sensor, 4-Display, 10-Attitude estimation device, 11-Attitude estimation unit, 12-Prompt unit, 30-External terminal, U-User.

Claims

1. A posture estimation and display system, characterized in that, have: The first sensor is worn by the user and measures air pressure; The second sensor is worn by the user or placed stationary in the space where the user is located and measures the air pressure. The attitude estimation unit acquires the pressure difference between the air pressure of the first sensor and the air pressure of the second sensor, and estimates the user's attitude based on the pressure difference; and The prompting unit provides the proportions of different postures of the user estimated based on the air pressure difference obtained from the first and second sensors during a specified period.

2. The attitude estimation and display system according to claim 1, characterized in that, The prompting unit prompts the user about the proportion of each posture category for each segment of the defined period, and the display order of the proportions of the different posture categories in each segment is the same, so that the user can visually identify how the proportions of the different posture categories shift in the consecutive segments.

3. The attitude estimation and display system according to claim 2, characterized in that, The prompting section corresponds to the plurality of intermediate segments that divide the specified period, and prompts at least one of the weather information and air pressure information of the surrounding area.

4. The attitude estimation and display system according to claim 2, characterized in that, The prompting unit indicates the time or number of times the user's standing or sitting action occurs in the vicinity of the region where the cumulative time of the user's posture in each of the plurality of intermediate segments of the specified period is indicated according to different posture categories.

5. The attitude estimation and display system according to claim 2, characterized in that, The prompting unit indicates whether the second sensor should be worn on the user's body or placed statically in the user's space. Furthermore, the prompting unit further segments the user's posture in each small segment of the middle section according to different posture categories, and the display order of the respective proportions of the different posture categories in each small segment is the same, so as to enable visual recognition of how the respective proportions of the different posture categories shift in a series of consecutive small segments.

Images