Posture evaluation device, fatigue estimation system, and posture evaluation method

Abstract

To provide a posture evaluation device or the like for enabling a user to appropriately have an estimated posture.SOLUTION: A posture evaluation device 200 includes: an image acquisition section (first acquisition section 101) for acquiring an image obtained by photographing an object person 11; an application acquisition section (third acquisition section 103 or posture estimation section 105) for acquiring an application which indicates a purpose of a behavior of the object person 11; a provisional position estimation section 105a for outputting an estimated provisional position by inputting the acquired image to a basic model for estimating the provisional position of a feature point of the posture of the object person 11 from the image; a position estimation section 105b for outputting the position of the feature point after performing correction corresponding to the application by inputting the output provisional position to one purpose specialization-type model corresponding to the acquired application among the plurality of prepared application specialization-type models; and a posture evaluation section 108 for evaluating the posture of the object person 11 based on the position of the output feature point after the correction, so as to output an evaluation result.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a posture evaluation device that evaluates the posture of a subject, a fatigue estimation system that estimates fatigue of a subject, and a posture evaluation method. [Background technology]

[0002] In recent years, there have been many cases where accumulated fatigue has led to poor health, injuries, accidents, etc. In response to this, attention has been drawn to technology that can prevent poor health, injuries, accidents, etc. by estimating the level of fatigue. For example, Patent Document 1 discloses a fatigue determination device as a fatigue estimation system for estimating the level of fatigue, which determines the presence or absence of fatigue and the type of fatigue based on force measurement and bioelectrical impedance measurement. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-023311 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when an estimated posture is used in a fatigue determination device or the like, it may not be possible to estimate the posture appropriately. As a result, it may be impossible to appropriately evaluate the posture using the estimated posture or estimate the fatigue level using the estimated posture. Therefore, the present disclosure provides a posture evaluation device or the like that can use a more appropriately estimated posture. [Means for solving the problem]

[0005] A posture evaluation device according to one aspect of the present disclosure includes an image acquisition unit that acquires an image of a subject; an application acquisition unit that acquires an application indicating a purpose of the subject's movement; a tentative position estimation unit that inputs the acquired image into a basic model for estimating tentative positions of feature points in the subject's posture from the image, and outputs the estimated tentative positions; a position estimation unit that inputs the output tentative positions into one application-specific model corresponding to the acquired application from among a plurality of application-specific models prepared in advance, and outputs positions of the feature points corrected according to the application; and a posture evaluation unit that evaluates the subject's posture based on the output corrected positions of the feature points and outputs an evaluation result.

[0006] Furthermore, a fatigue estimation system according to one aspect of the present disclosure includes an image acquisition unit that acquires an image of a subject; an application acquisition unit that acquires an application indicating the purpose of the subject's movement; a tentative position estimation unit that inputs the acquired image into a basic model for estimating tentative positions of feature points in the subject's posture from the image, and outputs the estimated tentative positions; a position estimation unit that inputs the output tentative positions into one application-specific model corresponding to the acquired application from among a plurality of application-specific models prepared in advance, and outputs positions of the feature points corrected according to the application; and a fatigue estimation unit that estimates a fatigue level of the subject based on the output corrected positions of the feature points.

[0007] Furthermore, a fatigue estimation method according to one aspect of the present disclosure is a posture evaluation method executed by a computer, and includes the steps of acquiring an image of a subject, acquiring an application indicating a purpose of the subject's movement, inputting the acquired image into a basic model for estimating tentative positions of feature points in the subject's posture, and outputting the estimated tentative positions, inputting the output tentative positions into one application-specific model corresponding to the acquired application from among a plurality of application-specific models prepared in advance, and outputting positions of the feature points corrected according to the application, and evaluating the posture of the subject based on the output corrected positions of the feature points, and outputting an evaluation result.

[0008] Furthermore, one aspect of the present disclosure can also be realized as a program for causing a computer to execute the posture evaluation method described above. [Effects of the Invention]

[0009] According to the posture evaluation device and the like according to one aspect of the present disclosure, a more appropriately estimated posture can be used. [Brief explanation of the drawings]

[0010] [Figure 1A] FIG. 1A is a first diagram illustrating posture estimation according to an embodiment. [Figure 1B] FIG. 1B is a second diagram illustrating posture estimation according to the embodiment. [Figure 2] FIG. 2 is a block diagram illustrating a functional configuration of the posture evaluation device according to the embodiment. [Figure 3] FIG. 3 is a diagram for explaining the configuration of a model in the posture evaluation device according to the embodiment. [Figure 4] FIG. 4 is a diagram for explaining the effect of correcting the provisional position according to the embodiment. [Figure 5A] FIG. 5A is a diagram for explaining the effect of correcting the provisional position according to the embodiment. [Figure 5B] FIG. 5B is a diagram for explaining the effect of correcting the provisional position according to the embodiment. [Figure 5C] 5C is a diagram for explaining the effect of correcting the provisional position according to the embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of the operation of the posture evaluation device according to the embodiment. [Figure 7] FIG. 7 is a diagram for explaining acquisition of a purpose according to the embodiment. [Figure 8] FIG. 8 is a diagram for explaining acquisition of a use according to another example of the embodiment. [Figure 9]FIG. 9 is a diagram illustrating an example of determining whether or not there is a risk associated with a posture according to the embodiment. [Figure 10A] FIG. 10A shows a subject standing still in posture A. [Figure 10B] FIG. 10B shows a subject standing still in posture B. [Figure 11A] FIG. 11A is a first diagram illustrating an estimated accumulation of a subject's fatigue level according to an embodiment. [Figure 11B] FIG. 11B is a second diagram illustrating the estimated accumulation of the subject's fatigue level according to the embodiment. [Figure 12] FIG. 12 is a diagram showing an example of displaying an estimation result according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection forms, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0012] It should be noted that the drawings are schematic diagrams and are not necessarily strict illustrations. In addition, in the drawings, substantially the same components are denoted by the same reference numerals, and overlapping descriptions may be omitted or simplified.

[0013] (Embodiment) [Posture evaluation device] The overall configuration of a posture evaluation device according to an embodiment will be described below. Fig. 1A is a first diagram for explaining posture estimation according to an embodiment. Fig. 1B is a second diagram for explaining posture estimation according to an embodiment.

[0014] In an embodiment, the analysis device 100 (see FIG. 2 described later) in the present disclosure is a device that estimates the posture of the subject 11 using an image output by capturing an image of the subject 11 using an imaging device 201, and issues an alert accompanied by a stimulus as feedback if there is a risk in the posture. The imaging device 201 is not limited to any particular form as long as it is a camera that captures an image of the subject 11 and outputs an image, and may be a fixed camera installed on a wall or ceiling of a building or the like as shown in FIG. 1A, or may be a camera mounted on a PC, smartphone, tablet terminal, or the like operated by the subject 11.

[0015] Here, the subject is seated in a chair 12. The posture evaluation device 200 of the present disclosure determines whether or not there is a risk associated with the subject's posture based on fatigue accumulated when the subject 11 maintains a fixed, static posture. In other words, the presence or absence of a risk of rising body temperature due to fatigue accumulated due to strain on at least one of the muscles and joints and impaired blood flow (hereinafter also referred to as a decrease in blood flow) caused by a fixed posture is determined. Therefore, the subject 11 maintains a static posture for at least a certain period of time. The certain period is the minimum period during which fatigue can be estimated by the analysis device 100, such as several tens of seconds or a few seconds. Such a period is determined depending on the processing capabilities of the analysis device 100 and the image capture device 201.

[0016] Examples of subjects 11 who assume such a static posture include desk workers in an office, drivers steering a moving object, people performing strength training using loads in a static posture, residents of facilities such as hospitals, and passengers and crew members of airplanes.

[0017] The image captured and output by the imaging device 201 is processed by the analysis device 100, and the posture of the subject 11 is estimated as shown in FIG. 1B. The estimated posture of the subject 11 is output as a rigid link model, for example. Specifically, as shown in FIG. 1B, straight lines are connected by key points indicated by black dots (in other words, feature points corresponding to the positions of body parts of the subject 11), and the posture of the subject 11 can be reproduced by the angle formed between two straight lines connected by one key point. The posture is estimated by image recognition, and is output as the above-mentioned rigid link model based on the positional relationship of the key points.

[0018] Next, the functional configuration of posture evaluation device 200 in the present disclosure will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the posture evaluation device according to an embodiment. In Fig. 2, (a) shows the overall configuration of posture evaluation device 200, and (b) shows a more detailed configuration of posture estimation unit 105.

[0019] 2(a), a posture assessment device 200 according to the present disclosure includes an imaging device 201 and an analysis device 100, and is used in a state in which the analysis device 100 is connected to the imaging device 201. The analysis device 100 is also connected to a timing device 202 and a reception device 204.

[0020] The analysis device 100 includes a first acquisition unit 101, a second acquisition unit 102, a third acquisition unit 103, a posture estimation unit 105, an evaluation unit 108, and an output unit 109.

[0021] The first acquisition unit 101 is an example of an image acquisition unit, and is a communication module that is connected to the imaging device 201 and acquires an image of the subject 11 from the imaging device 201. The connection between the first acquisition unit 101 and the imaging device 201 is wired or wireless, and there are no particular limitations on the method of communication that is performed via this connection.

[0022] The second acquisition unit 102 is a communication module that acquires the time from the connected timing device 202. The connection between the second acquisition unit 102 and the timing device 202 is wired or wireless, and there are no particular limitations on the method of communication that is performed via this connection.

[0023] The timing device 202 is a device that measures time, and is realized by a clock. The timing device 202 is capable of transmitting time to the connected second acquisition unit 102. Here, the time measured by the timing device 202 may be absolute time, or may be the time elapsed from a relative starting point. The timing device 202 may be realized in any form as long as it can measure the time between two points in time, the point in time when the subject 11 is detected to be stationary and an arbitrary point in time (i.e., the time during which the stationary posture is maintained).

[0024] The third acquiring unit 103 is a communication module that is connected to the accepting device 204 and acquires input information from the subject, such as personal information, from the accepting device 204. The connection between the third acquiring unit 103 and the accepting device 204 is made by wire or wirelessly, and there are no particular limitations on the method of communication made through the connection.

[0025] The reception device 204 is a user interface that receives input of personal information of the subject 11, and is realized by an input device such as a touch panel or a keyboard. The personal information includes at least one of information related to the subject's fatigue, such as age, sex, height, weight, muscle mass, stress level, body fat percentage, and exercise proficiency. The reception device 204 may also receive input of information related to the purpose of the movement from the subject 11. In this case, the third acquisition unit 103 can be said to be an example of a purpose acquisition unit that acquires a purpose indicating the purpose of the movement as information related to the purpose of the movement. This example will be described later.

[0026] The posture estimation unit 105 is a processing unit that is realized by executing a predetermined program using a processor and a memory. Through processing by the posture estimation unit 105, the posture of the subject 11 is estimated based on the image acquired by the first acquisition unit 101. In other words, the posture estimation unit 105 executes processing for calculating a posture feature amount in the posture of the subject 11 at that time for each unit time.

[0027] 2(b), posture estimation unit 105 includes two processing functions, tentative position estimation unit 105a and position estimation unit 105b, as well as information on multiple models such as basic model 105c, first purpose-specific model 105d, second purpose-specific model 105e, and third purpose-specific model 105f. The number of the multiple purpose-specific models corresponds to the type of purpose set, and specifically, the posture estimation unit 105 includes a purpose-specific model that corresponds one-to-one with each purpose. Furthermore, posture estimation unit 105 also includes a processing unit (not shown) that calculates posture feature amounts.

[0028] The posture feature amount calculation unit is a processing unit that calculates the output information on the positions of the body parts of the subject 11, i.e., the angle of a straight line that is a posture feature amount of the subject 11 from the rigid link model 11b. The posture feature amount calculation unit calculates, as a posture feature amount, an angle between two or more straight lines included in the rigid link model 11b, or an angle between a reference plane and one straight line included in the rigid link model 11b.

[0029] The tentative position estimation unit 105a is a processing unit that estimates information related to the tentative positions of body parts of the subject 11. The tentative position estimation unit 105a acquires an image of the subject 11 from the imaging device 201 via the first acquisition unit 101, and estimates the tentative position of each body part of the subject 11 based on the image. Each of the tentative positions of the body parts corresponds to the tentative positions of key points of the subject 11, and a tentative rigid link model 11b can be output by the estimation by the tentative position estimation unit 105a.

[0030] The position estimation unit 105b is a processing unit that corrects the provisional position to more accurately estimate the position of the subject's body part.

[0031] In this embodiment, the basic model 105c and a plurality of use-specific models (such as the first use-specific model 105d to the third use-specific model 105f) are combined to accurately estimate the positions of key points of the subject 11 from an image. When estimating the positions of key points from an image, it may not be possible to use a common model for estimation depending on the movement of the subject 11. For example, consider a case where the subject 11 is doing desk work and a case where the subject 11 is doing exercise that involves large movements such as aerobic exercise.

[0032] If the subject 11 is doing desk work, a large number of images of the subject 11 doing desk work are used as learning data in advance to construct a model for estimating the positions of key points of the subject 11. Naturally, the constructed model is specialized for estimating the positions of key points of the subject 11 doing desk work, so if it is used to estimate the positions of key points of the subject 11 engaged in large-scale exercise such as aerobic exercise, the estimated positions will be inaccurate. The opposite also occurs. For this reason, it is desirable to prepare models for estimating the positions of key points for each purpose and switch the model for inputting images depending on the purpose.

[0033] However, a model that estimates the positions of keypoints from an image in a series takes up a relatively large amount of space, so preparing a model for each purpose would require a large amount of resources. Moreover, if a single shared model is constructed using training data for all purposes without specializing it, it becomes difficult to guarantee estimation accuracy.

[0034] Therefore, in the present disclosure, a basic model 105c that estimates the tentative positions of keypoints from an image and application-specific models for each application that correct the tentative positions output by the basic model 105c according to the application are used. The basic model 105c and each of the multiple application-specific models are trained inference models that have been trained in advance.

[0035] Although the basic model 105c, which estimates the tentative positions of keypoints from an image, requires a certain number of intermediate layers, resulting in a relatively large capacity, it can be shared regardless of the application because a certain degree of estimation accuracy is sufficient since corrections are made in the subsequent application-specific models. Furthermore, each application-specific model is a model for correcting the tentative positions to more accurate positions, and because the input and output are the same type of information, namely coordinates, it is easy to reduce the number of intermediate layers, thereby maintaining a relatively small capacity.

[0036] 3, the basic model 105c and the first application-specific model 105d are used in combination for the first application, and the basic model 105c and the second application-specific model 105e are used in combination for the second application. This not only makes it possible to estimate the appropriate keypoint positions according to the application, but also makes it easier to reduce the capacity resources by the amount of the basic model 105c that is shared. Note that each application-specific model is configured so that the output of the basic model 105c can be input to any of the application-specific models.

[0037] 4 to 5C are diagrams for explaining the effect of correcting the temporary position according to the embodiment. FIG. 4 shows the positions of key points in a certain posture. (a) to (c) of FIG. 4 show the true values ​​of the positions, and (d) to (f) of FIG. 4 show the positions (hereinafter referred to as estimated values) estimated from the image using the basic model 105c. Note that (a) and (d) of FIG. 4 show coordinate values ​​in the XY plane, (b) and (e) of FIG. 4 show coordinate values ​​in the YZ plane, and (c) and (f) of FIG. 4 show coordinate values ​​in the XZ plane. In FIG. 4, the key points marked with arrows correspond to the key points on the head of the subject 11, and the thick line indicates the line connecting the key points on the right shoulder and the key points on the left shoulder of the subject 11 (the tip of the arrow corresponds to the key point on the left shoulder).

[0038] 4, for example, the coordinates of the key points of the head are significantly different between the true values ​​and the estimated values. In other words, estimation using only the basic model 105c is not suitable for estimating the purpose of this posture.

[0039] Therefore, the results of inputting this estimated value as a provisional position into the application-specific model are shown in Figures 5A to 5C. Figure 5A shows the relationship between the X-axis coordinate values ​​of the provisional position (dashed line), the corrected estimated position (solid line, the position of the keypoint finally output), and the true value (dash-dotted line) for the head keypoint. Figure 5B shows the relationship between the Y-axis coordinate values ​​of the provisional position (dashed line), the corrected estimated position (solid line), and the true value (dash-dotted line) for the head keypoint. Figure 5C shows the relationship between the Z-axis coordinate values ​​of the provisional position (dashed line), the corrected estimated position (solid line), and the true value (dash-dotted line) for the head keypoint. Note that the horizontal axis in Figures 5A to 5C indicates the serial number of each trial for multiple trials.

[0040] As shown in Figures 5A to 5C, when the data was input to the application-specific model, the estimated position after correction was shown to be significantly closer to the true value than the provisional position before correction, confirming the usefulness of combining the application-specific model with the basic model 105c. Note that in Figures 5B and 5C, the difference between the estimated position after correction and the true value is larger than in Figure 5A, for example. This is presumably because the application-specific model used for verification was not sufficiently trained. Therefore, for example, the estimated position after correction can be made closer to the true value by further training the application-specific model.

[0041] 2(a), based on the above, the tentative position estimation unit 105a inputs an image into the basic model 105c, thereby estimating and outputting the tentative positions of the keypoints. The tentative positions of the output keypoints can be used to form the tentative rigid link model 11b as described above.

[0042] Then, the position estimation unit 105b selects one application-specific model from the multiple application-specific models according to the application, and inputs the provisional position output by the provisional position estimation unit 105a, thereby outputting a more accurate keypoint position that has been corrected according to the application.

[0043] Here, in this embodiment, the purpose indicating the purpose of the movement of the subject person 11 is acquired by being estimated by the posture estimation unit 105. In other words, the posture estimation unit 105 also has the function of a purpose acquisition unit. An example of this will also be described later.

[0044] In the above description, the basic model 105c and the application-specific model are processed separately. However, if there is no need to extract an output from the basic model 105c, i.e., if the provisional rigid link model 11b is not used, a single model may be constructed by linking the basic model 105c and a single application-specific model according to the application, and an image may be input to the linked model. For example, this processing may be performed when the application is determined before inputting an image to the basic model 105c. As an example, if the application is acquired in advance or if the acquired application is used continuously, an image may be input to the linked model and the corrected keypoint positions may be directly output.

[0045] In this way, the position estimation unit 105b can form a more accurate rigid link model 11b of the subject 11 from the estimated positions of the key points. The posture feature amount calculation unit calculates and outputs posture feature amounts from the formed more accurate rigid link model 11b of the subject 11.

[0046] The evaluation unit 108 is a processing unit realized by executing a predetermined program using a processor and a memory. The evaluation unit 108 uses the posture estimated by the posture estimation unit 105 (i.e., posture feature amounts calculated from the positions of key points) to evaluate the posture of the subject 11 based on the cumulative time of the estimated posture using an evaluation standard conforming to ISO 11226, for example. The evaluation unit 108 outputs the evaluation result to the output unit 109.

[0047] The output unit 109 is a communication module that is connected to the display device 205 and the recovery device 206, and outputs content based on the posture evaluation results obtained by the posture evaluation device 200 to the display device 205 and the recovery device 206. The connection between the output unit 109 and the display device 205 or the recovery device 206 is made by wire or wirelessly, and there are no particular limitations on the method of communication made via this connection.

[0048] The display device 205 is a device for displaying content based on the posture evaluation result output by the output unit 109. The display device 205 displays an image showing content based on the posture evaluation result using a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel. The display device 205 is an example of a stimulation unit, and issues an alert to the subject 11 accompanied by a sensory stimulus that is relatively easy to perceive. For example, the alert displays an image indicating that there is a posture risk. Furthermore, instead of the display device 205, a device for issuing an alert accompanied by a stimulus to other senses, such as auditory, tactile, and olfactory stimuli other than such visual stimuli, may be provided. Furthermore, when the posture evaluation device 200 is configured to only reduce the fatigue level of the subject 11 using the recovery device 206, only the recovery device 206 may be provided, and the display device 205 is not essential.

[0049] As an example, the recovery device 206 is a device that reduces the subject 11's level of fatigue by promoting blood circulation in the subject 11 when the posture evaluation result is unfavorable (when a risky posture continues for a certain period of time or more). Specifically, the recovery device 206 actively changes the posture of the seated subject 11 by applying voltage, pressurizing, vibrating, or heating, or by changing the arrangement of various parts of the chair 12 using a mechanism provided in the chair 12. In this way, the recovery device 206 changes the load on at least one of the muscles and joints of the subject 11 and promotes blood circulation. In terms of blood flow, promoting blood circulation in this way reduces the impact of poor blood flow due to the subject 11 being in a stationary posture, and the subject 11 recovers from fatigue. The recovery device 206 is attached to or brought into contact with an appropriate body part of the subject 11 in advance, depending on the configuration of the device. When promoting blood circulation in subject 11 by heating, the entire space around subject 11 is heated, and in such a case, it is not necessary to attach or contact an appropriate body part of subject 11. Furthermore, when posture evaluation device 200 is configured only to display the posture evaluation result to subject 11, it is sufficient to include only display device 205, and recovery device 206 is not essential.

[0050] [Operation] Next, an example of the operation of the posture evaluation device 200 in the embodiment will be described with reference to Fig. 6 to Fig. 9. Fig. 6 is a flowchart showing an example of the operation of the posture evaluation device according to the embodiment.

[0051] As shown in FIG. 6, first, the first acquisition unit 101 acquires an image of the subject 11 (S101). Since the image capturing device 201 continuously captures images of the subject 11 and sequentially acquires the images, step S101 may be interpreted as starting to acquire the image of the subject 11. Next, the tentative position estimation unit 105a inputs the image of the subject 11 to the basic model 105c, causing it to output tentative positions of key points (S102). Here, a tentative rigid link model 11b is formed as described above. The posture estimation unit 105 estimates and acquires the purpose of the movement of the subject 11, i.e., the purpose, using the formed tentative rigid link model 11b (S103). For example, the posture estimation unit 105 calculates at least one of the speed of movement of the subject 11 (the speed of change in the time domain of the position of the key point) and the magnitude of movement (the magnitude of change in the time domain of the position of the key point) from the provisional rigid link model 11b of the subject 11, and estimates the use using at least one of the calculated speed and magnitude of movement of the subject 11.

[0052] FIG. 7 is a diagram for explaining acquisition of a use according to an embodiment. FIG. 7 shows estimated uses according to thresholds (corresponding to the intersections of the thresholds) when the horizontal axis represents the speed of movement and the vertical axis represents the magnitude of movement. Specifically, as shown in FIG. 7, when the speed of movement is greater than the threshold and the magnitude of movement is greater than the threshold, the use is estimated to be "muscle training." Similarly, when the speed of movement is greater than the threshold and the magnitude of movement is equal to or less than the threshold, the use is estimated to be "walking." When the speed of movement is equal to or less than the threshold and the magnitude of movement is greater than the threshold, the use is estimated to be "yoga, stretching." When the speed of movement is equal to or less than the threshold and the magnitude of movement is greater than the threshold, the use is estimated to be "desk work."

[0053] Note that the uses shown in this example are merely an example, and other uses may be set. Furthermore, when five or more uses are set, two or more thresholds may be set for each of the relationship between the speed and magnitude of the movement shown in FIG. 7. For example, when two thresholds are set for the magnitude of the movement, if the magnitude of the movement is equal to or less than one threshold, it is estimated to be a first use, if it is greater than the first threshold and equal to or less than another threshold, it is estimated to be a second use, and if it is greater than the other threshold, it is estimated to be a third use. In this way, if two thresholds are set for each of the speed and magnitude of the movement, a maximum of nine types of uses can be set.

[0054] Furthermore, the information used to estimate the purpose is not limited to the speed and magnitude of the movement. For example, the number of key points whose positions are estimated and the body parts whose positions are estimated as key points (in other words, the body parts whose positions of the key points are estimated) may be used to estimate the purpose.

[0055] On the other hand, FIG. 8 is a diagram for explaining acquisition of a purpose according to another example of the embodiment. In FIG. 8, for example, the reception device 204 is used to receive input from the subject 11, thereby receiving a selection of the purpose of the action by the subject 11, i.e., a selection of the purpose. That is, in this example, the subject 11 operates the reception device 204, and the third acquisition unit 103 directly acquires the selected purpose. For example, the reception device 204 displays a plurality of purpose options as options corresponding one-to-one to a plurality of prepared purpose-specific models. By receiving one selection from the options by the subject 11, the third acquisition unit 103 acquires the purpose corresponding to the selected purpose as the purpose acquisition unit.

[0056] Returning to the explanation of FIG. 6, for the use estimated from among the plurality of use-specific models as described above, the position estimation unit 105b selects a corresponding use-specific model (S104). Here, the posture estimation unit 105 determines whether or not a predetermined period of time has elapsed since the use was estimated (S105). The predetermined period differs depending on the use, for example, and is set to several hours for one use, and is set to several minutes to several tens of minutes for another use. In other words, the predetermined period is set appropriately depending on each use that is set.

[0057] If the posture estimation unit 105 determines that the predetermined period has elapsed (Yes in S105), the process returns to step S103, where the posture estimation unit 105 acquires a purpose again, and the position estimation unit 105b selects a purpose-specific model corresponding to the newly acquired purpose. That is, each time a purpose is acquired, the position estimation unit 105b selects a corresponding purpose-specific model from a plurality of purpose-specific models and uses it for subsequent processing. Conversely, the position estimation unit 105b continues to use the selected purpose-specific model for subsequent processing until a new purpose is acquired.

[0058] The position estimation unit 105b inputs the provisional positions output from the basic model 105c into the selected purpose-specific model, and causes the selected purpose-specific model to output the corrected positions of the key points of the subject 11 (S106). A more accurate rigid link model 11b is formed from the positions of the key points.

[0059] Then, the posture feature amount calculation unit calculates posture feature amounts from this more accurate rigid link model 11b (S107). After the posture feature amounts are obtained, it is determined whether or not there is a risk in the posture of the subject 11 (S108).

[0060] FIG. 9 is a diagram illustrating an example of determining whether or not a posture is at risk according to the embodiment. In FIG. 9, the horizontal axis represents the tilt angle, and the vertical axis represents the time the posture is maintained. The dotted hatched area in the graph indicates a region where the posture is determined to be at risk. The tilt angle is the angle between two straight lines that move at a specific key point in the estimated rigid link model 11b, i.e., the angle indicated by the posture feature. A reference angle range for the tilt angle that does not present a risk is set for each key point. When the tilt angle of the specific key point of interest deviates from the reference angle range (exceeds (a) in the figure), the posture estimation unit 105 determines that the posture is at risk after a specific time (set so that the greater the deviation, the shorter the time) has elapsed. The posture estimation unit 105 then determines that the posture is at risk (Yes in S108). Otherwise, since there is no posture risk, the process returns to step S101 and continues to repeat the same process. At this time, for example, the posture may be evaluated using evaluation criteria conforming to ISO 11226. The determination of whether or not there is a risk in the posture may be made in accordance with other standards, or may be made according to criteria established experimentally or empirically. After the result in step S108 is Yes, the output unit 109 displays an image on the display device 205 to issue an alert (S109).

[0061] In this way, it is possible to evaluate the posture of the subject 11 using the rigid link model 11b formed by more accurate keypoint positions, in other words, a more accurate posture.

[0062] [Variations] In the above description, an alert is issued when there is a posture risk. However, instead of issuing an alert, the degree of fatigue (also referred to as fatigue level) accumulated in the subject 11 may be estimated based on the elapsed time in a stationary posture. That is, instead of steps S108 and S109, a step of estimating the fatigue level may be performed using a rigid link model 11b formed based on more accurate keypoint positions, i.e., posture feature amounts calculated using a more accurate posture. Therefore, the evaluation unit 108 may perform an evaluation in which the fatigue level of the subject 11 is estimated using posture feature amounts calculated from the rigid link model 11b based on the corrected, more accurate keypoint positions. That is, the posture evaluation device 200 in this modification is an example of a fatigue estimation system, and the evaluation unit 108 is an example of a fatigue estimation unit.

[0063] 10A is a diagram showing a subject standing still in posture A. FIG. 10B is a diagram showing a subject standing still in posture B.

[0064] Similar to the subject 11 shown in FIG. 1A, the subject 11 shown in FIG. 10A and FIG. 10B is in a static sitting position in a chair 12. Although a table, PC, etc., which are not shown, are actually present in FIG. 10A and FIG. 10B, only the subject 11 and the chair 12 are shown here. The static posture of the subject 11 shown in FIG. 10A is posture A, which places a relatively heavy load on the shoulders. On the other hand, the static posture of the subject 11 shown in FIG. 10B is posture B, which places a relatively light load on the shoulders.

[0065] The fatigue level estimated for subject 11 standing still in posture A or posture B accumulates over time as shown in Figures 11A and 11B. Figure 11A is a first diagram illustrating the estimated accumulation of the subject's fatigue level according to the embodiment. Figure 11B is a second diagram illustrating the estimated accumulation of the subject's fatigue level according to the embodiment.

[0066] As shown in FIG. 11A, when subject 11 remains stationary in posture A shown in FIG. 10A or posture B shown in FIG. 10B, the fatigue level of subject 11 is expressed by a linear function whose slope is the load calculated from the posture.

[0067] As described above, posture A is a posture that imposes a greater load than posture B. Therefore, for example, in a certain muscle of subject 11 (here, a muscle related to shoulder movement), the load amount in posture A (the slope of the straight line in posture A) is greater than the load amount in posture B (the slope of the straight line in posture B). For this reason, subject 11 accumulates (accumulates) a greater degree of fatigue in a shorter period of time in posture A compared to when subject 11 is stationary in posture B.

[0068] On the other hand, as shown in FIG. 11B, when the posture of subject 11 changes from posture A shown in FIG. 10A to posture B shown in FIG. 10B, the fatigue level of subject 11 is expressed by a function that combines a linear function whose slope is the amount of load calculated from the posture and a linear function whose slope is the amount of change in posture.

[0069] Therefore, for example, while subject 11 remains stationary in posture A, the fatigue level of subject 11 is estimated as an accumulation (addition) of fatigue level using an increasing function with a positive slope corresponding to the load of posture A, as in FIG. 11A , and the accumulation (addition) turns to recovery (decrease) at the change point when subject 11 begins to change posture. Subject 11's fatigue level recovers (decreases) by an amount shown as the change width in the figure during the period when the posture change continues, using a decreasing function with a negative slope corresponding to the amount of posture change. After the change point when subject 11 remains stationary again in posture B, subject 11's fatigue level is estimated as an accumulation (addition) of fatigue level using an increasing function with a positive slope corresponding to the load of posture B.

[0070] In this way, the posture evaluation device 200 in the modified example estimates the fatigue level of the subject 11 that reflects accumulation and recovery in accordance with the stillness and change of the posture of the subject 11.

[0071] In this manner, the posture estimation unit 105 estimates the fatigue level of the subject 11 using the posture feature amount of the subject 11. FIG. 12 is a diagram showing an example of displaying an estimation result according to a modified example of the embodiment. As shown in FIG. 12, the posture evaluation device 200 according to the modified example can display and provide feedback on the estimation result of the fatigue level of the subject 11 using the display device 205. Specifically, as shown in FIG. 12, by visualizing the fatigue level of the subject 11, it is possible to visually understand how tired the subject 11 is. In the figure, a doll representing the subject 11 and the fatigue levels of the subject's shoulders, back, and main parts are displayed together on the display device 205. To make it easier for the subject 11 to intuitively understand the fatigue level, the fatigue level of the shoulders is displayed as a "stiff shoulder level," the fatigue level of the back is displayed as a "back pain level," and the fatigue level of the lower back is displayed as a "lower back pain level."

[0072] Here, in the display in the figure, fatigue levels of three parts of the subject 11 are displayed all at once, but the fatigue levels of these three parts are estimated from an image captured at the same time. In other words, the posture evaluation device 200 estimates the fatigue levels of the muscles and / or joints in each of multiple body parts including a first part (e.g., shoulders), a second part (e.g., back), and a third part (e.g., waist) of the subject 11 from posture feature amounts based on more accurate key points after correction, estimated from a single image of the subject 11. Therefore, even if the posture of the subject 11 is constant, the fatigue levels accumulated in the muscles and / or joints differ for each body part, but the posture evaluation device 200 can simultaneously and individually estimate such different fatigue levels.

[0073] The posture evaluation device 200 calculates the load amount for each of multiple body parts, and for one posture of the subject 11, it is possible to estimate the fatigue level of a first part (the above-mentioned shoulder stiffness level) based on the load amount calculated for the first part, the fatigue level of a second part (the above-mentioned back pain level) based on the load amount calculated for the second part, and the fatigue level of a third part (the above-mentioned lower back pain level) based on the load amount calculated for the third part.

[0074] In the example shown in the figure, the degree of stiff shoulders is estimated from the load on the trapezius muscle, the degree of back pain is estimated from the fatigue level of the latissimus dorsi muscle, and the degree of lower back pain is estimated from the load on the lumbar paraspinal muscles. In this way, a single fatigue level may be estimated from a load calculated from posture feature values ​​corresponding to a single muscle and / or joint. Alternatively, a single fatigue level may be estimated from a composite load calculated from posture feature values ​​corresponding to multiple muscles and / or joints. For example, the degree of stiff shoulders (i.e., a single fatigue level in the shoulder region) may be estimated from the average value of the loads on the trapezius muscle, levator scapulae muscle, rhomboid muscle, and deltoid muscle. Furthermore, in estimating the fatigue level, a more realistic estimation of the fatigue level may be performed by weighting the load on the muscle and / or joint that has a particularly large influence on the fatigue level of the relevant body part, rather than simply using an average value.

[0075] The fatigue levels estimated in this way may be shown as relative positions on a reference meter with a minimum value of 0 and a maximum value of 100, as shown in the figure. Here, a reference value is set at a predetermined position on the reference meter. Such a reference value is set to a relative position (or around that position, etc.) of the fatigue level that may cause subjective symptoms such as pain in a typical subject 11, which has been quantified in advance through an epidemiological survey or the like. Therefore, different reference values ​​may be set depending on the fatigue level of each body part.

[0076] Furthermore, the display device 205 may display a warning to the subject 11 as an estimation result when the estimated fatigue level of the subject 11 reaches a reference value. In other words, the display device 205 may also have an alert issuing function. In the figure, an example of such a warning is displayed at the bottom of the display device 205, saying, "The degree of stiff shoulders exceeds the reference value." In addition, in connection with such a warning, the display device 205 may also display a specific countermeasure, such as "We recommend taking a break," as also shown in the figure.

[0077] Furthermore, in addition to the configuration described above in which the estimation result is displayed to the subject 11 to encourage the subject 11 to deal with the accumulated fatigue level, a configuration in which the posture evaluation device 200 actively recovers the fatigue level of the subject 11 can also be considered. Specifically, the fatigue level of the subject 11 is recovered by operating the recovery device 206 shown in Fig. 2. The specific configuration of the recovery device 206 is as described above and will not be described further. However, when the estimated fatigue level of the subject 11 reaches a reference value, the recovery device 206 operates to change the load on at least one of the muscles and joints of the subject 11 and promote blood circulation, thereby reducing the subject's fatigue level.

[0078] [Effects, etc.] As described above, the posture evaluation device 200 according to the first aspect of the present disclosure includes an image acquisition unit (first acquisition unit 101) that acquires an image of the subject 11, an application acquisition unit (third acquisition unit 103 or posture estimation unit 105) that acquires an application indicating the purpose of the action of the subject 11, a tentative position estimation unit 105a that outputs estimated tentative positions by inputting the acquired image into a basic model for estimating tentative positions of feature points in the posture of the subject 11 from the image, a position estimation unit 105b that inputs the output tentative positions into one application-specific model corresponding to the acquired application out of multiple application-specific models prepared in advance, and outputs positions of the feature points corrected according to the application, and a posture evaluation unit (evaluation unit 108) that evaluates the posture of the subject 11 based on the output corrected positions of the feature points and outputs an evaluation result.

[0079] With this configuration, it is possible to obtain accurate feature point positions by combining the basic model with an appropriate application-specific model for the posture to be estimated. Compared to preparing a model for estimating feature point positions (i.e., posture) for each application, the basic model portion can be shared for each application, so posture estimation can be performed with reduced capacity resources such as memory capacity and communication capacity. Therefore, from the perspective of balancing such capacity and estimation accuracy, it is possible to evaluate the posture of the subject 11 using a more appropriately estimated posture and output the evaluation result.

[0080] Furthermore, the posture evaluation device 200 according to the second aspect of the present disclosure is the posture evaluation device 200 according to the first aspect, in which the purpose acquisition unit (posture estimation unit 105) acquires a purpose indicating a predetermined purpose for the magnitude of change in the time domain of the tentative position.

[0081] This makes it possible to automatically acquire a use indicating a predetermined purpose for the magnitude of change in the time domain of the provisional position.

[0082] Furthermore, the posture evaluation device 200 according to the third aspect of the present disclosure is the posture evaluation device 200 according to the first aspect, in which the purpose acquisition unit (posture estimation unit 105) acquires a purpose indicating a predetermined purpose for the speed of change in the time domain of the tentative position.

[0083] This makes it possible to automatically obtain a use indicating a predetermined purpose for the rate of change of the provisional position in the time domain.

[0084] Furthermore, the posture evaluation device 200 according to the fourth aspect of the present disclosure is the posture evaluation device 200 according to the first aspect, and the purpose acquisition unit (third acquisition unit 103) receives input regarding a purpose and thereby acquires a purpose indicating the purpose.

[0085] According to this, by receiving an input regarding a purpose, it is possible to obtain a use indicating the purpose.

[0086] Furthermore, the posture evaluation device 200 according to a fifth aspect of the present disclosure is the posture evaluation device 200 according to the first aspect, in which the usage acquisition unit (third acquisition unit 103 or posture estimation unit 105) acquires a usage every time a predetermined period of time elapses, and the position estimation unit 105b selects one usage-specific model from a plurality of usage-specific models every time a usage is acquired, and inputs the output tentative position into the selected one usage-specific model.

[0087] This allows the purpose-specific model to be reviewed every time a predetermined period of time elapses.

[0088] Furthermore, the posture evaluation device 200 according to a sixth aspect of the present disclosure is the posture evaluation device 200 according to the first aspect, in which the position estimation unit 105b inputs the output tentative position by continuously using the selected one purpose-specific model until a new purpose is acquired.

[0089] This allows the purpose-specific model to be used continuously until a new purpose is acquired.

[0090] In addition, the fatigue estimation system according to the seventh aspect of the present disclosure includes an image acquisition unit (first acquisition unit 101) that acquires an image of the subject 11, a use acquisition unit (third acquisition unit 103 or posture estimation unit 105) that acquires a use indicating the purpose of the action of the subject 11, a tentative position estimation unit 105a that outputs estimated tentative positions by inputting the acquired image into a basic model for estimating tentative positions of feature points in the posture of the subject 11 from the image, a position estimation unit 105b that inputs the output tentative positions into one use-specific model corresponding to the acquired use from among multiple use-specific models prepared in advance, and outputs positions of the feature points corrected according to the use, and a fatigue estimation unit (evaluation unit 108) that estimates the fatigue level of the subject 11 based on the output corrected positions of the feature points.

[0091] According to this, by combining the basic model with an appropriate application-specific model for the posture to be estimated, accurate positions of feature points can be obtained. Compared to preparing a model for estimating the positions of feature points (i.e., posture) for each application, the basic model portion can be shared for each application, so posture estimation can be performed with reduced capacity resources such as memory capacity and communication capacity. Therefore, from the perspective of balancing such capacity and estimation accuracy, it is possible to estimate the fatigue level of the subject 11 based on a more appropriately estimated posture.

[0092] Furthermore, a posture evaluation method according to an eighth aspect of the present disclosure is a posture evaluation method executed by a computer, and includes the steps of acquiring an image of a subject, acquiring an application indicating the purpose of the subject's movement, inputting the acquired image into a basic model for estimating tentative positions of feature points in the subject's posture, and outputting the estimated tentative positions, inputting the output tentative positions into one application-specific model corresponding to the acquired application from among a plurality of application-specific models prepared in advance, and outputting positions of the feature points corrected according to the application, and evaluating the subject's posture based on the output corrected positions of the feature points, and outputting an evaluation result.

[0093] This provides the same effects as those of the posture evaluation device 200 described above.

[0094] A program according to another aspect of the present disclosure is a program for causing a computer to execute the posture evaluation method according to the eighth aspect.

[0095] By executing this on a computer, the same effects as those of the posture evaluation device 200 described above can be achieved.

[0096] (Other embodiments) Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

[0097] For example, in the above embodiment, a process executed by a specific processing unit may be executed by another processing unit, the order of multiple processes may be changed, or multiple processes may be executed in parallel.

[0098] Furthermore, the posture assessment device of the present disclosure may be realized by multiple devices each having some of the multiple components, or may be realized by a single device having all of the multiple components. Also, some of the functions of a component may be realized as the functions of another component, and each function may be distributed in any way among the components. Any form having a configuration that substantially provides all of the functions that can realize the posture assessment device of the present disclosure is included in the present disclosure.

[0099] In the above-described embodiments, each component may be realized by executing a software program suitable for that component, or by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0100] Furthermore, each component may be realized by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0101] Furthermore, the general or specific aspects of the present disclosure may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0102] In the above embodiment, the posture of the subject is estimated from the image using a rigid link model generated by image recognition, but the posture estimation method is not limited to this. Any existing method may be used as a method for estimating the posture of the subject from the image.

[0103] The present disclosure may also be realized as a posture evaluation method executed by a posture evaluation device, as a program for causing a computer to execute such a posture evaluation method, or as a computer-readable non-transitory recording medium on which such a program is recorded.

[0104] In addition, this disclosure also includes forms obtained by applying various modifications to the embodiments that a person skilled in the art would conceive, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope that does not deviate from the intent of this disclosure. [Explanation of symbols]

[0105] 11. Target Audience 11b Rigid link model 12 chairs 100 Analyzer 101 First acquisition part 102 Second acquisition part 103 Third acquisition part 105 Posture estimation section 105a Temporary position estimation unit 105b Position estimation section 105c basic model 105d First purpose-specific model 105e Second purpose specialized model 105f Third-purpose specialized model 108 Evaluation Department 109 Output section 200 Posture evaluation device 201 Imaging device 202 Timing device 204 Reception Device 205 Display device 206 Recovery Device

Claims

1. an image acquisition unit that acquires an image of a subject; a purpose acquisition unit that acquires a purpose indicating a purpose of the subject's action; a tentative position estimation unit that inputs the acquired image into a basic model for estimating tentative positions of feature points of the posture of the subject from the image, and outputs the estimated tentative positions; a position estimation unit that outputs the position of the feature point corrected according to the purpose by inputting the output tentative position into one purpose-specific model according to the acquired purpose out of a plurality of purpose-specific models prepared in advance; a posture evaluation unit that evaluates the posture of the subject based on the output corrected positions of the feature points and outputs an evaluation result. Posture assessment device.

2. The purpose acquisition unit acquires the purpose indicating a predetermined purpose for a magnitude of change in the time domain of the provisional position. The posture evaluation device according to claim 1 .

3. The purpose acquisition unit acquires the purpose indicating a predetermined purpose for a speed of change of the provisional position in a time domain. The posture evaluation device according to claim 1 or 2.

4. The purpose acquisition unit receives an input regarding a purpose and acquires the purpose indicating the purpose. The posture evaluation device according to claim 1 .

5. the purpose acquisition unit acquires the purpose every time a predetermined period of time elapses, The position estimation unit selects the one purpose-specific model from the plurality of purpose-specific models each time the purpose is acquired, and inputs the output tentative position to the selected one purpose-specific model. The posture evaluation device according to claim 1 .

6. The location estimation unit continues to use the selected one purpose-specific model to input the output tentative location until the purpose is newly acquired. The posture evaluation device according to claim 5 .

7. an image acquisition unit that acquires an image of a subject; a purpose acquisition unit that acquires a purpose indicating a purpose of the subject's action; a tentative position estimation unit that inputs the acquired image into a basic model for estimating tentative positions of feature points of the posture of the subject from the image, and outputs the estimated tentative positions; a position estimation unit that outputs the position of the feature point corrected according to the purpose by inputting the output tentative position into one purpose-specific model according to the acquired purpose out of a plurality of purpose-specific models prepared in advance; a fatigue estimation unit that estimates a fatigue level of the subject based on the output corrected positions of the feature points. Fatigue estimation system.

8. 1. A computer-implemented method for posture assessment, comprising: acquiring an image of a subject; acquiring a purpose indicating a purpose of the subject's action; a step of inputting the acquired image into a basic model for estimating tentative positions of feature points of the posture of the subject, and outputting the estimated tentative positions; a step of inputting the output tentative position into one application-specific model corresponding to the acquired application out of a plurality of application-specific models prepared in advance, thereby outputting the position of the feature point corrected according to the application; and evaluating the posture of the subject based on the output corrected positions of the feature points, and outputting an evaluation result. Posture assessment methods.

Images