Information processing method and information processing apparatus
The method and device estimate body load on a body part by analyzing object information and posture, offering accurate load assessment and reduction recommendations.
Patent Information
- Application Number
- JP2023071823
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-25
- Publication Date
- 2024-11-07
AI Technical Summary
Existing methods fail to estimate the load exerted on a body part when a subject is holding an object.
An information processing method and device that calculates body load applied to a body part by acquiring object information and estimated posture using machine learning models and sensors, outputting body load information.
Accurately estimates body load applied to a body part, providing reliable and intuitive visual feedback on load distribution and recommending load reduction strategies.
Smart Images

Figure 2024157447000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing method and an information processing device. [Background technology]
[0002] Patent Document 1 describes a method of estimating a posture of a subject, and estimating a fatigue level of the subject based on the estimated posture and the duration of the posture. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] International Publication No. 2021 / 112096 Summary of the Invention [Problem to be solved by the invention]
[0004] However, the method described in Patent Document 1 cannot estimate the load that an object exerts on a body part of a subject when the subject is holding the object.
[0005] The problem that the present invention aims to solve is to provide an information processing method and an information processing device that can estimate the physical load placed on a specified body part due to an object when a subject is holding the object. [Means for solving the problem]
[0006] The present invention solves the above problem by, when a subject is holding a specified object, obtaining object information of the object based on an image of the subject, calculating a physical load estimate that is an estimate of the physical load on a specified body part based on the object information and the subject's estimated posture, and outputting physical load information indicating the physical load estimate to an information display unit. Effect of the Invention
[0007] According to the present invention, a physical load estimate is calculated from object information and an estimated posture, so that when a subject is holding an object, the physical load acting on a predetermined body part due to the object can be estimated. [Brief description of the drawings]
[0008] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first embodiment. [Diagram 2] 2 is a diagram showing an example of an image of a subject input to an image input unit of the information processing device shown in FIG. 1. [Diagram 3] 1. FIG. 4 is a diagram showing an example of display of physical load information output by the information processing device shown in FIG. 1, specifically, an example of a graph showing the distribution of accuracy of estimated physical load estimates included in the physical load information. [Figure 4] 1. FIG. 4 is a diagram showing an example of display of physical load information output by the information processing device shown in FIG. 1, in which the distribution of estimated value accuracy of physical load estimated values included in the physical load information is displayed in shades of color. [Diagram 5] 4 is a flowchart showing the procedure of an information processing method according to the first embodiment. [Figure 6] 10 is a flowchart showing the procedure of an information processing method according to a second embodiment. [Figure 7] 13 is a flowchart showing the procedure of an information processing method according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] First Embodiment A first embodiment of the present invention will be described with reference to FIGS. The information processing device 100 shown in FIG. 1 estimates the physical load on a predetermined body part of a subject P. The subject P is a person for whom the information processing device 100 estimates the physical load (see FIG. 2). As shown in FIG. 1, the information processing device 100 acquires an image input to an image input unit 1. The information processing device 100 also outputs physical load information related to the physical load of the subject to an information display unit 2. The information processing device 100 also has a processor 10.
[0010] The image input unit 1 is, for example, a camera. That is, the image input to the image input unit 1 is an image captured by the camera serving as the image input unit 1. The image input unit 1 may be a camera already installed in the vicinity of the subject P or in a facility around the subject P, or may be a camera built into a mobile terminal.
[0011] The information display unit 2 is, for example, a display that visually displays character information, etc. The information display unit 2 may also be a touch panel display into which a user (the subject P and / or the evaluator of the subject P's physical load) can manually input information. The information display unit 2 may also be a speaker that outputs information by voice.
[0012] The processor 10 includes a ROM (Read Only Memory) in which a program is stored, a CPU (Central Processing Unit) that executes the program stored in the ROM, and a RAM (Random Access Memory) that functions as an accessible storage device. The processor 10 includes an object holding determination unit 11, an estimated posture acquisition unit 12, an object information acquisition unit 13, a physical load estimated value calculation unit 14, an information generation unit 15, and an output unit 16. The processor 10 executes the functions of the object holding determination unit 11, the estimated posture acquisition unit 12, the object information acquisition unit 13, the physical load estimated value calculation unit 14, the information generation unit 15, and the output unit 16 by using the CPU to execute the program stored in the ROM.
[0013] The configuration of the processor 10 will be described in detail below. The object holding determination unit 11 determines whether or not the subject P holds a specific object X based on the image V input to the image input unit 1 (see FIG. 2). That is, the object holding determination unit 11 determines whether or not the object X is held in a specific body part (right hand B1 in FIG. 2) of the subject P based on the image V of the subject P. The specific body part is a body part for holding the object X. The object holding determination unit 11 may determine whether or not the object X is attached to the subject P's specific body part such as an arm, a leg, or a head, not limited to the hand (right hand B1 or left hand) of the subject P. The object holding determination unit 11 may determine that "the subject P holds the object X" by receiving a signal from a predetermined storage facility indicating that the object X has been removed from the storage facility. The object holding determination unit 11 may determine that "the subject P holds the object X" by receiving a signal from the object X indicating that the object X has been held in the hand of the subject P or that the object X has been attached to the subject P's body. Note that object X may be, for example, a tool used by subject P when working in a factory, but is not limited to this and may also be a bat or racket used by subject P when playing sports.
[0014] When the object holding determination unit 11 determines that "the subject P is holding the object X", the estimated posture acquisition unit 12 acquires an estimated posture of the subject P using the trained first machine learning model A1. The first machine learning model A1 is, for example, a posture detection model such as Openpose. The first machine learning model A1 outputs an estimated posture of the subject P (posture information including position information of the joints and skeletal information of the subject P) based on the image V input to the image input unit 1.
[0015] Furthermore, the estimated posture acquisition unit 12 may acquire an estimated posture of the subject P based on detection signals of one or more sensors attached to the body of the subject P, without using the first machine learning model A1. Furthermore, the estimated posture acquisition unit 12 may acquire an estimated posture of the subject P by detecting positions of multiple markers attached to the body of the subject P, based on the image V input to the image input unit 1, without using the first machine learning model A1.
[0016] When the object holding determination unit 11 determines that "the subject P is holding the object X", the object information acquisition unit 13 acquires object information of the object X using the trained second machine learning model A2 based on the image V. Specifically, as shown in FIG. 2, the object information acquisition unit 13 first acquires the position of a specific body part (the right hand B2 in FIG. 2) for holding the object X based on the estimated posture acquired by the estimated posture acquisition unit 12, and extracts an area including the coordinate position of the specific body part (the right hand B2) in the image V as a specific area image Vx. The specific area image Vx is an image within an area defined by a so-called bounding box. The specific area image Vx includes the object X held by the specific body part (the right hand B2) of the subject P. That is, the object information acquisition unit 13 extracts the specific area image Vx including the object X from the image V. The size of the specific area image Vx may be set in advance or may be automatically determined according to the size of the object X. In addition, the object information acquisition unit 13 may extract area images (images of areas surrounded by bounding boxes corresponding to each object) relating to all objects (including object X and objects other than object X) contained in image V, and extract from the extracted area images those that include the coordinate positions of specific body parts as specific area images Vx.
[0017] Then, the object information acquisition unit 13 acquires object information based on the specific area image Vx using the second machine learning model A2. Specifically, the object information acquisition unit 13 acquires the type candidates of the object X output by the second machine learning model A2 based on the specific area image Vx through a class classification process. For example, the second machine learning model A2 estimates that the object X is a tool manufactured by company A based on the specific area image Vx. The second machine learning model A2 calculates the type candidates of the object X as well as the accuracy (prediction reliability) of the type candidates. Then, the object information acquisition unit 13 acquires object information associated with the type candidates by referring to a database (not shown). Here, when the object information acquisition unit 13 acquires a plurality of type candidates, it acquires the accuracy output by the second machine learning model A2 for each of the plurality of type candidates. Then, the object information acquisition unit 13 acquires only object information associated with type candidates whose accuracy is equal to or higher than a predetermined threshold. For example, the second machine learning model A2 outputs a tool manufactured by company A, a tool manufactured by company B, and a tool manufactured by company C as candidate types of object X, and calculates the accuracy of 90%, 8%, 2%, etc. for each candidate type. Then, when the predetermined threshold is 10%, the object information acquisition unit 13 acquires only the object information of the tool manufactured by company A, which has an accuracy of 90%. The second machine learning model A2 may output candidate types of the object X by segmentation without extracting the specific region image Vx. The object information includes at least one of the mass, center of gravity, and moment of inertia of the object X.
[0018] Furthermore, the object information acquisition unit 13 may acquire object information output by the second machine learning model A2 based on the specific area image Vx through regression processing. That is, the second machine learning model A2 may directly calculate the object information based on the specific area image Vx without determining a class candidate for the object X.
[0019] The physical load estimated value calculation unit 14 calculates a physical load estimated value F on a predetermined body part (the right arm B0 in FIG. 2) based on the estimated posture acquired by the estimated posture acquisition unit 12 and the object information acquired by the object information acquisition unit 13. The physical load estimated value F is a value indicating the magnitude of the force (pulling force or compressive force) applied to the muscle of the body part, or the magnitude of the moment applied to the joint of the body part. Specifically, as shown in FIG. 2, when the body part for which the physical load estimated value F is calculated is the right arm B0, the physical load estimated value F is the magnitude of the force applied to the forearm muscle or the biceps muscle of the right arm B0, or the magnitude of the moment applied to the elbow joint. The physical load estimated value calculation unit 14 may calculate a plurality of physical load estimated values F for one body part. That is, for example, the physical load estimated value calculation unit 14 may calculate the magnitude of the force applied to the forearm muscle, the magnitude of the force applied to the biceps muscle, and the magnitude of the moment applied to the elbow joint for the right arm B0. Furthermore, the physical load estimated value calculation unit 14 may calculate the physical load estimated value F for a plurality of body parts.
[0020] Furthermore, the physical load estimated value calculation unit 14 calculates the physical load estimated value F as a value having a predetermined range. That is, the physical load estimated value calculation unit 14 calculates the physical load estimated value F by the following formula (1). In other words, the physical load estimated value F is estimated to take a value in the range of F0-dF to F0+dF. Note that the possible range of the physical load estimated value F (F0-dF to F0+dF) is referred to as the estimated value range Q in the following explanation. F = F0 ± dF … (1)
[0021] Furthermore, when there are multiple candidate types of object X, the physical load estimated value calculation unit 14 calculates an estimated value range Q so as to include all of the physical load estimated values F calculated based on the object information of each of the candidate types.
[0022] Furthermore, the physical load estimate calculation unit 14 calculates an estimate accuracy C, which is the accuracy of the physical load estimate F in the estimate range Q. When there are multiple type candidates for the object X, the physical load estimate calculation unit 14 calculates the estimate accuracy C of the physical load estimate F in the estimate range Q based on the accuracy of each of the type candidates. Furthermore, the physical load estimate calculation unit 14 may calculate the estimate accuracy C of the median value F0 in the estimate range Q to be the highest, and calculate the estimate accuracy C to become lower as it moves away from the median value F0.
[0023] The information generating unit 15 generates physical load information indicating an estimated physical load value F. The physical load information includes information indicating an estimated value range Q of the estimated physical load value F. The physical load information also includes information visually displaying the distribution of the estimated value accuracy C of the estimated physical load value F within the estimated value range Q using colors or a graph.
[0024] For example, Fig. 3 is a display example of physical load information that shows in graph form the distribution of the estimation accuracy C of the estimated physical load F. In Fig. 3, the range of the estimated physical load F = F1 to F2 (F1 = F0 - dF, F2 = F0 + dF) is the estimation range Q. Furthermore, the estimation accuracy C of the median value F0 in the estimation range Q takes the highest value C0. Moreover, as the estimated physical load F moves away from the median value F0 and approaches the minimum value F1 or maximum value F2, the estimation accuracy C decreases and approaches 0.
[0025] FIG. 4 is a display example of physical load information in which the distribution of the estimated value accuracy C of the estimated physical load F is expressed by color shading. That is, in FIG. 4, the distribution of the estimated value accuracy C is expressed by color shading along the scale M representing the estimated physical load F. Note that in FIG. 4, the distribution of the estimated value accuracy C is displayed only in white, gray, and black, but it may be displayed in, for example, red-based color shading. In the example shown in FIG. 4, the estimated physical load F is 5.0±2.7 [N]. That is, in the example shown in FIG. 4, the estimated physical load F is in the range of 2.3 to 7.7, and the median value F0 of the estimated physical load F is 5.0 [N], the minimum value F1 is 2.3 [N], and the maximum value F2 is 7.7 [N]. Here, since the estimated value accuracy C is highest at the median value F0=5.0[N] of the estimated physical load F, the range Q0 (4.6-5.4[N]) near the median value F0=5.0[N] is displayed in the darkest color. For example, if the distribution of the estimated value accuracy C is displayed in shades of red, the range Q0 of 4.6-5.4[N] is displayed in dark red. Moreover, as the estimated physical load F moves away from the median value F0=5.0[N], the color becomes lighter. For example, the range Q1 of the estimated physical load F=3.7-4.6[N] and the range Q2 of the estimated physical load F=5.4-6.3[N] are displayed in orange. Furthermore, the range Q3 of the estimated physical load F=2.3-3.7[N] and the range Q4 of the estimated physical load F=6.3-7.7[N] are displayed in light yellow. That is, the distribution of the estimated value accuracy C is displayed in a darker color the higher the estimated value accuracy C, and in a lighter color the lower the estimated value accuracy C. The distribution of the estimated value accuracy C may be expressed not only by the shade of color, but also by changes in hue. For example, the distribution of the estimated value accuracy C may be displayed in a warmer color the higher the estimated value accuracy C, and in a cooler color the lower the estimated value accuracy C.
[0026] Furthermore, the information generator 15 determines whether the estimated value range Q is wider than a predetermined range, and if the estimated value range Q is wider than the predetermined range, generates information to prompt the subject P to manually input object information. Note that the predetermined range is a threshold for determining whether the calculation result of the estimated physical load value F has a predetermined level of reliability or higher. Furthermore, when there are multiple type candidates output by the second machine learning model A2, the information generator 15 may generate information that presents the output type candidates and prompts the user to select the type of the object X.
[0027] Furthermore, the output unit 16 shown in FIG. 1 outputs information including the physical load information generated by the information generation unit 15 to the information display unit 2.
[0028] Next, the procedure of the information processing method executed by the processor 10 of the information processing device 100 will be described with reference to the flowchart of FIG. First, in step S1, the processor 10 acquires the image V input to the image input unit 1. Then, in step S2, the object holding determination unit 11 determines whether or not the subject P is holding the object X. If the subject P is not holding the object X, the process ends.
[0029] On the other hand, if it is determined in step S2 that "the subject P is holding the object X", the estimated posture acquisition unit 12 acquires an estimated posture of the subject P in step S3. Then, in step S4, the object information acquisition unit 13 extracts a specific area image Vx including the object X. Next, in step S5, the object information acquisition unit 13 acquires object information based on the specific area image Vx using the second machine learning model A2. Further next, in step S6, the physical load estimate calculation unit 14 calculates a physical load estimate F, which is an estimate of the physical load applied to a predetermined body part, based on the estimated posture of the subject P and the object information of the object X. Furthermore, in step S7, the physical load estimated value calculation section 14 calculates the estimated value range Q of the physical load estimated value F.
[0030] Next, in step S8, the information generator 15 determines whether or not the estimated value range Q of the estimated physical load value F is wider than a predetermined range.
[0031] If it is determined in step S8 that "the estimated value range Q of the estimated physical load value F is wider than the predetermined range," in step S9, the information generator 15 generates information to prompt the subject P to manually input object information of the object X, and the output unit 16 outputs the information to the information display unit 2. Then, if the object information of the object X is manually input, the process returns to step S5, and the processor 10 acquires the object information.
[0032] On the other hand, if it is determined in step S8 that “the estimated value range Q of the estimated physical load value F is within the predetermined range,” then in step S10, the output unit 16 outputs physical load information indicating the estimated physical load value F to the information display unit 2.
[0033] As described above, when the subject P is holding an object X, the information processing device 100 according to this embodiment acquires an estimated posture of the subject P, acquires object information of the object X based on the image V of the subject P input to the image input unit 1, calculates a physical load estimate F that is an estimate of the physical load on a body part based on the estimated posture and the object information, and outputs physical load information indicating the physical load estimate F to the information display unit 2. As a result, the information processing device 100 calculates the physical load estimate F from the object information and estimated posture, and can estimate the physical load on a specific body part caused by the object X when the subject P is holding the object X. Furthermore, since the information processing device 100 acquires the object information of the object X based on the image V, the user does not need to manually input the object information of the object X.
[0034] Furthermore, the information processing device 100 acquires the estimated posture output by the first machine learning model A1 based on the image V. This allows the information processing device 100 to acquire the estimated posture based only on the image V without the need to attach a sensor or the like to the body of the subject P in order to acquire the estimated posture.
[0035] The information processing device 100 may acquire an estimated posture based on detection signals of one or more sensors attached to the body of the subject P. This allows the information processing device 100 to acquire an estimated posture with high accuracy based on the detection signals of the sensors.
[0036] Furthermore, the object information acquired by the information processing device 100 includes at least one of the mass, center of gravity position, and moment of inertia of the object X. This allows the information processing device 100 to calculate a specific physical load estimate based on the object information.
[0037] The physical load estimated value F calculated by the information processing device 100 is a value indicating the magnitude of the force applied to the muscle of the body part or the magnitude of the moment applied to the joint of the body part. This allows users including the subject P to specifically grasp and evaluate the physical load applied to the body part of the subject P.
[0038] Furthermore, the information processing device 100 extracts a specific area image Vx including the object X from the image V, and acquires object information based on the specific area image Vx using the second machine learning model A2. This allows the information processing device 100 to acquire object information of the object X with high accuracy using the second machine learning model A2.
[0039] Furthermore, the information processing device 100 acquires the type candidates of the object X output by the second machine learning model A2 based on the specific area image Vx, and acquires the object information associated with the type candidates. This allows the information processing device 100 to acquire the object information of the object X by utilizing the class classification process of the second machine learning model A2.
[0040] Furthermore, when multiple type candidates are acquired, the information processing device 100 acquires the accuracy of each of the multiple type candidates and acquires object information associated with a type candidate whose accuracy is equal to or greater than a predetermined threshold. As a result, the information processing device 100 does not acquire object information of a type candidate whose accuracy is less than a predetermined threshold, and therefore can acquire highly reliable object information.
[0041] Furthermore, the information processing device 100 may acquire object information output by the second machine learning model A2 based on the specific area image Vx. This allows the information processing device 100 to acquire object information of the object X by using the regression process of the second machine learning model A2.
[0042] Furthermore, the physical load information output by the information processing device 100 includes information indicating an estimated value range Q of the estimated physical load F. This allows the user to confirm the estimated physical load F as a value having a certain degree of range.
[0043] Furthermore, the information processing device 100 calculates an estimate accuracy C, which is the accuracy of the physical load estimate F, and the physical load information includes information that displays, by color or graph, the distribution of the estimate accuracy C within the estimate range Q. This allows the user to intuitively recognize the distribution of the physical load estimate F and the estimate accuracy C.
[0044] Furthermore, when the estimated value range Q of the estimated physical load value F is wider than a predetermined range, the information processing device 100 outputs information prompting the user to manually input object information to the information display unit 2. As a result, when the estimated value range Q of the estimated physical load value F is wider than a predetermined range, the information processing device 100 determines that the reliability of the calculation result of the estimated physical load value F is lower than a predetermined value, and allows the user to manually input object information.
[0045] Second Embodiment Next, a second embodiment of the present invention will be described with reference to Fig. 6. Note that the configuration of an information processing device 100 according to the second embodiment is the same as that shown in Fig. 1. In the following description, the same reference numerals in the flowcharts indicate the same or similar processes, and detailed description thereof will be omitted. Note that the flowchart shown in Fig. 6 is obtained by adding the processes of steps S21 to S24 to the flowchart of the first embodiment shown in Fig. 5, and omitting the processes of steps S7 to S9.
[0046] As shown in FIG. 6, the processor 10 calculates the physical load estimate F for the object X in step S6, and then in step S21, determines whether or not there is another object that can be used for the same purpose as the object X in addition to the object X. Specifically, if the object X is a tool made by company A, the processor 10 determines whether or not a tool made by another company is included in the image V. Note that a case where a tool made by another company is included in the image V is a case where a tool made by another company (such as a tool made by company B or C) is placed next to the subject P who is working while holding the object X. Furthermore, the processor 10 may determine that "there is another object that can be used for the same purpose as the object X in addition to the object X" when acquiring information from a database (not shown) indicating that another object (for example, a tool made by another company) is stored near the subject P.
[0047] If the processor 10 determines in step S21 that "there is no other object other than object X that can be used for the same purpose as object X", the process proceeds to step S10 and the output unit 16 outputs the physical load information for object X. On the other hand, if it is determined in step S21 that "other than object X, there is another object that can be used for the same purpose as object X", in step S22, the processor 10 calculates a physical load estimate F' for the other object. Specifically, the processor 10 acquires object information of the other object, and calculates a physical load estimate F' assuming that the subject P is holding the other object, based on the estimated posture of the subject P and the object information of the other object. Note that the processor 10 may acquire the object information of the other object included in the image V using the second machine learning model A2, or may acquire the object information of the other object from a database (not shown).
[0048] Next, in step S24, the processor 10 estimates whether the physical load estimated value F' for the other object is lower than the physical load estimated value F for the object X. If the physical load estimated value F' for the other object is equal to or greater than the physical load estimated value F for the object X, the process proceeds to step S10, and the output unit 16 outputs the physical load information for the object X.
[0049] On the other hand, if it is determined in step S24 that "the physical load estimated value F' for the other object is lower than the physical load estimated value F for the object X," the information generator 15 generates information recommending that the user change from object X to another object. Then, in step S10, the output unit 16 outputs the physical load information about the object X and the information generated in step S24 to the information display unit 2.
[0050] The details of the processing in steps S21 to S24 in the case where there are a plurality of other objects that can be used for the same purpose as object X will be described below. First, if it is determined in step S21 that "there are multiple other objects that can be used for the same purpose as object X," in step S22, the processor 10 acquires object information for each other object, and calculates a physical load estimate F' for each other object based on the estimated posture of the subject P and the object information. Then, in step S23, the processor 10 compares the estimated physical load F for the object X with the estimated physical load F' for each of the other objects, and determines whether the estimated physical load F' of any of the multiple other objects is lower than the estimated physical load F of the object X. That is, in step S23, the processor 10 determines whether the estimated physical load F will be lowered by the subject P changing his / her grip from the object X to any one of the multiple other objects.
[0051] Then, if it is determined in step S23 that "the physical load estimated value F' of any of the multiple other objects is equal to or greater than the physical load estimated value F of object X," then in step S10, the processor 10 outputs physical load information about object X.
[0052] On the other hand, if it is determined in step S23 that "the estimated physical load F' of at least one of the multiple other objects is lower than the estimated physical load F of object X," that is, if the estimated physical load F will be lowered by subject P switching from object X to one of the multiple other objects, then in step S24, the information generator 15 selects the other object with the lowest estimated physical load F' as the recommended object to hold. Then, the information generator 15 generates information recommending switching object X to the recommended object to hold. Then, in step S10, the output unit 16 outputs the physical load information about the object X and the information generated in step S24 to the information display unit 2.
[0053] In the flowchart shown in FIG. 6, the processor 10 may execute the processes of steps S7 to S9, similarly to the flowchart according to the first embodiment shown in FIG.
[0054] From the above, when there is another object other than object X that can be used for the same purpose as object X, the information processing device 100 according to this embodiment judges whether or not the estimated physical load F will be lowered by subject P switching object X to the other object. Then, when the estimated physical load F will be lowered by subject P switching object X to the other object, the information processing device 100 outputs information recommending switching object X to the other object to the information display unit 2. As a result, subject P can reduce the physical load while achieving the purpose of using object X by switching object X to the other object based on the information output by the information processing device 100.
[0055] Furthermore, when there are multiple other objects and the estimated physical load F would be lowered by the subject P changing his / her holding object X to any one of the multiple other objects, the information processing device 100 selects the other object with the lowest estimated physical load F as the recommended object to hold, and outputs information recommending changing the holding of object X to the recommended object to the information display unit 2. As a result, even when there are multiple other objects, the subject P can reduce the physical load while achieving the purpose of using object X by changing his / her holding of object X to the recommended object to hold based on the information output by the information processing device 100.
[0056] Third Embodiment Next, a third embodiment of the present invention will be described with reference to Fig. 7. The configuration of an information processing device 100 according to the third embodiment is similar to that shown in Fig. 1. The flowchart shown in Fig. 7 is obtained by adding the processes of steps S31 and S32 to the flowchart of the first embodiment shown in Fig. 5 and omitting the processes of steps S7 to S9.
[0057] 7, the processor 10 calculates the estimated physical load F for the object X in step S6, and then in step S31 determines whether there is a recommended holding method that can reduce the estimated physical load F. Specifically, the processor 10 calculates the estimated physical load F when the subject P takes a posture other than the estimated posture. Alternatively, the processor 10 calculates the estimated physical load F when the subject P holds the object X with a body part other than the current specific body part (e.g., the right hand B1). That is, the processor 10 simulates a plurality of patterns of holding methods for the subject P to hold the object X, calculates the estimated physical load F for each pattern, and compares it with the current estimated physical load F calculated in step S6.
[0058] If the processor 10 determines in step S31 that there is no recommended maintenance method that can lower the estimated physical load value F, the process proceeds to step S10, and the output unit 16 outputs physical load information indicating the current estimated physical load value F.
[0059] On the other hand, if it is determined in step S31 that "there is a recommended maintenance method that can lower the estimated physical load value F", the information generating unit 15 generates information recommending the recommended maintenance method in step S32. Note that, if there are multiple maintenance methods that can lower the estimated physical load value F, the information generating unit 15 generates information recommending the maintenance method that results in the lowest estimated physical load value F as the recommended maintenance method. Then, in step S10, the output unit 16 outputs information recommending the recommended maintenance method to the information display unit 2 together with the physical load information.
[0060] In the flowchart shown in FIG. 7, similar to the flowchart according to the first embodiment shown in FIG. 5 and the flowchart according to the second embodiment shown in FIG. 6, the processor 10 may execute the processing of steps S7 to S9 and / or steps S21 to S24.
[0061] As described above, the information processing device 100 according to this embodiment determines whether there is a recommended holding method for the object X to make the estimated physical load F lower than the current value, and if there is a recommended holding method, outputs information recommending the recommended holding method to the information display unit 2. This allows the subject P to reduce the physical load by changing the holding method for the object X based on the information output by the information processing device 100.
[0062] 100...Information processing device 1...Image input section 2…Information display section 10…Processor 11...Object holding determination section 12… Estimated posture acquisition unit 13…Object information acquisition unit 14...Physical load estimation calculation section 16...Output section A1…First machine learning model A2: Second machine learning model C: Estimated value accuracy F: Estimated physical load Q: Estimated value range V…Image Vx…Specific image area X…object
Claims
1. An information processing method for estimating a physical load applied to a predetermined body part of a subject using a processor, comprising: The processor, determining whether the subject is holding a predetermined object; If the subject is holding the object, obtaining an estimated posture of the subject; acquiring object information of the object based on an image of the subject inputted to an image input unit; calculating a physical load estimate that is an estimate of a physical load applied to the body part based on the estimated posture and the object information; An information processing method, comprising: outputting physical load information indicating the estimated physical load value to an information display unit.
2. The information processing method according to claim 1 , wherein the processor obtains the estimated pose output by a first machine learning model based on the image.
3. The information processing method according to claim 1 , wherein the processor obtains the estimated posture based on detection signals of one or more sensors attached to a body of the subject.
4. The information processing method according to claim 1 , wherein the object information includes at least one of a mass, a center of gravity position, and a moment of inertia of the object.
5. The information processing method according to claim 1 , wherein the estimated body load value is a value indicating a magnitude of a force acting on a muscle of the body part, or a magnitude of a moment acting on a joint of the body part.
6. The processor, Extracting a specific area image including the object from the image; The information processing method according to claim 1 , further comprising: acquiring the object information based on the specific area image using a second machine learning model.
7. The processor, Obtaining a candidate type of the object output by the second machine learning model based on the specific area image; The information processing method according to claim 6 , further comprising acquiring the object information associated with the type candidate.
8. The processor, If multiple type candidates are obtained, Obtaining a probability of each of the plurality of type candidates; The information processing method according to claim 7 , further comprising acquiring the object information associated with the type candidate whose certainty is equal to or greater than a predetermined threshold value.
9. The processor, The information processing method according to claim 6 , further comprising acquiring the object information output by the second machine learning model based on the specific area image.
10. The information processing method according to claim 1 , wherein the physical load information includes information indicating an estimated value range of the estimated physical load value.
11. The processor calculates an estimate accuracy, which is an accuracy of the physical load estimate; The information processing method according to claim 10 , wherein the physical load information includes information that displays a distribution of the estimated value accuracy in the estimated value range using colors or a graph.
12. The information processing method according to claim 1 , wherein the processor outputs, when the estimated value range of the physical load estimated value is wider than a predetermined range, information prompting the user to manually input the object information to the information display unit.
13. The processor, determining whether there is another object other than the object that can be used for the same purpose as the object; If the other object is present, determining whether or not the estimated physical load will be reduced by the subject switching from the object to the other object; The information processing method according to claim 1 , further comprising: outputting, to the information display unit, information recommending that the subject change his / her grip from the object to the other object when the estimated physical load value decreases by changing the object to the other object.
14. The processor, When there are a plurality of other objects, if the subject changes the object to one of the plurality of other objects, the physical load estimation value becomes lower, selecting the other object having the lowest estimated physical load value as the object recommended to be held; The information processing method according to claim 13 , further comprising outputting information to the information display unit, the information recommending that the user change the object to the recommended object to be held.
15. The processor, determining whether there is a recommended way to hold the object to reduce the estimated physical load below a current value; The information processing method according to claim 1 , wherein, when the recommended storage method is available, information recommending the recommended storage method is output to the information display unit.
16. An information processing device that estimates a physical load applied to a predetermined body part of a subject, an object holding determination unit that determines whether the subject is holding a predetermined object; an estimated posture acquisition unit that acquires an estimated posture of the subject; an object information acquisition unit that acquires object information of the object based on an image of the subject inputted to an image input unit; a physical load estimation value calculation unit that calculates a physical load estimation value that is an estimate of a physical load applied to the body part based on the estimated posture and the object information; and an output unit that outputs physical load information indicating the estimated physical load value to an information display unit.
Citation Information
Patent Citations
Held object recognition device, held object recognition method and held object recognition program
JP2012133665A
Work management system and work management method
JP2021162967A
Information processing apparatus, information processing method, and program
JP2022030683A
Work support system
JP2022150491A
Fatigue estimation system, estimation device, and fatigue estimation method
WO2021112096A1
