A method for evaluating the degree of arm fatigue in an in-air interaction process and a related device

Abstract

The application belongs to a fatigue evaluation method, aiming at the technical problems that the existing arm fatigue evaluation method is difficult to capture the complete arm posture, the evaluation result is unreasonable, and it is difficult to run in real time, provides an arm fatigue degree evaluation method in air interaction process and related device, simplifies the arm into a two-section arm model including hand-small arm and large arm. According to the physical constraint of the arm model, the elbow joint candidate position is uniformly sampled on the quarter circle close to the ground at the current arm interaction position. For each elbow joint candidate position, the fatigue degree value is calculated according to the center position of the to-be-interacted AR object, the position of the shoulder joint, the mass center of the arm and the mass of the arm. Select the minimum value of the fatigue degree value of all candidate positions as the fatigue degree value at the current arm interaction position. The evaluation result is more reasonable, and accurate evaluation can be performed when the interaction position exceeds the reachable range of the arm.

Description

TECHNICAL FIELD

[0001] The application belongs to a fatigue evaluation method, and particularly relates to a hand arm fatigue degree evaluation method in an air interaction process and a related device. BACKGROUND

[0002] Researchers in the field of Human-Computer Interaction (HCI) have conducted a large number of researches in the past decade and proposed a variety of indexes for evaluating hand arm fatigue. However, these evaluation indexes are not suitable for guiding the design of UI (User Interface). The mainstream subjective evaluation indexes include Borg CR10 scale, Likert scale and NASA task load index. These methods do not require invasive operation and additional equipment, but require a large amount of preparation work, such as designing questionnaires and conducting user research, and can only provide rough estimates. Further, some researchers propose to use some data reflecting the physiological state of the user to measure fatigue to provide objective evaluation results, such as heart rate, electromyography (EMG) of muscle activation, etc. Although these devices provide accurate and objective evaluation results, these evaluation techniques rely on invasive data acquisition devices and may interfere with user interaction. Therefore, some non-invasive objective evaluation methods are proposed. For example, the Consumed Endurance (CE) index quantifies muscle fatigue by monitoring the joints of the user's hand arm. The CE index can estimate the shoulder torque, and then calculate the maximum time that the hand arm can maintain a certain posture before needing to rest. The calculation of CE relies on an RGB-D (Red, Green, Blue, Depth) camera to track the hand arm posture, and is limited by the field of view angle. Complete hand arm posture is often difficult to capture completely by augmented reality devices. At the same time, when the interaction position is beyond the user's reach or higher than the height of the shoulder joint, the method based on shoulder torque will provide unreasonable results. In addition, other evaluation methods, such as approximating the estimation of hand arm torque by estimating muscle state (active, rest, fatigue), and predicting muscle activation in motion by using static optimization, are usually difficult to run in real time. SUMMARY

[0003] The present application provides a hand arm fatigue degree evaluation method in an air interaction process and a related device to solve the technical problems that the existing evaluation methods for hand arm fatigue are difficult to capture complete hand arm posture, the evaluation results are unreasonable, and it is difficult to run in real time.

[0004] To achieve the above purpose, the application adopts the following technical solutions:

[0005] In a first aspect, the application provides a method for evaluating the fatigue degree of an arm in an air interaction process, comprising:

[0006] simplifying the arm into a two-segment arm model comprising a hand-forearm and a forearm;

[0007] according to the physical constraints of the arm model, uniformly sampling candidate positions of an elbow joint on a quarter of the circumference close to the ground at the current arm interaction position;

[0008] for each candidate position of the elbow joint, calculating a fatigue degree value according to the center position of an AR object to be interacted, the position of a shoulder joint, the mass center of the arm and the mass of the arm;

[0009] selecting the minimum value of the fatigue degree values of all candidate positions as the fatigue degree value at the current arm interaction position.

[0010] Further, the method for calculating the mass center of the arm and the mass of the arm comprises:

[0011] calculating the mass of the arm and the mass center of the arm according to the mass distribution and relative mass center of each part of the human body.

[0012] Further, the method for calculating the mass center of the arm further comprises:

[0013] when the arm interaction position is beyond the reach of the arm, considering that the arm is in a straightened state, and taking the distance from the shoulder joint to the shoulder joint on the line connecting the shoulder joint and the interaction position as the candidate position of the elbow joint;

[0014] when the arm interaction position is within the reach of the arm or beyond the reach of the arm, the mass center of the arm is calculated by the following formula:

[0015]

[0016] wherein, represents the mass center of the arm, represents the mass of the arm, represents the mass center position of the forearm, represents the mass of the forearm, represents the mass center position of the hand-forearm, represents the mass of the hand-forearm, represents the mass center position of the palm, represents the mass of the palm, represents the distance from the interaction position to the position of the shoulder joint, represents the length of the arm, represents the position of the shoulder joint, represents the directional mass of the shoulder joint to the interaction position, represents the relative position ratio of the mass center to the shoulder joint when the arm is completely straightened.

[0017] Further, the method of calculating the fatigue degree value comprises:

[0018]

[0019] wherein, represents the center position of the AR object to be interacted, represents the gravity moment of the current arm with the shoulder joint as the pointing point, represents the maximum moment that the arm can bear.

[0020] Further, the maximum moment that the arm can bear is calculated by the method comprising:

[0021]

[0022] wherein, represents the maximum force that the arm can bear.

[0023] Further, the maximum force that the arm can bear is determined by:

[0024] For a male, = 101.6 N; and for a female, = 87.2 N.

[0025] Further, the gravity moment of the current arm with the shoulder joint as the pointing point is calculated by the method comprising:

[0026]

[0027] wherein, represents the angle between and the direction of gravity, represents the gravity acceleration.

[0028] In a second aspect, the application provides an arm fatigue degree evaluation system in an aerial interaction process, comprising:

[0029] a modeling module, configured to simplify the arm into a two-segment arm model comprising a hand-forearm and a large arm;

[0030] a candidate module, configured to uniformly sample on a quarter of the circumference close to the ground at the current arm interaction position as the candidate position of the elbow joint according to the physical constraints of the arm model;

[0031] a calculation module, configured to calculate the fatigue degree value according to the center position of the AR object to be interacted, the position of the shoulder joint, the mass center of the arm and the mass of the arm for each candidate position of the elbow joint, respectively;

[0032] The evaluation module is configured to select the minimum value of the fatigue degree values of all candidate positions as the fatigue degree value of the current arm interaction position.

[0033] In a third aspect, the present application provides an electronic device, comprising a memory and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program codes, the computer program codes comprise computer instructions, when the computer instructions are executed by the processor, the electronic device executes the steps of the arm fatigue degree evaluation method in the air interaction process.

[0034] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, when the computer program is executed by a processor, the steps of the arm fatigue degree evaluation method in the air interaction process are implemented.

[0035] Compared with the prior art, the present application has the following beneficial effects:

[0036] The present application provides an arm fatigue degree evaluation method in an air interaction process, simplifies the arm into a two-segment arm model comprising a hand-forearm and an upper arm, determines candidate positions of elbow joints according to physical constraints of the arm model, calculates fatigue degree values for each candidate position of the elbow joint, and selects the minimum value of the fatigue degree values of all candidate positions as the fatigue degree value of the current arm interaction position. The present application can be evaluated in real time, adjusted with user movement, and provides guidance for adaptive UI design. Even if the user moves in the environment, real-time evaluation results can be provided. Compared with the prior art, external tracking devices or other invasive devices are not required in the evaluation process. And through actual verification, compared with the prior art, the evaluation method of the present application provides more reasonable evaluation results, and accurate evaluation can be performed when the interaction position exceeds the reachable range of the arm.

[0037] The present application also provides an arm fatigue degree evaluation system in an air interaction process, an electronic device and a computer storage medium, which have all the advantages of the arm fatigue degree evaluation method in the air interaction process. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0039] Figure 1The first flowchart of the arm fatigue degree evaluation method in the air interaction process of the application;

[0040] Figure 2 The second flowchart of the arm fatigue degree evaluation method in the air interaction process of the application;

[0041] Figure 3 The schematic diagram of the two-section arm model in the embodiment of the application;

[0042] Figure 4 The schematic diagram of the mass and center of mass distribution of the upper arm, lower arm and palm in the embodiment of the application;

[0043] Figure 5 The arm schematic diagram when the interaction position is higher than the shoulder joint plane;

[0044] Figure 6 The evaluation result schematic diagram of the CE index when the interaction position is higher than the shoulder joint plane;

[0045] Figure 7 The evaluation result schematic diagram of the ECE when the interaction position is higher than the shoulder joint plane;

[0046] Figure 8 The comparison diagram of the evaluation results of the CE index and the ECE when the interaction position is higher than the shoulder joint plane;

[0047] Figure 9 The correlation analysis result schematic diagram of the ECE and the Borg CR 10 in the air click task;

[0048] Figure 10 The schematic diagram of the arm fatigue degree evaluation system in the air interaction process of the application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical scheme and advantages of the embodiments of the application clearer, the technical scheme of the embodiments of the application will be described clearly and completely below in combination with the drawings in the embodiments of the application. Obviously, the described embodiments are some embodiments of the application, rather than all the embodiments. The components of the embodiments of the application described and shown in the drawings can be arranged and designed in various different configurations.

[0050] Therefore, the following detailed description of the embodiments of the application provided in the drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the application.

[0051] It should be noted that like reference numerals and characters refer to like items throughout the drawings, and once an item is defined in one drawing, further definition and explanation of the item in subsequent drawings is not required.

[0052] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "horizontal", "inner" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the application is usually placed, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", and the like are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0053] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly inclined. For example, "horizontal" only means that its direction is relatively more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly inclined.

[0054] In the description of the embodiments of the present application, it should also be noted that unless otherwise explicitly specified and limited, if the terms "arrangement", "installation", "connection", "connection" appear, they should be understood in a broad sense, for example, they can be fixedly connected, or can be detachably connected, or integrally connected; can be mechanically connected, or can be electrically connected; can be directly connected, or indirectly connected through an intermediate medium; can be connected inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0055] With the deepening of research on computer vision and graphics, and the popularity of commercial augmented reality devices, augmented reality has been widely studied and gradually applied to medical, educational, industrial design and other industries. The growth of applications has also brought huge development needs. How to design the most optimized augmented reality user interface for user experience has become a hot research topic. Unlike applications based on two-dimensional display devices such as mobile phones and desktops, in augmented reality applications, the interaction paradigm between users and systems has undergone a huge change, from traditional mouse, touch screen input to three-dimensional interaction relying on line of sight, gestures and other information. Among these interaction methods, bare-handed mid-air interaction has become the mainstream interaction method. Under this interaction paradigm, one of the main concerns of user interface design is how to minimize user arm fatigue to the greatest extent. The most basic problem is to quantify the fatigue that a certain point in space as an interaction position will bring to the user. The indicators that can provide guidance for developers or users should at least have the following three characteristics:

[0056] 1)The evaluation of fatigue at any point in space as an interaction location for the user. In the augmented reality environment, users can access interaction objects at any location in space through touch-based interaction and ray-based interaction. Therefore, the definition domain of this evaluation index should be the entire space.

[0057] 2)Convenient calculation. In the augmented reality environment, the user interface often needs to adaptively move according to the user's location, and only a computationally efficient index can provide real-time UI design guidance.

[0058] 3)Less dependent on equipment and does not affect the user's interaction experience. Users often need to move in the scene and perform tasks while wearing augmented reality equipment, so the data input equipment required for evaluation should not affect the user's interaction experience.

[0059] The CE (Consumed Endurance) index is a non-invasive objective evaluation method designed to quantify muscle fatigue and consumed endurance by monitoring the user's arm joint activity. This method estimates the shoulder torque and then calculates the maximum time the arm can maintain a certain posture before needing a rest. This evaluation method is important for understanding ergonomics, optimizing human-computer interaction design, and preventing muscle fatigue. The calculation of the CE index mainly relies on the tracking of arm posture by an RGB-D camera. The RGB-D camera can simultaneously acquire color images and depth information, enabling accurate perception of three-dimensional space. By tracking the position and angle changes of the arm joints, the system can estimate the shoulder torque, which is the torque generated by the muscles to maintain the arm posture. Based on the estimation of shoulder torque, the system can further calculate the endurance consumption of the arm in a given posture. However, there are the following limitations:

[0060] (1) The estimation of muscle state depends on multiple factors such as electromyographic signals and muscle morphology. However, the acquisition and processing of these factors are relatively complex and are easily affected by noise and individual differences, resulting in limited accuracy of the estimation results.

[0061] (2) Static optimization methods usually calculate muscle activation based on certain assumptions and simplified models. However, in actual movements, the working mode and torque distribution of muscles change, making it difficult for static optimization methods to accurately predict muscle activation during movements. In addition, the calculation process requires processing a large amount of data, making it difficult to meet real-time requirements.

[0062] (3) The CE index and similar evaluation indexes require external tracking devices to obtain complete arm joint positions during the evaluation process.

[0063] Based on the above situation, the application proposes a method for evaluating the degree of arm fatigue in an air interaction process and related devices. The application will be described in detail below in conjunction with the embodiments and drawings.

[0064] As shown in Figure 1 , it is the first flowchart of the method for evaluating the degree of arm fatigue in the air interaction process of the application, which can include:

[0065] S101, the arm is simplified into a two-segment arm model including hand-forearm and upper arm.

[0066] In the air interaction process, the movement of the arm is complex and multi-dimensional. In order to simplify the calculation and improve the evaluation efficiency, we simplify the arm into a two-segment model, i.e. hand-forearm and upper arm. This simplification is based on the structure of human anatomy, the arm is mainly connected by shoulder joint, elbow joint and wrist joint, among which the shoulder joint connects the upper arm and the trunk, the elbow joint connects the upper arm and the forearm, and the wrist joint connects the forearm and the hand. In the two-segment arm model, the hand-forearm part includes palm, wrist and forearm bones and muscles, responsible for fine movements of the hand and grasping of objects. The upper arm part includes the upper arm bones and muscles, mainly responsible for the swing of the arm and the transmission of force.

[0067] S102, according to the physical constraints of the arm model, uniformly sample on the quarter circle near the ground at the current arm interaction position as the candidate position of the elbow joint.

[0068] It should be noted that the movement of the arm is subject to the constraints of human anatomy and biomechanics, such as the range of joint movement, muscle strength and endurance, etc. In the air interaction process, the arm needs to maintain a certain stability and flexibility to ensure the accuracy and comfort of the interaction. Choosing the quarter circle near the ground as the sampling range is because in the air interaction, the arm is usually in front of the body or on the side, and the position of the elbow joint is relatively low, and uniform sampling in this range can get a series of candidate positions of the elbow joint. The number of sampling points can be determined according to the accuracy of the evaluation and the calculation efficiency.

[0069] In practical application, each candidate position represents a possible arm posture, and different postures correspond to different muscle strength and endurance requirements. By evaluating the fatigue degree of different candidate positions, the optimal arm posture can be selected to reduce muscle fatigue and improve interaction efficiency.

[0070] S103, for each candidate position of the elbow joint, respectively calculate the fatigue degree value according to the center position of the AR object to be interacted, the position of the shoulder joint, the mass center of the arm and the mass of the arm.

[0071] The center position of the AR object to be interacted determines the target position that the arm needs to reach. The position of the shoulder joint is the starting point of the arm movement and the basis for calculating the arm posture and muscle strength. The center of mass and mass of the arm affect the inertial force and power demand of the arm during movement.

[0072] For each candidate position of the elbow joint, the vector chain of the arm from the shoulder joint to the center of mass and then to the center position of the AR object to be interacted can be calculated, and then the muscle strength and endurance required by the arm during movement can be calculated. The fatigue degree value calculated reflects the degree of fatigue that the arm may produce after maintaining a certain posture for a certain time. The larger the value, the more likely the arm is to fatigue in that posture; the smaller the value, the less likely the arm is to fatigue in that posture.

[0073] S104, selects the minimum value of the fatigue degree value of all candidate positions as the fatigue degree value at the current arm interaction position.

[0074] Among all candidate positions, the position with the minimum fatigue degree value represents the optimal arm posture. Selecting the optimal arm posture can reduce muscle fatigue, improve interaction efficiency and comfort. In practical applications, the minimum fatigue degree value can be provided as feedback information to the user to guide the user to adjust the arm posture to reduce fatigue. Therefore, the evaluation method can be integrated into the AR interaction system to automatically adjust the interaction interface and object position to adapt to the user's arm posture and fatigue degree.

[0075] The arm fatigue degree evaluation method in the air interaction process of the present application achieves objective evaluation of the arm fatigue degree through steps such as simplifying the arm model, sampling the candidate positions of the elbow joint, calculating the fatigue degree value, and selecting the minimum value. This method can provide strong support for the design and optimization of the AR interaction system and improve the user's interaction experience and comfort.

[0076] As shown in Figure 2 , it is a second flowchart of the arm fatigue degree evaluation method in the air interaction process of the present application, which can include:

[0077] S201, estimate the posture of the arm.

[0078] As shown in Figure 3 , it is a schematic diagram of the two-segment arm model of the present application. In the present application, the arm is simplified into a two-segment arm model of hand-forearm (A1 segment) and upper arm (A2 segment). When the shoulder joint and finger tip positions are fixed, the elbow moves only on a circumference. According to the physical constraints of the arm model, candidate positions of the elbow joint are uniformly sampled on the quarter circumference outside the body near the ground. When the interaction position is beyond the reach of the arm, it is assumed that the user's arm is straightened and points to the interaction position, and the elbow joint falls in the direction of the shoulder joint pointing to the interaction position, with a distance of the length of the upper arm from the shoulder joint.

[0079] S202, estimate the center of mass position of the arm.

[0080] After obtaining the candidate position of the elbow joint, the mass and the center of mass of the arm can be calculated according to the mass distribution of each part of the human body and the relative center of mass. As shown in Table 1, the relative mass and the relative center of mass of the arm are shown. As shown in Figure 4 Fig. 1 is a diagram of a mass and center of mass distribution of a large arm, a small arm and a palm.

[0081] Table 1 Mass and center of mass distribution of a large arm, a small arm and a palm

[0082]

[0083] When the interaction position is beyond the reach of the arm, the center of mass of the arm can be corrected and supplemented. The center of mass of the arm can be calculated by the following formula when the arm interaction position is within the reach of the arm or beyond the reach of the arm:

[0084]

[0085] wherein, represents the center of mass of the arm, represents the mass of the arm, represents the center of mass position of the large arm, represents the mass of the large arm, represents the center of mass position of the small arm, represents the mass of the small arm, represents the center of mass position of the palm, represents the mass of the palm, represents the distance from the interaction position to the position of the shoulder joint, represents the length of the arm, represents the position of the shoulder joint, represents the direction mass of the shoulder joint to the interaction position, represents the relative position ratio of the center of mass to the shoulder joint when the arm is fully stretched.

[0086] S203, calculate the fatigue degree value ECE.

[0087] The calculation method of the fatigue degree value ECE is as follows:

[0088]

[0089] wherein, represents the center position of the AR object to be interacted, represents the gravity moment of the current arm with the shoulder joint as the pointing position, represents the maximum moment that the arm can bear.

[0090] It should be noted that, You can use the maximum force that your arm can withstand. Make an estimate. As an example, for men, the maximum force Can be taken For women, the greatest strength Can be taken .

[0091] Specifically, The calculation method can be the following formula:

[0092]

[0093] The calculation method is as follows:

[0094]

[0095] in, express The angle with the direction of gravity (vertically downwards), Represents gravitational acceleration, with a value of .

[0096] For each candidate location of the elbow joint, its ECE is calculated. Then, the candidate location with the lowest ECE value is selected as the estimated value of the elbow joint, and its ECE value is used as the fatigue level value at that interaction location.

[0097] The domain defined in this application is the entire space. When the interaction position exceeds the reach of the arm, the arm posture is pre-estimated. The arm is assumed to be straight, and the center of mass position is corrected. With this definition, ECE can provide the degree of arm fatigue caused by any point in space as the interaction position. Since the evaluation method in this application can be calculated in real time, it can provide guidance for the design of adaptive UIs that adjust as the user moves. Even if the user moves in the environment, ECE can provide real-time evaluation results. No external tracking or other intrusive devices are required. Existing augmented reality devices, such as HoloLens and Vision Pro, can track hand joints in real time. Using these hand joints, the position of the elbow joint can be estimated in reverse, and then the arm torque can be calculated to obtain the evaluation result. The final evaluation result is more reasonable than the CE metric and shows a high correlation with the mainstream Borg CR10 metric. Figure 5 The image shows a schematic diagram of the arm when the interaction position is above the shoulder joint plane. Figure 6 The image shows a schematic diagram illustrating the evaluation results of the CE index when the interaction position is higher than the shoulder joint plane. Figure 7 The image shows a schematic diagram of ECE evaluation results when the interaction position is above the shoulder joint plane. Figure 8As shown, it is a comparison chart of CE index and ECE when the interaction position is higher than the shoulder joint plane. From Figures 6 to 8 It can be seen that when the interaction position is higher than the shoulder joint plane, the CE index starts to rise and then falls, indicating that the fatigue degree of the arm first rises and then falls; when the interaction position is above the head, the CE index falls to the same as when the arm is naturally drooping, which does not conform to the subjective feeling of a person, and the ECE index rises as the interaction position becomes higher. In order to verify, the application also designs the task of clicking in the air, and the Borg CR 10 evaluation results of the user are obtained through the questionnaire form. As shown in Figure 9 , it is a diagram showing the correlation analysis results of ECE and Borg CR 10 in the air clicking task. It is shown through analysis that ECE and Borg CR 10 index present a high correlation.

[0098] As shown in Figure 10 , it is a diagram of an arm fatigue degree evaluation system in the air interaction process of the application, which can include:

[0099] A modeling module for simplifying the arm into a two-segment arm model including a hand-forearm and a large arm;

[0100] A candidate module for uniformly sampling on a quarter of the circumference close to the ground as a candidate position of the elbow joint according to the physical constraints of the arm model at the current arm interaction position;

[0101] A calculation module for calculating a fatigue degree value according to the center position of the to-be-interacted AR object, the position of the shoulder joint, the mass center of the arm and the mass of the arm, respectively, for each candidate position of the elbow joint;

[0102] An evaluation module for selecting the minimum value of the fatigue degree values of all candidate positions as the fatigue degree value at the current arm interaction position.

[0103] It should be noted that in the several embodiments provided by the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, multiple modules can be combined or integrated into another device, or some features can be ignored or not executed. The modules described as separate components can be or can not be physically separated, and the components displayed as modules can be one physical unit or multiple physical units, that is, they can be located in one place, or also can be distributed to multiple different places. According to actual needs, part or all of the modules can be selected to achieve the purpose of the embodiment scheme.

[0104] In addition, each module in various embodiments of the present application can be integrated in one processing unit, or each module can be physically present separately, or two or more modules can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0105] Embodiments of the present application also provide an electronic device, which can include one or more processors, memories and communication interfaces.

[0106] The memory, the communication interface and the processor are coupled together, for example, through a bus.

[0107] The communication interface is configured to perform data transmission with other devices. The memory stores computer program codes. The computer program codes include computer instructions, which, when executed by the processor, cause the electronic device to perform the steps of the hand fatigue degree evaluation method in the air interaction process.

[0108] The processor can be a processor or a controller, for example, a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in combination with the present disclosure. The processor can also be a combination of computing functions, such as one or more microprocessor combinations, combinations of DSP and microprocessor, etc. The processor can be used to support the electronic device to perform the method steps provided in the above embodiments.

[0109] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0110] The embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program.

[0111] The computer readable storage medium involved in the present application includes random access memory (RAM), memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the technical field.

[0112] The above is only the preferred embodiment of the present application and is not used to limit the present application, and the present application can have various changes and variations for those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for evaluating the degree of arm fatigue in an in-flight interaction process, characterized in that, The method comprises the following steps: simplifying the arm into a two-segment arm model comprising a hand-lower arm and an upper arm; uniformly sampling on a quarter of the circumference close to the ground at the current arm interaction position as candidate positions of the elbow joint according to the physical constraints of the arm model; calculating a fatigue degree value for each candidate position of the elbow joint according to the center position of the AR object to be interacted, the position of the shoulder joint, the mass center of the arm, and the mass of the arm; the method for calculating the mass center of the arm and the mass of the arm comprises: calculating the mass of the arm and the mass center of the arm according to the mass distribution and relative mass center of each part of the human body; when the arm interaction position is beyond the reach of the arm, the arm is considered to be in a straight state, and the distance from the shoulder joint to the shoulder joint on the line connecting the shoulder joint and the interaction position is taken as the length of the upper arm to serve as the candidate position of the elbow joint; when the arm interaction position is within the reach of the arm or beyond the reach of the arm, the mass center of the arm is calculated by the following formula: wherein, represents the center of mass of the arm, represents the mass of the arm, represents the center of mass position of the upper arm, represents the mass of the upper arm, represents the center of mass position of the lower arm, represents the mass of the lower arm, represents the center of mass position of the hand, represents the mass of the hand, represents the distance from the interaction position to the shoulder joint position, represents the length of the arm, represents the position of the shoulder joint, represents the directional mass from the shoulder joint to the interaction position, represents the relative position proportion of the center of mass to the shoulder joint when the arm is fully extended; the method for calculating the fatigue degree value comprises: wherein, represents the center position of the AR object to be interacted, represents the gravity moment of the current arm with the shoulder joint as the fulcrum, represents the maximum moment that the arm can bear; selecting the minimum value of the fatigue degree values of all candidate positions as the fatigue degree value at the current arm interaction position.

2. The method of claim 1, wherein the degree of arm fatigue is evaluated during the in-flight interaction process. the maximum torque that the arm can withstand a method of calculating, comprising: wherein, represents the maximum force the arm can withstand.

3. The method of claim 2, wherein the degree of arm fatigue is evaluated during the air interaction process. the maximum force that the arm can withstand the value of the maximum force that the arm can withstand For men, = 101.6 N; for women, = 87.2 N.

4. The method of claim 3, wherein the degree of arm fatigue is evaluated during the air interaction process. the current arm's gravity moment about the shoulder joint a method of calculating, comprising: wherein represents an angle of the direction of gravity, represents the acceleration of gravity.

5. A system for assessing the degree of arm fatigue in an in-flight interaction process, characterized by The method comprises the following steps: a modeling module for simplifying the arm into a two-segment arm model comprising a hand-lower arm and an upper arm; a candidate module for uniformly sampling on a quarter of the circumference close to the ground at the current arm interaction position as candidate positions of the elbow joint according to the physical constraints of the arm model; a calculation module for calculating a fatigue degree value for each candidate position of the elbow joint according to the center position of the AR object to be interacted, the position of the shoulder joint, the mass center of the arm, and the mass of the arm; the method for calculating the mass center of the arm and the mass of the arm comprises: calculating the mass of the arm and the mass center of the arm according to the mass distribution and relative mass center of each part of the human body; when the arm interaction position is beyond the reach of the arm, the arm is considered to be in a straight state, and the distance from the shoulder joint to the shoulder joint on the line connecting the shoulder joint and the interaction position is taken as the length of the upper arm to serve as the candidate position of the elbow joint; when the arm interaction position is within the reach of the arm or beyond the reach of the arm, the mass center of the arm is calculated by the following formula: wherein, represents the center of mass of the arm, represents the mass of the arm, represents the center of mass position of the upper arm, represents the mass of the upper arm, represents the center of mass position of the lower arm, represents the mass of the lower arm, represents the center of mass position of the hand, represents the mass of the hand, represents the distance from the interaction position to the shoulder joint position, represents the length of the arm, represents the position of the shoulder joint, represents the directional mass of the shoulder joint to the interaction position, represents the relative position proportion of the center of mass to the shoulder joint when the arm is fully extended; the method for calculating the fatigue degree value comprises: wherein, represents the center position of the AR object to be interacted, represents the gravity moment of the current arm with the shoulder joint as the fulcrum, represents the maximum moment that the arm can bear; an evaluation module for selecting the minimum value of the fatigue degree values of all candidate positions as the fatigue degree value at the current arm interaction position.

6. An electronic device, comprising: The method comprises the following steps: a memory and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program code, the computer program code comprises computer instructions, when the computer instructions are executed by the processor, the electronic device executes the steps of the arm fatigue degree evaluation method in the air interaction process as claimed in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium, and the computer program is executed by the processor to realize the steps of the arm fatigue degree evaluation method in the air interaction process as claimed in any one of claims 1-4.

Images