System and method for determining impact on body part and recommending exercise

Abstract

A method for determining an impact on a body part and recommending exercise when using a user equipment (UE) is disclosed. The method includes receiving inertial sensor data and touch screen panel data and a type of application running on the UE. The method includes predicting a holding direction of a UE through a neural network. The method includes determining a body posture and an affected body part of the user in response to predicting that the user is holding and currently operating the UE. The method includes determining an influence level of an affected body part based on the body posture, a grip direction of the UE, and inertial sensor data of the UE, and recommending body posture corrections and exercises for the affected body part.

Description

Technical Field

[0001] The present invention relates generally to recommending exercises, and more particularly, to systems and methods for determining the impact on a body part while using a user equipment (UE) and recommending exercises. Background Art

[0002] Electronic devices have become an important element in human life. Everyday activities revolve around and are completed using electronic devices. In particular, mobile phones have become full-time companions for users. Users are constantly interacting with their mobile phones and often neglect their health while using their mobile phones.

[0003] Excessive use of mobile phones and incorrect body posture of users while using mobile phones may lead to various health hazards (e.g., text neck, etc.). Incorrect body posture (e.g., flexed posture resulting in bending of the neck, head, spine) may affect the body parts in terms of pain, life fluctuations, mental and overall physical problems.

[0004] Figure 1 A scene of body posture when using a user device (preferably a mobile phone) according to the prior art is shown. As shown in the figure, the user may bend the neck and spine when holding and using the mobile phone. Depending on the angle at which the mobile phone is held, the user may bend his body posture accordingly, and therefore may be at risk of excessive physical stress on the body due to incorrect posture. In addition, using the mobile phone for a long time with a bent body part causing incorrect body posture may cause acute or chronic pain in different parts of the body, affect metabolic activity, increase the chance of heart attack, and other serious physical and mental diseases.

[0005] Currently, there are several mechanisms available for correcting a user's body posture and recommending exercises to eliminate body stress. However, most of the prior art lacks real-time, easy analysis of body posture and recommended exercises to eliminate body stress. The prior art discloses the use of multiple wearable devices to capture body posture, which makes the technology quite cumbersome for users. Alternatively, the prior art suggests using a camera to capture body posture. However, the camera of a mobile phone may not be active at every moment and cannot continuously monitor body posture.

[0006] Therefore, there is a need for a method that can determine the impact on body parts when using a mobile phone and recommend exercises to remove any physical stress. Summary of the invention

[0007] Solution to the problem

[0008] This summary is provided to introduce some concepts in a simplified format that are further described in the detailed description of the invention. This summary is neither intended to identify key or essential inventive concepts of the invention nor to determine the scope of the invention.

[0009] According to one embodiment of the present disclosure, a method for determining the impact on at least one body part when using a user equipment (UE) and recommending at least one exercise. The method includes: receiving inertial sensor data and touch screen panel data of the user equipment (UE). The method includes: determining the type of application running on the UE. The method includes: predicting the holding direction of the UE based on the inertial sensor data, the application type and the touch screen panel data through a neural network, wherein the holding direction of the UE indicates whether the user is holding and currently operating the UE. The method includes: in response to predicting that the user is holding and currently operating the UE, determining the user's body posture and at least one affected body part based on the inertial sensor data through a neural network. The method includes: determining the impact level of at least one affected body part based on the body posture, the holding direction of the UE and the inertial sensor data of the UE through a neural network. The method includes: recommending body posture correction and at least one exercise for the affected body part based on the impact level.

[0010] According to one embodiment of the present disclosure, a system for determining the impact on at least one body part when using a user equipment (UE) and recommending at least one exercise. The system includes a processor configured to receive inertial sensor data and touch screen panel data of a user equipment (UE). The system includes a processor configured to determine the type of application running on the UE. The system includes a processor configured to predict the holding direction of the UE based on the inertial sensor data, the application type and the touch screen panel data through a neural network, wherein the holding direction of the UE indicates whether the user is holding and currently operating the UE. The system includes a processor configured to determine the user's body posture and at least one affected body part based on the inertial sensor data through a neural network in response to predicting that the user is holding and currently operating the UE. The system includes a processor configured to determine the impact level of at least one affected body part based on the body posture, the holding direction of the UE and the inertial sensor data of the UE through a neural network. The system includes a processor configured to recommend body posture correction and at least one exercise for the affected body part based on the impact level.

[0011] To further illustrate the advantages and features of the present invention, a more specific description of the present invention will be presented by reference to the specific embodiments thereof shown in the accompanying drawings. It should be recognized that these drawings only depict typical embodiments of the present invention and therefore should not be considered to limit its scope. The present invention will be described and explained with additional features and details and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] These and other features, aspects and advantages of the present invention will become more readily understood when the following detailed description is read with reference to the accompanying drawings, wherein like reference numerals refer to like parts throughout the drawings, and wherein:

[0013] Figure 1 An exemplary scenario of body posture when using a user equipment (UE) according to the prior art is shown;

[0014] Figure 2 A schematic block diagram depicting an environment for implementing the present invention is shown according to an embodiment of the present invention;

[0015] Figure 3a and Figure 3b An exemplary process flow including a method for determining the impact on a body part when using a UE and recommending exercise according to an embodiment of the present invention is shown;

[0016] Figure 4a to Figure 4b An exemplary processing flow including a method for predicting a holding direction of a UE according to an embodiment of the present invention is shown;

[0017] Figure 5 An exemplary process flow including a method for determining body posture according to an embodiment of the present invention is shown;

[0018] Figure 6 An exemplary predefined table for determining affected body parts according to an embodiment of the present invention is shown;

[0019] Figure 7a to Figure 7b An exemplary process flow including a method for determining an impact level of an affected body part according to an embodiment of the present invention is shown;

[0020] Figure 8a An exemplary use case depicting providing a posture record of a body posture and affected body parts according to an embodiment of the present invention is shown;

[0021] Figure 8b An exemplary use case depicting a predefined gesture record table providing predefined gestures according to an embodiment of the present invention is shown;

[0022] Figure 9aAnother exemplary use case of depicting affected body parts and impact levels according to an embodiment of the present invention is shown;

[0023] Figure 9b Another exemplary use case of depicting affected body parts and exercise types according to an embodiment of the present invention is shown;

[0024] Fig.10 An exemplary process flow including a method for receiving feedback from a user and modifying a recommended exercise according to an embodiment of the present invention is shown;

[0025] Fig.11 shows a system architecture according to an embodiment of the present disclosure; and

[0026] Fig.12 Another system architecture according to an embodiment of the present disclosure is shown.

[0027] Furthermore, those skilled in the art will recognize that elements in the drawings are illustrated for the sake of brevity and may not necessarily be drawn to scale. For example, a flow chart illustrates a method in terms of the most salient steps involved to help improve understanding of the various aspects of the present invention. Furthermore, with respect to the construction of the device, one or more components of the device may have been represented in the drawings by conventional symbols, and the drawings may show only those specific details relevant to understanding the embodiments of the present invention so as not to obscure the drawings with details that would be apparent to one of ordinary skill in the art having the benefit of the description herein. DETAILED DESCRIPTION

[0028] For the purpose of promoting an understanding of the principles of the invention, reference will now be made to various embodiments and specific language will be used to describe the same. It will be understood, however, that this is not intended to limit the scope of the invention and that such changes and further modifications in the illustrated systems, and such further applications of the principles of the invention illustrated therein, would normally occur to one skilled in the art to which the invention relates.

[0029] Those skilled in the art will understand that the foregoing general description and the following detailed description are illustrative of the present invention and are not intended to be restrictive thereof.

[0030] References throughout this specification to "one aspect," "another aspect," or similar language indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the phrases "in an embodiment," "in another embodiment," and similar language appearing throughout this specification may (but do not necessarily) all refer to the same embodiment.

[0031] The terms "comprises", "comprising", or any other variation thereof are intended to cover a non-exclusive inclusion such that a process or method that includes a list of steps may include not only those steps but also other steps not expressly listed or inherent to such process or method. Similarly, one or more devices or subsystems or elements or structures or components preceded by "comprises..." do not exclude the presence of other devices or other subsystems or other elements or other structures or other components or additional devices or additional subsystems or additional elements or additional structures or additional components, without further constraints.

[0032] The present invention relates to a method and system for determining the impact on a body part when using a user equipment (UE) and recommending exercises for the affected body part. Examples of UE may be a laptop computer, a mobile phone, a PDA (personal digital assistant), a smart phone, a multimedia device, a wearable device, etc. More specifically, the present invention provides a mechanism for determining an incorrect body posture and determining the body part affected by the incorrect body posture while the user is using the UE, and recommending exercises to correct the body posture and the impact on the body part.

[0033] Figure 2 A schematic block diagram depicting an environment for implementing the present invention is shown according to an embodiment of the present invention.

[0034] In some embodiments, the present invention is implemented between a UE 202 (eg, but not limited to, a laptop, desktop computer, personal computer (PC), notebook, smart phone, tablet, smart watch, e-book reader) and a user 204 holding and operating the UE 202 .

[0035] In various embodiments of the present invention, UE 202 is configured to acquire inertial sensor data, touch screen panel data, and determine the type of application running on UE 202. In an example, inertial sensor data can be collected via an accelerometer and a gyroscope installed in UE 202. The accelerometer and the gyroscope can provide the use angle of UE 202, the use duration of UE 202, and the proximity of UE 202 to the face of user 204. In an example, the touch screen panel data can be collected via touch coordinates and hover distribution as the user 204 interacts with the touch display of UE 202. In another example, UE 202 can determine the type of application running on UE 202 (e.g., video, game, call, chat, etc.).

[0036] In some embodiments, the UE 202 is configured to display the affected body parts of the user due to the incorrect posture of the user 204 when the user 204 is holding and operating the UE 202. The UE 202 may display an exercise repetition distribution including the frequency and type of exercises to be performed by the user 204 for the affected body parts and a video.

[0037] In some embodiments, UE 202 is configured to receive feedback from user 204 via a touch display of UE 202. In an example, the feedback may include whether user 204 performed the recommended exercise and the level of relief achieved by user 204 after performing the recommended exercise. Accordingly, UE 202 may update the exercise repetition distribution based on the feedback.

[0038] Figure 3a and Figure 3b 2 shows an exemplary process flow including a method 300 for determining the impact on a body part when using a UE 202 and recommending an exercise according to an embodiment of the present invention. The method 300 may be a computer-implemented method executed, for example, by the UE 202. Figure 1 , Figure 2 Figure 3 Fig.11 and Fig.12 The structural and operational features are explained in the description.

[0039] At step 302, method 300 may include UE 202 receiving inertial sensor data and touch screen panel data. The inertial sensor data may be received from a gyroscope, accelerometer installed in UE 202. The touch screen panel data may be received as a result of user 204 interacting with a touch display of UE 202.

[0040] At step 304, the method 300 may include the UE 202 determining the type of application running on the UE 202. In an example, the application may be a video, a song, a game, or a call.

[0041] At step 306, the method 300 may include predicting, by a neural network, a holding direction of the UE 202. In an example, the holding direction of the UE 202 indicates whether the user 204 is holding and currently operating the UE 202. In an embodiment, the neural network is a fully connected artificial neural network and is trained to predict that the user 204 is holding the UE 202 and is not actively involved in operating the UE 202. In another embodiment, the neural network is trained to predict that the user 204 is holding and currently operating the UE 202. In the method 300, the neural network is configured to predict the holding direction based on inertial sensor data, application type, and touch screen panel data.

[0042] At step 308, the method 300 may include determining, by a neural network, a body posture of the user 204 and an affected body part. In an example, the body posture of the user 204 is predicted based on the inertial sensor data because the user 204 is holding and currently operating the UE 202. In an example, corresponding to the body posture of the user 204, the neural network provides a body part that may be affected due to the body posture when the user 204 is holding and currently operating the UE 202.

[0043] At step 310, the method 300 may include the neural network determining an impact level of the affected body part based on the body posture, the holding direction of the UE 202, and the inertial sensor data of the UE 202. In an example, the impact level may include a level of impact on the affected body part (e.g., high, medium, low for each affected body part of the user 204).

[0044] At step 312, method 300 may include the neural network recommending body posture corrections and exercises for the affected body parts based on the level of impact.

[0045] At step 314, the method 300 may include displaying an exercise repetition distribution on the UE 202. In an example, the exercise repetition distribution may represent the frequency and type of recommended exercises to be performed by the user 204 in order to correct body posture. In an example, the exercise repetition distribution is based on the impact level and is displayed for each affected body part.

[0046] At step 316, method 300 may include displaying a video of the recommended exercise based on the affected body part and the level of impact on UE 202. In an example, the video may be pre-stored on a cloud server. Thus, in an example, based on the recommended exercise, UE 202 may be configured to obtain a video stored corresponding to the recommended exercise from the cloud server and display it to user 204.

[0047] At step 318, the method 300 may include the UE 202 receiving feedback from the user 204 via the touch display of the UE 202. In an embodiment, the feedback indicates a relief level of posture correction achieved by the user 204 after performing the recommended exercise.

[0048] At step 320 , method 300 may include UE 202 updating the exercise repetition distribution based on feedback received from user 204 .

[0049] Figure 4aAn exemplary process flow including a method 400 for predicting a holding direction of a UE 202 according to an embodiment of the present invention is shown.

[0050] As previously referenced Figure 3a As discussed, at step 402, method 400 may include receiving, within N seconds, inertial sensor data, touch input data, and a type of application running on UE 202. In an example, N seconds may include a minimum of 1 second and a maximum of 5 seconds of data.

[0051] At step 404, method 400 may include: a data collection application installed in UE 202 and adapted to create two sets of input features. In an embodiment, the data collection application may be adapted to capture all types of possible scenarios indicating whether the user is holding or clamping UE 202. In an embodiment, a first vector feature may be created from inertial sensor data, touch screen panel data, and the type of application collected for N seconds. The neural network may be adapted to process the first vector feature to determine whether user 204 is holding and currently operating UE 202. In another embodiment, a second vector feature may be created from the inertial sensor data. The neural network may be adapted to process the second vector feature to determine the body posture of user 204.

[0052] At step 406, method 400 may include: preparing a training data set for training a neural network. In an example, the training data set is prepared using a first vector feature. In an embodiment, a label is provided for each data of the prepared first vector feature. The label may indicate whether the feature indicates that the user 204 is holding the UE 202 or clamping the UE 202. In an example, a label with holding the UE 202 may indicate that the UE 202 may be in the hand of the user 204 and the user 204 may currently be operating the UE 202. This may include scenarios such as texting, watching videos, playing games, and calls in progress. In another example, a label with clamping the UE 202 may indicate that the UE 202 may be inactive and the user 204 may not currently be operating the UE 202. This may include scenarios such as the screen is off, the UE 202 is not activated or is not used by the user 204, etc. Therefore, the prepared training data set classifies various scenarios with labels of holding or clamping. Due to less free movement, the inertial sensor data during clamping of the UE 202 may be less, and similarly, the touch screen panel data may also be less, indicating that the UE 202 is inactive or that the user is clamping the UE 202 .

[0053] At step 408, the method 400 may include: training the neural network using the training data set. In an embodiment, as the training data set is fed to the neural network, the neural network may be adapted to provide separation between labels. The first vector feature is provided as an input to the neural network to predict the holding direction of the UE 202. In an example, the holding direction of the UE 202 may be that the user 204 is holding and currently operating the UE 202. In an embodiment, the neural network may be a sequential artificial neural network with a fully connected layer, and is adapted to receive the first vector feature as an input and provide two output probabilities. In an example, the output layer of the neural network may provide the holding direction of the UE 202 as two probabilities, for example:

[0054] Y1: indicates the probability that the user 204 is holding and currently operating the UE 202;

[0055] Y2: indicates the probability that the user 204 is holding the UE 202 and the UE 202 is not in use.

[0056] Continuing with step 408, at step 410, method 400 may include the trained neural network being adapted to predict whether user 204 is holding and currently operating UE 202. In an embodiment, the neural network is adapted to continuously predict in real time the holding direction of UE 202 based on inertial sensor data, application type, and touch screen panel data.

[0057] At step 412 , the method 400 can include determining a body posture of the user 204 and an affected body part when finding a probability that the user 204 is holding and currently operating the UE 202 .

[0058] Figure 5 An exemplary process flow including a method 500 for determining body posture according to an embodiment of the present invention is shown.

[0059] At step 510, the method 500 may include: the neural network is adapted to classify the body posture of the user 204. In an example, the body posture of the user 204 may be classified as one of Good, Bad, Worse, and In-Call. In the method 500, the body posture is classified only if the neural network predicts that the user 204 is holding and currently operating the UE 202. In an embodiment, the body posture is classified based on inertial sensor data and the type of application running on the UE 202. Figure 4aAs depicted in , the second vector feature is derived from inertial sensor data. In an example, inertial sensor data from an accelerometer, a gyroscope, may provide an average angle. The average angle may be derived from the angle of use of the UE 202, the duration of use of the UE 202, the proximity of the UE 202 to the face of the user 204. The average angle may form an input for a trained neural network. The trained neural network then determines the posture category as one of good, bad, poor, and in-call based on the second vector feature provided as input.

[0060] At step 504, method 500 may include comparing the classified body posture to a predefined table to determine the affected body parts. In an embodiment, the predefined table provides the affected body parts corresponding to the classification of the body posture.

[0061] Figure 6 An exemplary table for determining affected body parts according to an embodiment of the present invention is shown.

[0062] The predefined table 602 depicts the classification of the body posture 602a mapped in the predefined table 602 and the corresponding affected body parts 602b. In the example, for the body posture 602a classified as good, the affected body parts 602b of the user 204 may be shoulders, wrists, eyes. In the example, since several body parts are connected to each other as joints, the classified body posture may affect multiple body parts. For example, since the body posture is classified as the worst, the affected body parts may be the neck, back, shoulders, because the neck and back are joints connected to each other in the body.

[0063] In addition, table 604 depicts a gesture record including a classification of body gestures 602a corresponding to average angles 604a. In the example, average angle 604a is derived from the angle of use of UE 202, the duration of use of UE 202, and the proximity of UE 202 to the face of user 204 as part of the inertial sensor data. At 604b in table 604, the affected body parts corresponding to the classification of body gestures 602a are depicted. In an embodiment, a weight is assigned to each affected body part for determining the level of influence of each affected body part. In the example, the gesture record stores the average duration, average angle, and affected body part data for further processing.

[0064] FIG. 7 illustrates an exemplary process flow including a method 700 for determining an impact level of an affected body part according to an embodiment of the present invention.

[0065] At step 702, method 700 may include determining, based on inertial sensor data, a usage angle of UE 202, a usage duration of UE 202, and a proximity of UE 202 to a face of user 204. In an embodiment, the influence level is calculated when user 204 is holding and currently operating UE 202. In an example, the score Score(angle) calculated from the usage angle of UE 202 may be calculated using the following formula:

[0066]

[0067] In another example, the score Score(duration) calculated from the usage duration of the UE 202 may be calculated using the following formula:

[0068]

[0069] In an example, when the classified body postures at different N seconds are determined, the neural network is adapted to derive a cumulative score by summing Score(angle) and Score(duration).

[0070] In an embodiment, weights per angle and weights per duration are predefined in the neural network.

[0071] Now, at step 704, method 700 may include calculating an influence score for each affected body part based on the angle of use of the UE, the duration of use of the UE, and the proximity of the UE to the user's face. In an example, the influence score may be calculated based on the occurrence of the affected body part among the total number of affected body parts. In addition, the influence score is based on an average score calculated from the angle of use and the duration of use of the UE 202, respectively.

[0072] At step 706, method 700 may include determining an impact level for each affected body part based on the calculated impact score. In an embodiment, the impact level is one of a high level, a medium level, and a low level. The impact level is determined from the calculated maximum weight sum. A threshold value of the impact score calculated using the above formula classifies the impact level as one of a high level, a medium level, or a low level.

[0073] In an embodiment, method 700 may include determining an exercise repetition distribution based on the impact level. The exercise repetition distribution includes the frequency and type of exercise to be performed by user 204. In an example, the type of exercise is displayed based on the determined impact level.

[0074] In an example, the frequencies in the exercise repetition distribution are calculated by a neural network. The neural network can calculate the exercise repetition distribution for the affected body part based on the impact score, the exercise type count, and the occurrence count of the affected body part. In an example, the occurrence count indicates a normalized ratio of the number of times a particular affected body part is recorded as affected within the duration among all body parts that appear in the posture record table.

[0075] In addition, based on the determined exercise repetition distribution, the neural network derives a video of a recommended exercise type based on at least one affected body part and the level of impact. In an example, the video can be displayed on the UE 202 along with the determined exercise repetition distribution.

[0076] Figure 7b An exemplary process flow including a method 700b for determining an influence level of a user's 204 eye is shown.

[0077] At step 702b, method 700b may include determining light intensity data of UE 202 and proximity of UE 202 to a face of user 204 when predicting that user 204 is holding and currently operating UE 202. In an example, inertial sensor data may provide light intensity data of UE 202 and proximity of UE 202 to a face of user 204.

[0078] At step 704 b , the method 700 b may include determining, by a neural network, a level of impact on the eyes of the user 204 based on the light intensity data of the UE 202 and the proximity of the UE 202 .

[0079] At step 706b, the method 700b may include recommending, by the neural network, exercises for the eyes of the user 204 based on the determined level of impact.

[0080] Figure 8a An exemplary use case depicting a gesture log table 802 providing a determined body gesture 802a and a corresponding affected body part 802b according to an embodiment of the present invention is shown. As shown in FIG8 , the gesture log table 802 may be stored in a database and may be used for display on the UE 202 .

[0081] Figure 8bAn exemplary use case is shown depicting a predefined posture record table 804 providing predefined postures 802a and corresponding average duration and weight 806 for each predefined posture 802a. Another predefined posture record table 808 provides predefined postures 802a and corresponding average angles and weights 806 for each predefined posture 802a. Weights can be assigned based on research data and provided to a neural network for calculating impact scores. In this case use case, the impact score is calculated for the body part - Neck:

[0082]

[0083]

[0084] Now, based on obtaining the weight per duration and the weight per angle from the predefined posture record table 804 and the predefined posture record table 808, respectively. In the example, the posture record table 804 depicts the body posture of the user 204 as bad, where the average duration is 30 minutes, and the weight assigned is (0.2). Similarly, referring to the predefined posture record table 808, the average angle is 20 degrees, and the weight assigned is (0.2). Therefore, the influence score calculated for the body part-neck using the present invention is as follows:

[0085] Score (Duration) = Neck (Call (0.3) + Worst (0.3) + Bad (0.2)) = 0.8

[0086] Score (angle) = Neck (call (0.2) + worst (0.2) + bad (0.2)) = 0.6

[0087] Average score (angle + duration) = Neck ((0.8+0.6) / 2) = 0.7

[0088] Therefore, the impact score calculated for the affected body part, namely the neck, is 0.70. In addition, the calculated impact score classifies the impact level as one of a high level, a medium level, or a low level. The impact levels can be classified using a predefined threshold for classifying the impact levels within the range of the impact score. In the present illustrative example, the impact score can be classified as a high level of impact level for the affected body part - the neck. In the example, based on a predefined table, a type of exercise is defined for each impact level. For example, for the neck, two types of exercises can be provided, namely, rotation and stretching.

[0089] In the example, further, the exercise repetition distribution for the neck is calculated. The exercise repetition distribution is based on the occurrence count, for example, the occurrence count is equal to 3, the impact score is 0.70, and the exercise type count is 2. Therefore, the exercise repetition distribution can be equal to about 3.

[0090] Figure 9a Another exemplary use case is shown depicting a table 902 displayed on the UE 202. The table 902 may include an affected body part 904 and an impact level 906, according to an embodiment of the invention.

[0091] Figure 9b Another exemplary use case is shown depicting a table 908 displayed on UE 202. Table 908 may include affected body parts 904, exercise types 910, and frequencies 912 for the exercises, according to an embodiment of the invention.

[0092] Fig.10 An exemplary process flow including a method 1000 for receiving feedback from a user 204 and modifying a recommended workout is shown in accordance with an embodiment of the present invention.

[0093] At step 1002, method 1000 may include: displaying Figure 3b The exercise repetition distribution discussed in .

[0094] At step 1004, the method 1000 may include receiving feedback from the user 204 based on whether the user 204 has performed the exercise. In an embodiment, the UE 202 may display a prompt message in which the user 204 objectively provides feedback indicating whether the user 204 has performed the recommended exercise. In addition, the user 204 may objectively provide a relief level indicating the impact after performing the recommended exercise.

[0095] At step 1006, method 1000 may include adjusting the weights and recalculating the exercise repetition distribution. In an embodiment, the recalculation of the exercise repetition distribution may be calculated by a neural network for the affected body part based on the impact score, the exercise type count, and the occurrence count of the affected body part.

[0096] At step 1008, method 1000 may include changing the exercise repetition distribution based on the feedback received from user 204. In an embodiment, the recommended video may also be changed based on the changed exercise repetition distribution.

[0097] Fig.11 A representative architecture 1100 providing the tools and implementation environment described herein, a technical implementation of a system 1104 for determining the impact on a body part while using a UE 202 and recommending exercises is shown. Fig.11 This is only a non-limiting example, and it will be appreciated that many other architectures may be implemented to facilitate the functionality described herein. Architecture 1100 may be implemented on hardware, for example, Fig.11 The UE 202 includes a processor, a memory, and various dedicated hardware components.

[0098] In the system 1104, the UE 202 may include an operating system, a library, a framework, or middleware. The operating system may manage hardware resources and provide general services. The operating system may include, for example, a kernel, services, and a driver that defines a hardware interface layer. The driver may be responsible for controlling or interfacing with the underlying hardware. For example, the driver may include a display driver, a camera driver, a Bluetooth® driver, a flash driver, a serial communication driver (e.g., a universal serial bus (USB) driver), a Wi-Fi® driver, an audio driver, a power management driver, etc., depending on the hardware configuration.

[0099] The hardware interface layer includes libraries, which may include system libraries such as a file system (e.g., a C standard library), which may provide functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. In addition, these libraries may also include API libraries, such as an audio-visual media library (e.g., a multimedia database that supports the presentation and manipulation of various media formats such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.), a database library (e.g., SQLite that can provide various relational database functions), a web library (e.g., WebKit that can provide network browsing functions), etc.

[0100] Middleware may provide higher-level general infrastructure, e.g., various graphical user interface (GUI) capabilities, advanced resource management, advanced location services, etc. Middleware may provide a wide range of other APIs that may be utilized by applications and / or other software components / modules, some of which may be specific to a particular operating system or platform.

[0101] The term "module" used in the present disclosure may refer to a specific unit including one of hardware, software and firmware or any combination thereof. For example, a module may be used interchangeably with a unit, logic, logic block, component or circuit. A module may be the smallest unit or a part thereof that performs one or more specific functions. A module may be formed mechanically or electronically. For example, the module disclosed herein may include at least one of a known or future developed ASIC (application specific integrated circuit) chip, an FPGA (field programmable gate array) and a programmable logic device.

[0102] In addition, the system 1104 according to an embodiment of the present disclosure may include a UE 202 and a user 204. The UE 202 may include a set of instructions that can be executed via the processor 1112 to enable the UE 202 to perform any one or more of the disclosed methods. The UE 202 may operate as a standalone device or may be connected to other computer systems or peripheral devices, for example, using a network.

[0103] In an embodiment, the processor 1112 is configured to receive inertial sensor data 1106 and touch screen panel data 1108 of the UE 202. In an example, the inertial sensor data 1106 is received from an accelerometer and a gyroscope installed in the UE 202. The processor 1112 is also configured to determine the type of application running on the UE 202.

[0104] Processor 1112 communicates with neural network 1114 and is configured to predict the holding direction of UE 202 based on inertial sensor data, application type and touch screen panel data through neural network 1114. In an example, the holding direction of UE 202 indicates whether user 204 is holding and currently operating UE 202.

[0105] In an embodiment, the processor 1112 is configured to determine, via the neural network 1114 , a body posture and an affected body part of the user 204 based on the inertial sensor data in response to predicting that the user 204 is holding and currently operating the UE 202 .

[0106] In an embodiment, the processor 1112 is configured to determine, through the neural network 1114 , the impact level of the affected body part based on the body posture, the holding direction of the UE 202 , and the inertial sensor data of the UE 202 .

[0107] In an embodiment, the processor 1112 is configured to recommend body posture corrections and exercises for the affected body parts based on the impact level.

[0108] Fig.12 Another system architecture of the system 1104 is shown in the form of a computer system 1200. The computer system 1200 may include a set of instructions that may be executed to cause the computer system 1200 to perform any one or more of the disclosed methods. The computer system 1200 may operate as a standalone device or may be connected to other computer systems or peripheral devices, for example using a network.

[0109] In a networked deployment, the computer system 1200 can operate as a server or as a client user computer in a server-client user network environment, or as a peer computer system in a peer (or distributed) network environment. The computer system 1200 can also be implemented as or incorporated into various devices, such as a personal computer (PC), a tablet PC, a personal digital assistant (PDA), a mobile device, a palmtop computer, a laptop computer, a desktop computer, a communication device, a wireless phone, a landline phone, a network device, a network router, a switch or a bridge, or any other machine capable of executing (sequentially or otherwise) a set of instructions, wherein the instructions specify the actions to be taken by the machine. In addition, although a single computer system 1200 is shown, the term "system" should also be understood to include any collection of systems or subsystems that execute one or more sets of instructions individually or jointly to perform one or more computer functions.

[0110] The computer system 1200 may include a processor 1112, such as a central processing unit (CPU), a graphics processing unit (GPU), or both. The processor 1112 may be a component in a variety of systems. For example, the processor 1112 may be part of a standard personal computer or workstation. The processor 1112 may be one or more general-purpose processors, digital signal processors, application-specific integrated circuits, field programmable gate arrays, servers, networks, digital circuits, analog circuits, combinations thereof, or other devices now known or developed in the future for analyzing and processing data. The processor 1112 may implement a software program, such as a manually generated (i.e., programmed) code.

[0111] The computer system 1200 may include a memory 1208, for example, a memory 1208 that can communicate via a bus 1208. The memory 1208 may include, but is not limited to, computer-readable storage media such as various types of volatile and non-volatile storage media, including but not limited to random access memory, read-only memory, programmable read-only memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, tape or disk, optical media, etc. In one example, the memory 1208 includes a cache or random access memory for the processor 1112. In an alternative example, the memory 1208 is separate from the processor 1112, such as a cache memory of the processor, a system memory, or other memory. The memory 1208 may be an external storage device or database for storing data. The memory 1208 is operable to store instructions that can be executed by the processor 1112. The functions, actions, or tasks shown or described in the figures can be performed by the processor 1112 programmed to execute the instructions stored in the memory 1208. These functions, acts or tasks are independent of a particular type of instruction set, storage medium, processor or processing strategy, and may be performed by software, hardware, integrated circuits, firmware, microcode, etc., operating alone or in combination. Similarly, processing strategies may include multi-processing, multi-tasking, parallel processing, etc.

[0112] As shown, the computer system 1200 may or may not include a display 1210, such as a liquid crystal display (LCD), an organic light emitting diode (OLED), a flat panel display, a solid state display, a cathode ray tube (CRT), a projector, a printer, or other display devices known now or developed in the future for outputting the determined information. The display 1210 may serve as an interface for a user to view the functions of the processor 1112, or specifically as an interface with the drive unit 1216 or the software stored in the memory 1208.

[0113] Additionally, the computer system 1200 may include an input device 1212 configured to allow a user to interact with any component of the system 1204. The computer system 1200 may also include a disk or optical drive unit 1216. The disk drive unit 1216 may include a computer-readable medium 1222 in which one or more sets of instructions 1224 (e.g., software) may be embedded. Furthermore, the instructions 1224 may embody one or more of the described methods or logic. In a specific example, the instructions 1224 may reside completely or at least partially within the memory 1208 or within the processor 1112 during execution by the computer system 1200.

[0114] The present invention contemplates a computer-readable medium that includes instructions 1224, or receives and executes instructions 1224 in response to a propagating signal, so that a device connected to a network 1226 can transmit voice, video, audio, images, or any other data on the network 1226. In addition, instructions 1224 can be sent or received on the network 1226 via a communication port or interface 1220 or using a bus 1208. The communication port or interface 1220 can be part of the processor 1206, or can be a separate component. The communication port 1220 can be created in software, or can be a physical connection in hardware. The communication port 1220 can be configured to connect to the network 1226, external media, display 1210, or any other component or combination thereof in the system 1204. The connection to the network 1226 can be a physical connection (e.g., a wired Ethernet connection), or can be established wirelessly as discussed later. Similarly, additional connections to other components of the system 1204 can be physical, or can be established wirelessly. The network 1226 may alternatively be connected directly to the bus 1208 .

[0115] The network 1226 may include a wired network, a wireless network, an Ethernet AVB network, or a combination thereof. The wireless network may be a cellular telephone network, an 802.11, 802.16, 802.20, 802.1Q, or WiMax network. In addition, the network 826 may be a public network (e.g., the Internet), a private network (e.g., an intranet), or a combination thereof, and may utilize a variety of networking protocols that are currently available or will be developed in the future (including but not limited to networking protocols based on TCP / IP). The system is not limited to operation using any particular standard and protocol. For example, standards for Internet and other packet-switched network transmissions (e.g., TCP / IP, UDP / IP, HTML, and HTTP) may be used.

[0116] Although specific language has been used to describe the present disclosure, no limitation is intended thereby. As will be apparent to those skilled in the art, various working modifications may be made to the methods to implement the inventive concepts taught herein.

[0117] The accompanying drawings and the foregoing description give examples of embodiments. It will be appreciated by those skilled in the art that one or more of the elements described may be well combined into a single functional element. Alternatively, some elements may be divided into multiple functional elements. Elements from one embodiment may be added to another embodiment. For example, the order of the processing described herein may be changed and is not limited to the manner described herein.

Claims

1. A method for determining an impact on at least one body part when using a user equipment UE and recommending at least one exercise, the method comprising: Receiving inertial sensor data and touch screen panel data of a user equipment UE; Determining the type of application running on the UE; predicting, by a neural network, a holding direction of the UE based on the inertial sensor data, the application type, and the touch screen panel data, wherein the holding direction of the UE indicates whether a user is holding and currently operating the UE; In response to predicting that the user is holding and currently operating the UE, determining, by the neural network, a body posture and at least one affected body part of the user based on the inertial sensor data; determining, by the neural network, an impact level of the at least one affected body part based on the body posture, the holding direction of the UE, and the inertial sensor data of the UE; Body posture correction and at least one exercise for the affected body part are recommended based on the impact level.

2. The method according to claim 1, wherein: It is recommended that the body posture correction and the at least one exercise include: displaying an exercise repetition distribution indicating a frequency and type of the at least one exercise to be performed by the user; and A video for the at least one exercise is displayed based on the at least one affected body part and the impact level.

3. The method according to claim 2, further comprising: receiving feedback from the user via a touch display of the UE while the user performs the at least one exercise for the affected body part; The exercise repetition distribution is updated based on the feedback.

4. The method according to claim 1, wherein: Determining the user's body posture and the corresponding affected body part includes: When the user is holding and currently operating the UE, classifying the user's body posture as one of good, bad, poor, and in call based on the inertial sensor data and the type of application running on the UE; and The classified body posture is compared to a predefined table to determine the at least one affected body part, wherein the predefined table indicates at least one body part corresponding to the classification of the body posture.

5. The method according to claim 1, wherein: Determining the impact level of the at least one affected body part based on the body posture comprises: Based on the inertial sensor data, when the user is holding and currently operating the UE, determining a usage angle of the UE, a usage duration of the UE, and a proximity of the UE to the face of the user; calculating an impact score for each affected body part based on an angle of use of the UE, a duration of use of the UE, and a proximity of the UE to the face of the user; and An impact level of each affected body part is determined based on the score, wherein the impact level is indicated as one of a high level, a medium level, and a low level.

6. The method according to claim 5, further comprising: When predicting the holding direction of the UE, determining the light intensity data of the UE and the proximity of the UE according to the inertial sensor data; determining, by the neural network, a level of impact on the user's eyes based on the light intensity data of the UE and the proximity of the UE; as well as At least one exercise for the user's eyes is recommended by the neural network based on the impact level.

7. The method according to claim 1, wherein: The inertial sensor data includes data from one or more of an accelerometer and a gyroscope.

8. The method according to claim 1, wherein: The touch screen panel data includes one or more of touch coordinates, hover distribution, and touch duration on the screen of the UE.

9. A system for determining an impact on at least one body part when using a user equipment (UE) and recommending at least one exercise, the system comprising: The processor is configured as: Receiving inertial sensor data and touch screen panel data of the UE; Determining the type of application running on the UE; predicting, by a neural network, a holding direction of the UE based on the inertial sensor data, the touch screen panel data, and the application type, wherein the holding direction of the UE indicates whether a user is holding and currently operating the UE; In response to predicting that the user is holding and currently operating the UE, determining, by the neural network, a body posture and at least one affected body part of the user based on the inertial sensor data; determining, by the neural network, an impact level of the at least one affected body part based on the body posture, the holding direction of the UE, and the inertial sensor data of the UE; Body posture correction and at least one exercise for the affected body part are recommended based on the impact level.

10. The system according to claim 9, wherein: The processor is configured to: displaying an exercise repetition distribution indicating a frequency and type of the at least one exercise to be performed by the user; and A video for the at least one exercise is displayed based on the at least one affected body part and the impact level.

11. The system of claim 10, wherein the processor is further configured to: receiving feedback from the user via a touch display of the UE while the user performs the at least one exercise for the affected body part; The exercise repetition distribution is updated based on the feedback.

12. The system according to claim 9, wherein: The processor configured to determine a body posture of the user and a corresponding affected body part is further configured to: When the user is holding and currently operating the UE, classifying the user's body posture as one of good, bad, relatively poor, and in call based on the inertial sensor data and the type of application running on the UE; as well as The classified body posture is compared to a predefined table to determine the at least one affected body part, wherein the predefined table indicates at least one body part corresponding to the classification of the body posture.

13. The system according to claim 9, wherein: The processor configured to determine an impact level of the at least one affected body part based on the body posture is further configured to: Based on the inertial sensor data, when the user is holding and currently operating the UE, determining a usage angle of the UE, a usage duration of the UE, and a proximity of the UE to the face of the user; calculating an impact score for each affected body part based on an angle of use of the UE, a duration of use of the UE, and a proximity of the UE to the face of the user; and An impact level of each affected body part is determined based on the score, wherein the impact level is indicated as one of a high level, a medium level, and a low level.

14. The system of claim 13, wherein the processor is further configured to: When predicting the holding direction of the UE, determining the light intensity data of the UE and the proximity of the UE according to the inertial sensor data; determining, by the neural network, a level of impact on the user's eyes based on the light intensity data of the UE and the proximity of the UE; and At least one exercise for the user's eyes is recommended by the neural network based on the impact level.

15. The system according to claim 9, wherein: The inertial sensor data includes data from one or more of an accelerometer and a gyroscope.

Images