A method for determining exercise guidance information, an electronic device, and an exercise guidance system

Abstract

The application provides a determination method of movement guidance information, an electronic device and a movement guidance system, and relates to the technical field of human-computer interaction. The method is applied to the electronic device, and includes the following steps: displaying a first preset image and a collected user image; identifying a deflection angle of a virtual user in the user image; determining a second preset image corresponding to the deflection angle from a demonstration action set, wherein the demonstration action set stores preset images corresponding to different deflection angles; determining movement guidance information of the user according to the user image and the second preset image; and the first preset image and the second preset image are the same or different. Through the technical scheme provided in the embodiment of the application, the accuracy of user action recognition can be improved. In the process of movement exercise, the user does not need to be restricted by the deflection angle, and can select the deflection angle relative to the electronic device according to personal preferences, thereby achieving good user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human-computer interaction, and in particular to a method for determining exercise guidance information, an electronic device, and an exercise guidance system. BACKGROUND

[0002] With the development of computer technology, electronic devices (such as smart TVs) equipped with cameras can guide users to exercise, for example, to guide users to do bodybuilding, practice yoga, dance, play taijiquan, and play basketball. Taking the case of guiding users to do bodybuilding by using an electronic device, the electronic device displays a preset demonstration image for the user to imitate and learn, captures a user image by using a camera to identify the user's body posture, and guides the user's bodybuilding action according to the body posture.

[0003] However, in the process of performing an action according to the demonstration image, the user must face the electronic device at a specific angle. For example, when the demonstration image displayed by the electronic device is a front view of a demonstrator, the user must face the electronic device frontally; or when the demonstration image displayed by the electronic device is a side view of a demonstrator, the user must face the electronic device laterally, otherwise the electronic device will not be able to identify or accurately identify the user's action, and the user experience is poor. SUMMARY

[0004] Embodiments of the present application provide a method for determining exercise guidance information, an electronic device, and an exercise guidance system, which can improve the accuracy of the electronic device in identifying a user's action.

[0005] To achieve the above object, the present application adopts the following technical solutions:

[0006] In a first aspect, the embodiments of the present application provide a method for determining exercise guidance information, applied to an electronic device, which includes: displaying a first preset image and a captured user image; identifying a deflection angle of a virtual user in the user image; determining a second preset image corresponding to the deflection angle from a set of demonstration actions, wherein the set of demonstration actions stores preset images corresponding to different deflection angles; determining exercise guidance information of the user according to the user image and the second preset image; and wherein the first preset image and the second preset image are the same or different.

[0007] In the method provided by the embodiments of the present application, the electronic device compares the user image with the second preset image corresponding to the deflection angle of the user image after determining the deflection angle, and determines the exercise guidance information of the user, which can improve the accuracy of the user action identification. Based on this, the user does not need to be restricted by the deflection angle during the exercise, and can choose the deflection angle relative to the electronic device according to his or her own preference, which has a good user experience.

[0008] In some embodiments, the method further comprises: displaying the second preset image while displaying the first preset image; or, replacing the first preset image with the second preset image. Through the method provided by this embodiment, the user can observe the preset image corresponding to the deflection angle of the user image on the electronic device.

[0009] In some embodiments, the motion guidance information comprises at least one of an action score, an action comment, an action evaluation icon, posture correction information, calories consumed, and motion duration.

[0010] In the second aspect, the embodiments of the present application provide an electronic device configured to: display a first preset image and a collected user image; identify a deflection angle of a virtual user in the user image; determine a second preset image corresponding to the deflection angle from a set of demonstration actions, wherein the set of demonstration actions stores preset images corresponding to different deflection angles; determine motion guidance information of the user according to the user image and the second preset image; and wherein the first preset image and the second preset image are the same or different.

[0011] In some embodiments, the electronic device is further configured to: display the second preset image while displaying the first preset image; or, replace the first preset image with the second preset image.

[0012] In some embodiments, the motion guidance information comprises at least one of an action score, an action comment, an action evaluation icon, posture correction information, calories consumed, and motion duration.

[0013] In the third aspect, the embodiments of the present application provide a motion guidance system comprising a first electronic device and at least one second electronic device; the first electronic device is configured to perform the method shown in the first aspect; the first electronic device determines motion guidance information of the user according to local collected user data and user data collected by the at least one second electronic device; and wherein the at least one second electronic device is the same or different.

[0014] The system provided by the embodiments of the present application can determine the motion guidance information of the user by combining multi-dimensional data obtained from multiple channels on the basis of determining the corresponding second preset image according to the deflection angle of the virtual user and then identifying the user action according to the second preset image, thereby improving the accuracy of the motion guidance.

[0015] In some embodiments, the user data collected by the first electronic device is a first user image, and the user data collected by the second electronic device is a second user image, wherein the deflection angle of the virtual user in the first user image is different from the second user image.

[0016] In some embodiments, the user data collected by the second electronic device is body fat information; and the first electronic device recommends a corresponding exercise item to the user according to the body fat information. The system provided in the embodiment can recommend a suitable exercise item to the user. For example, when the leg body fat rate of the user is out of standard, the user is recommended a leg fat reduction item. When the waist and abdomen body fat rate of the user is out of standard, the user is recommended a waist and abdomen fat reduction item.

[0017] In some embodiments, the user data collected by the second electronic device is a personal physical parameter; and the first electronic device displays health prompt information according to the personal physical parameter, the health prompt information being used to prompt whether the user can perform exercise.

[0018] In some embodiments, the personal physical parameter includes at least one of a heart rate, a blood pressure, a body temperature, and a blood oxygen saturation.

[0019] In some embodiments, the user data collected by the second electronic device includes a jumping height and a jumping frequency of the user.

[0020] In some embodiments, the first electronic device and the at least one second electronic device are connected through a far field communication technology, a near field communication technology, or a physical entity.

[0021] In a fourth aspect, an embodiment of the present application provides a chip system, which includes a memory and a processor. The processor executes a computer program stored in the memory to implement the method shown in the first aspect. The chip system can be a single chip or a chip module composed of multiple chips.

[0022] In a fifth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the method shown in the first aspect.

[0023] It can be understood that the beneficial effects of the second aspect, the fourth aspect, and the fifth aspect can be referred to the related description in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1A FIG. 1 is a schematic diagram of a motion guidance scene of an electronic device provided in an embodiment of the present application;

[0025] Figure 1B FIG. 2 is a schematic diagram of a motion guidance scene of an electronic device provided in an embodiment of the present application; Figure Two

[0026] Figure 2 FIG. 3 is a schematic flowchart of a method for determining motion guidance information provided in an embodiment of the present application;

[0027] Figure 3 ​A schematic diagram of a deflection angle provided for an embodiment of the present application;

[0028] Figure 4 A schematic diagram of a determination process of a deflection angle provided for an embodiment of the present application;

[0029] Figure 5A A schematic diagram of an interface display of an electronic device provided for an embodiment of the present application;

[0030] Figure 5B A schematic diagram of an interface display of an electronic device provided for an embodiment of the present application Figure Two ;

[0031] Figure 6 A schematic diagram of an application scenario of a motion guidance system provided for an embodiment of the present application;

[0032] Figure 7 A schematic diagram of an image acquisition scenario of a large-screen device in an embodiment of the present application;

[0033] Figure 8A A schematic diagram of user data processing provided for an embodiment of the present application;

[0034] Figure 8B A schematic diagram of user data processing provided for an embodiment of the present application Figure Two ;

[0035] Figure 8C A schematic diagram of user data processing provided for an embodiment of the present application Figure Three ;

[0036] Figure 9 A schematic diagram of a center computing device selection process provided for an embodiment of the present application;

[0037] Figure 10 A schematic diagram of user height calculation provided for an embodiment of the present application;

[0038] Figure 11 A schematic diagram of user data processing provided for an embodiment of the present application;

[0039] Figure 12A A schematic diagram of a first user image acquisition scenario provided for an embodiment of the present application;

[0040] Figure 12B A schematic diagram of a second user image acquisition scenario provided for an embodiment of the present application;

[0041] Figure 12C A schematic diagram of a third user image acquisition scenario provided for an embodiment of the present application;

[0042] Figure 13 A schematic diagram of user jump height calculation provided for an embodiment of the present application;

[0043] Figure 14 A score interface schematic diagram provided by an embodiment of the present application;

[0044] Figure 15 A schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solutions provided by the embodiments of the present application will be described below with reference to the accompanying drawings.

[0046] In the embodiment, the electronic device can be a large-screen device (such as a smart television, a smart screen, a projector, etc.), a mobile phone, a tablet computer, a notebook computer, a desktop computer, a smart television, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc.

[0047] With the development of computer technology, electronic devices (such as smart televisions and mobile phones) can guide users to perform exercise, for example, guiding users to perform artificial intelligence (AI) fitness, practicing yoga, dancing, playing Tai Chi, playing basketball, etc. Taking the example of guiding users to perform AI fitness by an electronic device, in the process of guiding users to perform exercise, the electronic device will display a demonstration image for users to imitate and learn the demonstration action. In addition, the electronic device will also collect and display a user image, and compare the user image with the demonstration image displayed by the electronic device to identify the human posture of the user, determine the completion of the demonstration action by the user, and generate exercise guidance information. Exemplarily, the exercise guidance information can be an action score, an action comment, an action evaluation icon, posture correction information, calories consumed, exercise duration, etc.

[0048] Based on this, at present, the user must face the electronic device at a specific angle to ensure that the user image and the demonstration image are images at the same angle, so that the electronic device can accurately identify the user action. However, when the user does not face the electronic device at a specific angle, the user image collected by the electronic device and the preset demonstration image are not images at the same angle. At this time, even if the user actually accurately performs the demonstration action, due to the large difference between the user image and the demonstration image, the electronic device cannot accurately identify the user action, resulting in poor user experience.

[0049] For example, referring to Figure 1AAs shown, when the exemplary image displayed by the electronic device is a front-facing motion image of the demonstrator, the user faces the electronic device with the front of the body, and the demonstrator image and the user image are images of the same angle (i.e., front-facing images). The electronic device compares the front-facing user image with the front-facing demonstrator image to accurately identify the user's action and output more accurate motion guidance information.

[0050] Otherwise, see Figure 1B As shown, when the exemplary image displayed by the electronic device is a front-facing motion image of the demonstrator, if the user faces the electronic device with the left side of the body, the user image collected by the electronic device will be an image of the left side of the user's body. Then, even if the user actually accurately completes the demonstration action, since the user image and the demonstration image are not images of the same angle, there is a large difference between them, and the electronic device cannot accurately identify the user's action when comparing the user image with the demonstration image, and may even think that the user has not completed the demonstration action, thereby outputting inaccurate motion guidance information, resulting in poor user experience.

[0051] Therefore, the embodiments of the present application provide a method for determining motion guidance information, which can improve the accuracy of the electronic device in identifying the user's action, and can also enable the user to be free from the above-mentioned specific angle constraint during the exercise process, so that the user can freely choose the angle facing the electronic device, thereby improving the user experience.

[0052] In the following, taking the electronic device as a large-screen device as an example, the method for determining motion guidance information provided by the embodiments of the present application is exemplarily described in combination with the scene of the electronic device guiding the user to exercise. It should be understood that the technical solutions provided by the embodiments are also applicable to other motion exercise scenes based on human-computer interaction, such as dancing, playing Tai Chi, playing basketball, and playing a body-sensing game.

[0053] Figure 2 is a schematic flowchart of the method for determining motion guidance information provided by the embodiments of the present application, which shows the process of the electronic device intelligently identifying the user's action according to the deflection angle of the virtual user in the user image, thereby determining the motion guidance information. Specifically, the following steps S201-S203 are included.

[0054] S201, the electronic device displays a first preset image and a collected user image.

[0055] Electronic devices guide users through exercise by running fitness applications (Apps). Based on the App's pre-configured settings, the electronic device can offer various exercise programs. For example, an AI fitness application might offer exercises such as "neck and shoulder circles," "chest expansion exercises," "stretching exercises," and "leg kicks." In this embodiment, an exercise program (e.g., "neck and shoulder circles") refers to a sub-application within the application providing the exercise guidance service. An application typically includes multiple sub-applications, and the electronic device can provide different guidance content by running different sub-applications.

[0056] In this embodiment, for each sport, the application pre-sets a set of demonstration movements. This set includes multiple sets of demonstration images, each showing the demonstrator at a different deflection angle. For example, there may be images of the demonstrator from the front, back, or left. The demonstration images corresponding to different deflection angles can display the demonstration movements from different perspectives, allowing users to imitate and learn.

[0057] It should be noted that the demonstrator refers to a virtual character displayed in the demonstration area of ​​the electronic device, used to demonstrate preset actions. The user refers to a real person participating in the sport. The virtual user refers to a virtual character in the user image captured by the electronic device. It should be understood that during the real-time acquisition of user images by the electronic device, the user and the virtual user are usually in a mirror-symmetric relationship.

[0058] In this embodiment, see Figure 3 As shown, the deflection angle refers to the angle of rotation required for an object to move from a reference position to its current position, rotating clockwise or counterclockwise around its body as the central axis. Specifically, when the object rotates clockwise, the deflection angle is positive, ranging from 0° to 360°; when the object rotates counterclockwise, the deflection angle is negative, ranging from 0° to -360°. For the demonstration image and user image, the reference position is the location of the demonstrator or virtual user when they are directly facing the electronic device screen. For the user, the reference position is the location when the user is directly facing the electronic device screen.

[0059] It should be understood that since virtual users and real users are usually mirror images of each other, the deflection angles of virtual users and real users are typically opposite. For example, see... Figure 1A As shown, when the actual user stands facing the electronic device with their left hand raised, the virtual user appears to have their right hand raised in the image captured by the electronic device. When the actual user rotates clockwise, the virtual user rotates counterclockwise.

[0060] For the same demonstration action, the demonstration effect of the demonstration action performed by the demonstrator is usually different when the demonstrator is in a deflection angle. For example, for the "shoulder and neck circling action", the demonstration image of the demonstrator with a deflection angle of 0° (i.e., the demonstration image of the demonstrator facing forward) usually has a better demonstration effect, and the actions of the left and right arms can be simultaneously demonstrated. In the demonstration image of the demonstrator with a deflection angle of 90° (i.e., the demonstration image of the demonstrator on the left side), the right arm of the demonstrator is blocked, and the electronic device cannot clearly display the action of the right arm.

[0061] Based on this, in some embodiments, for each demonstration action, the App developer can pre-set one or more high-priority deflection angles according to the characteristics of the action. When displaying the demonstration image, the electronic device can display the demonstration image corresponding to the high-priority deflection angle as the first preset image. For example, for the "shoulder and neck circling action", the demonstration image with a deflection angle of 0° (i.e., the demonstration image of the demonstrator facing forward) is displayed on the screen of the electronic device as the first preset image, so as to clearly demonstrate the demonstration action to the user.

[0062] In some embodiments, the electronic device can also determine and display the first preset image according to the user instruction. For example, the demonstration image of the demonstrator with a deflection angle of 90° (i.e., the demonstration image of the demonstrator on the left side) is displayed according to the user instruction. Alternatively, the demonstration image with a deflection angle of 0° (i.e., the demonstration image of the demonstrator facing forward) and the demonstration image with a deflection angle of 90° (i.e., the demonstration image of the demonstrator on the left side) are simultaneously displayed according to the user instruction.

[0063] In S202, the electronic device determines the deflection angle of the virtual user in the user image.

[0064] In this embodiment, the electronic device can calculate the deflection angle of the virtual user in the user image according to the Residual Neural Network (ResNet) algorithm. For example, refer to Figure 4As shown, the electronic device can pre-acquire a training set, each image in the training set including a person with a different yaw angle, and a label of the yaw angle. For example, a front view (yaw angle = 0°), a left side view (yaw angle = 90°), a right side view (yaw angle = -90°), and a back view (yaw angle = 180°) of a person, and the corresponding labels. The electronic device can learn the images of the person in the training set according to the ResNet algorithm, and determine a series of algorithm model values (i.e., Model values), wherein each Model value corresponds to a yaw angle. After the electronic device acquires the user image, the electronic device can calculate the Model value of the user image according to the ResNet algorithm, and determine the yaw angle of the virtual user according to the correspondence between the Model value and the yaw angle. Alternatively, the electronic device can also identify the yaw angle of the virtual user in the user image according to a pre-set yaw angle recognition model, which is not limited in the present embodiment.

[0065] In S203, the electronic device determines a second preset image corresponding to the yaw angle of the virtual user from the set of demonstration images, and determines the movement guidance information according to the user image and the second preset image, wherein the first preset image and the second preset image are the same or different.

[0066] For example, referring to Figure 1B As shown, when the demonstration image displayed by the electronic device is a front view of the demonstrator, if the electronic device detects that the yaw angle of the virtual user in the user image is -90° (at this time, the left side of the actual user is facing the screen of the electronic device, and the electronic device acquires the image of the left side of the user, while the right side of the virtual user is displayed on the electronic device), the second preset image, i.e., the demonstration image of the demonstrator with a yaw angle of -90° (the right side image of the demonstrator), can be determined from the set of demonstration images. Subsequently, the electronic device compares the demonstration image of the demonstrator with a yaw angle of -90° with the user image with a yaw angle of -90° of the virtual user, identifies the movement of the user, and determines the movement guidance information.

[0067] In addition, optionally, after determining the yaw angle of the virtual user in the user image, the electronic device can also display the demonstration image corresponding to the yaw angle (i.e., the second preset image) in the demonstration image display area.

[0068] In some embodiments, the electronic device can display the first preset image and the second preset image simultaneously. For example, referring to Figure 5A As shown, when the first preset image displayed by the electronic device is a front view of the demonstrator, if the yaw angle of the virtual user is detected to be -90°, the demonstration image of the demonstrator with a yaw angle of -90° can be displayed as the second preset image simultaneously with the first preset image (i.e., the image of the front view of the demonstrator) on the electronic device.

[0069] In some embodiments, the electronic device can replace the first preset image with a second preset image. For example, referring to Figure 5B As shown in FIG. 6, when the first preset image displayed by the electronic device is the image of the demonstrator facing forward, if the deflection angle of the virtual user is detected to be -90°, the electronic device can display the image of the demonstrator with a deflection angle of -90° as the second preset image, instead of the first preset image (i.e., the image of the demonstrator facing forward).

[0070] In addition, the user can change the deflection angle relative to the electronic device during the exercise, causing the deflection angle of the virtual user to change accordingly. Therefore, during the display of the second preset image by the electronic device, if the electronic device detects a change in the deflection angle of the virtual user, the electronic device can update the second preset image according to the changed deflection angle.

[0071] In summary, in this embodiment, the electronic device can accurately identify the user's action even if the presentation angle of the user image and the first preset image displayed by the electronic device is different. Through the method provided in this embodiment, the user can choose the deflection angle relative to the electronic device according to his / her preference during the exercise, without being restricted by the deflection angle, thus providing a better user experience.

[0072] Based on the electronic device provided in the above embodiments of this application, an embodiment of this application further provides a sports guidance system for identifying the user's action from multiple dimensions through multiple electronic devices during the guidance of the user's exercise, so as to more accurately determine the user's sports guidance information.

[0073] Figure 6 FIG. 7 is an application scenario of the sports guidance system provided in an embodiment of this application. As shown in FIG. 7, the system includes at least one electronic device provided in the above embodiments of this application, such as a large-screen device, a mobile phone, and the like. In addition, the system can further include wearable devices, sensor devices, cameras, and other electronic devices. Figure 6

[0074] In this embodiment, the wearable device can be a smart watch, a smart bracelet, a wearable wrist device, smart glasses, an augmented reality (AR) / virtual reality (VR) device, or the like.

[0075] ​It should be understood that in the embodiments of the present application, the wearable device can also be a general term of devices that can be worn, such as glasses, gloves, watches, clothing, and shoes, which are designed and developed by applying wearable technology to daily wear. The wearable device is a portable device that can be directly worn on the body or integrated into the clothes or accessories of the user. The wearable device is not only a hardware device, but also a powerful function realized through software support and data interaction and cloud interaction. The general wearable smart device includes a full function, a large size, and can realize complete or partial functions without relying on a smart phone, such as a smart watch or smart glasses, and focuses on only one type of application function and needs to cooperate with other devices, such as a smart phone, such as various types of smart bracelets, smart jewelry, and the like.

[0076] In the embodiments, the sensor device can be a floor mat, a wearable device, or the like, which is installed with a sensor. The sensor can be a pressure sensor, an acceleration sensor, a gyroscope sensor, or the like, and the embodiments are not limited thereto.

[0077] The pressure sensor is used to sense a pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor can be arranged inside a floor mat, such as a dance mat or a yoga mat. There are many types of pressure sensors, such as a resistance type pressure sensor, an inductance type pressure sensor, a capacitance type pressure sensor, and the like. The capacitance type pressure sensor can include at least two parallel plates with conductive material. When a force acts on the pressure sensor, the capacitance between the electrodes changes, and the electronic device connected to the pressure sensor determines the intensity of the pressure according to the change in the capacitance. When pressure acts on the floor mat, the electronic device connected to the floor mat can detect the intensity of the pressure according to the pressure sensor and calculate the position of the pressure.

[0078] The acceleration sensor (ACC) can be installed on the wearable device. When the user wears the wearable device on the arm and moves, the wearable device can be used to detect the magnitude of the acceleration of the user's arm in each direction (generally three axes), which can be used as a user action collector to perceive the moving action of the arm in the front, back, left, right, up, and down directions, and can be converted into virtual scene actions such as punching, swinging a racket, jumping, and swinging a fishing rod in a game or the like.

[0079] The gyroscope sensor can be used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor. The gyroscope sensor 280B can be used for shooting anti-shake, navigation, and motion sensing games.

[0080] In the embodiments, the camera can be a component of an electronic device or an independent electronic device, and the embodiments are not limited thereto.

[0081] The operation process of the exercise guidance system provided in this embodiment will be described below.

[0082] The process of providing exercise guidance to users by an exercise guidance system is continuous. During this process, the various electronic devices within the exercise guidance system need to continuously display relevant exercise information and collect user data.

[0083] Large-screen devices (such as smart TVs) are typically used to install apps that provide exercise guidance services due to their large display screens. After the user opens an exercise program in the app, the large-screen device displays a first preset image corresponding to that exercise program and a first user image captured locally. For details, please refer to the preceding description; this embodiment will not repeat them here.

[0084] The mobile phone is used to capture images of a second user. Specifically, based on the mobile phone's portability and ease of placement, the user can place the phone in a different location than the large-screen device (e.g., opposite the large-screen device) to capture images of the second user from a different angle.

[0085] Compared to large-screen devices, mobile phones have stronger image capture capabilities. For example, see... Figure 7 As shown, cameras on large-screen devices typically lack zoom capabilities, and their image capture range is usually a fixed area. When a user is obstructed by obstacles (such as walls or sofas in the living room), and there is insufficient space for movement, the camera on the large-screen device may only capture a partial image of the user, making it impossible to recognize user movements. Unlike large-screen devices, mobile phones usually have automatic zoom capabilities. By automatically adjusting the focal length of the lens in the camera, they can change the image capture range, thereby capturing a complete image of the user. This addresses, to some extent, the problem of large-screen devices requiring a large amount of space when guiding users through exercise. Optionally, in the system provided in this embodiment, the mobile phone can be controlled to increase the number of second user images captured, while the large-screen device can be controlled to reduce the number of first user images captured, thereby improving the quality of the user image capture.

[0086] Wearable devices (such as smartwatches) can monitor a user's vital signs, including heart rate (HR), blood pressure, body temperature, and blood oxygen saturation. Furthermore, when a user exercises while wearing a wearable device, it can record the movement trajectory and amplitude of the area where the device is worn (e.g., wrist or ankle), as well as the vibration and intensity of the exercise. The vibration can be used to assess the stability of the user's movements, while the intensity indicates the severity of the exercise.

[0087] The pressure sensor device (for example, a floor mat with pressure sensors) can be used to detect the distance between the user's feet, the user's weight, the number of footstep jumps, the jump frequency, and the like.

[0088] It should be noted that in this embodiment, when only the user's feet are in contact with the pressure sensor device, the distance between the two pressure points detected by the pressure sensor device is the distance between the user's feet. During the jumping process, the user's feet repeatedly perform the action of stepping on the ground. The pressure sensor device can count the number of jumps of the user according to the change of the stepping pressure of the user. According to the number of jumps and the length of time of jumping, the jump frequency can be calculated.

[0089] The camera is used to capture the third user image. Specifically, based on the characteristics that the camera is convenient to move and place, the user can place the camera at a position different from the large-screen device and the mobile phone, so as to capture the user image from an angle different from the large-screen device and the mobile phone.

[0090] In the system provided in this embodiment, each electronic device can communicate with each other through wireless communication technologies such as wireless fidelity (WiFi), Bluetooth, cellular data service, or can be connected through a physical entity (for example, a USB line). In this embodiment, the user data collected by each electronic device is referred to as one-dimensional data. After each electronic device collects different dimensional user data, the user data is first sent to the central computing device in the system for comprehensive calculation to determine the multi-dimensional comprehensive motion guidance information of the user. Then, the central display device displays the comprehensive motion guidance information. In this embodiment, the central computing device and the central display device can be the same or different.

[0091] For example, as shown in the system in Figure 6 In some embodiments, as shown in Figure 8A The mobile phone as the central computing device performs comprehensive calculation on the user data collected by each electronic device in the system, and the large-screen device as the central display device displays the motion guidance information of the user according to the calculation result. Compared with the large-screen device, the mobile phone usually has strong computing power. Therefore, taking the mobile phone as the central computing device can improve the calculation rate of multi-dimensional user data.

[0092] In other embodiments, as shown in Figure 8BAs shown in the figure, the large-screen device simultaneously serves as the central computing device and the central display device. That is, the smart TV needs to not only comprehensively calculate the user data collected by each electronic device in the system, but also display the guidance information of the user according to the calculation result. The system provided in this embodiment can reduce the data interaction between each electronic device, save the resource consumption of each electronic device, and the like.

[0093] In other embodiments, as shown in the figure, the mobile phone simultaneously serves as the central computing device and the central display device. That is, the mobile phone needs to not only comprehensively calculate the user data collected by each electronic device in the system, but also display the guidance information of the user according to the calculation result. The system provided in this embodiment can reduce the data interaction between each electronic device, improve the calculation rate of multi-dimensional user data, and save the resource consumption of each electronic device, and the like. Figure 8C Since the large-screen device has poor computing capability relative to the mobile phone, when the large-screen device has the situation of running lag, long calculation time, or being unable to recognize user data, as shown in the figure, the mobile phone is set as the central computing device and the central display device to guide the user to perform the exercise. If the large-screen device does not have the above situation, the user data is calculated locally. In the calculation process, for the user images obtained by the central computing device from different electronic devices, such as the first user image and the second user image, if the deflection angles of the virtual users in the two images are the same or similar (for example, within 10°), one of them can be selected for recognition, so as to reduce the calculation pressure of the central computing device and improve the calculation efficiency. For example, the first user image is recognized by the large-screen device to guide the user to perform the exercise, and the mobile phone assists the large-screen device to recognize other user data, such as personal physical parameters, user weight, jumping height, jumping frequency, and the like.

[0094] Figure 9 It should be noted that in this embodiment, when the mobile phone serves as the central computing device or the central display device, the mobile phone can install the same exercise App as the large-screen device and log in to the same user account, so as to directly obtain the relevant exercise data, such as the demonstration image, from the application server in the process of calculating the multi-dimensional user data.

[0095] It can be understood that when the mobile phone installs the exercise App and simultaneously serves as the central computing device or the central display device, the mobile phone can directly guide the user to exercise in other scenes such as outdoors, thereby solving the problem that the current user can only exercise indoors or must be in front of the smart screen.

[0096] It can be understood that when the mobile phone installs the exercise App and simultaneously serves as the central computing device or the central display device, the mobile phone can directly guide the user to exercise in other scenes such as outdoors, thereby solving the problem that the current user can only exercise indoors or must be in front of the smart screen.

[0097] The exercise guidance information that can be determined by the system provided in this embodiment includes the project recommendation information before the exercise starts, and the action score, posture correction information, action evaluation information, and health prompt information after the exercise starts.​

[0098] After opening a fitness app on a large-screen device, the app can identify the user's body type before the workout begins and recommend suitable workouts based on that body type.

[0099] In some embodiments, the large-screen device identifies the user's body shape based on the user's image. Specifically, the large-screen device pre-stores multiple standard body shape images, in which the demonstrator's deflection angle differs. Therefore, when identifying the user's body shape, the large-screen device can first identify the deflection angle of the virtual user in the user image and determine the corresponding standard body shape image based on that deflection angle. Subsequently, the user image is compared with the corresponding standard body shape image to determine the deviation between the user's body shape and the standard body shape. When the user's body shape is heavier than the standard body shape, the smart TV can recommend fat-loss exercises, such as "aerobics." When the user's body shape is thinner than the standard body shape, the smart TV can recommend muscle-building and shaping exercises, such as "sculpting yoga" or "whole-body stretching."

[0100] In other embodiments, when the system is equipped with a body fat measurement device, the large-screen device can obtain the user's body fat information through the device. This information may include overall body fat percentage and localized body fat percentage. Based on the user's body fat information, the large-screen device can recommend suitable exercise programs. For example, if the user's overall body fat percentage is higher than a preset value, fat-reducing exercises, such as aerobics, can be recommended. If the user's leg body fat percentage is higher than a preset value, leg fat-reducing programs can be recommended. If the user's waist body fat percentage is higher than a preset value, waist fat-reducing programs can be recommended.

[0101] In some other embodiments, see Figure 10 As shown, the large-screen device can capture user images and measure the distance L1 between the centers of the virtual user's feet and the virtual user's height H1. Then, based on parameters fed back from the pressure sensor, it measures the distance L2 between the centers of the user's feet. Finally, based on the ratio H1 / L1 = H2 / L2, it estimates the user's height H2. The large-screen device can recommend corresponding sports activities to the user based on their height. For example, if the user's height is less than 1.3 meters, it considers the user to be a child and recommends children's dance, children's gymnastics, and other sports activities.

[0102] In some other embodiments, the large-screen device can also recommend exercise programs to the user based on the user's exercise habits. For example, if the electronic device detects that the user usually engages in "aerobics," then the electronic device can prioritize recommending various "aerobics" exercises to the user.

[0103] After the user selects the target exercise item, the large-screen device starts to guide the user to exercise. Since the user exercise is a continuous process, the center computing device needs to determine exercise guidance information according to the user data of each dimension every interval of a preset time (for example, 0.5 s, 1 s, 2 s, etc.) during the operation of the exercise guidance system. The exercise guidance information includes: action score, action evaluation icon, posture correction prompt information, user health prompt information, etc. The following will be described respectively. Figure 11

[0104] (1) Determination of action score

[0105] In this embodiment, the center computing device can calculate the action score of the user according to the user image. The action score can include the stage score for the single-dimension user action, the total score, and the comprehensive score for the multi-dimension user action.

[0106] The stage score refers to the score calculated by the center computing device for the user data at a certain time in a single dimension, such as the stage score at T1 time, the stage score at T2 time, etc. The stage score includes the local stage score for a specific body part of the user and the overall stage score for the whole body of the user. Among them, the local stage score includes the head stage score, the arm stage score, the waist stage score, the leg stage score, etc. The overall stage score is the average of all local stage scores at the same time (for example, at T1 time).

[0107] The total score is used to represent the overall exercise situation of the user participating in a certain exercise item, including the local total score and the overall total score. Among them, the local total score is used to represent the exercise situation of the user's local (such as the head and arms) in the whole exercise process, and the local total score is equal to the average of all corresponding local stage scores in the corresponding dimension. The overall total score is the average of all local total scores in the corresponding dimension.

[0108] For example, for the first user image collected by the large-screen device, the total score of the user's arms is equal to the average of all arm stage scores related to the first user image. The overall total score of the user is equal to the average of the head total score, the arm total score, the waist total score, and the leg total score related to the first user image.

[0109] The comprehensive score is used to represent the comprehensive situation of the multi-dimension user action. In one example, the comprehensive score is equal to the average of the overall total scores of multiple dimensions. For example, the total score is equal to the average of the front overall total score, the left overall total score, and the back overall total score of the user.

[0110] ​As described above, the overall stage score, local total score, overall total score, and comprehensive score are all calculated based on the local stage scores. Therefore, the central computing device calculates the local stage scores using image recognition algorithms, and then determines the remaining scores through mathematical operations.

[0111] The following example, using the calculation of the local stage score of the first user image at time T, illustrates the method for determining the local stage score. First, the central computing device, referring to the method shown in step S202 above, uses the ResNet algorithm to identify the deflection angle of the virtual user in the first user image and determines the corresponding demonstration image based on this deflection angle. Then, the electronic device uses a skeletal node algorithm to identify the key skeletal nodes of the demonstrator in the demonstration image and the key skeletal nodes of the virtual user in the first user image. These key skeletal nodes include those of the head, shoulders, elbows, wrists, hips, knees, and ankles. Finally, the positional information of the key skeletal nodes of each part of the demonstrator and the virtual user is compared. Based on the positional differences, the similarity between the virtual user's action and the demonstration action is determined, thereby determining the user's local stage score at time T.

[0112] For example, the positional information of the virtual user's head skeletal nodes is compared with that of the demonstrator to determine the head stage score. The positional information of the virtual user's arm skeletal nodes (including shoulder, elbow, and wrist skeletal nodes) is compared with that of the demonstrator to determine the arm stage score. The positional information of the virtual user's waist skeletal nodes (e.g., hip skeletal nodes) is compared with that of the demonstrator's legs skeletal nodes (e.g., knees, ankles, etc.) to determine the leg stage score.

[0113] In one example, for Figure 6 The motion scene shown below, the user images acquired by the central device are respectively Figure 12A The first user image shown (the virtual user's deflection angle is 90°, representing the image from the user's right side) Figure 12B The second user image shown (the virtual user's deflection angle is -90°, which is the image of the user's left side) Figure 12C The third user image shown is a virtual user image with a deflection angle of 0°, representing a frontal view of the user. Referring to Table 1, after recognizing these images, the central computing device can obtain user action scores, as shown in Table 1.

[0114] Table 1 User Action Rating Table

[0115]

[0116] Furthermore, in some embodiments, the central computing device can also compare the motion trajectory information collected by the wearable device with the corresponding standard motion trajectory information to identify user actions and determine various scores for the user actions. Alternatively, based on the acceleration changes of the user's limbs detected by the wearable device, the device can determine the limb shaking, thereby determining the stability of the user's limbs, and further determining the score of the user's actions based on the stability.

[0117] In other embodiments, user actions can be categorized into static actions and rapid actions based on the duration of the action. Static actions are typically held for a longer period, such as yoga poses and stretching exercises. Conversely, static actions are usually sustained for a shorter time, such as jumping. For rapid actions, such as jumping, the central computing device can also identify and score the rapid action based on user data collected by pressure sensors. For example, the central computing device can compare the user's jump height and frequency with parameters such as the jump height and frequency of a current action standard, and determine the score of the user's jumping action based on the comparison result.

[0118] For example, see Figure 13 As shown, when calculating the user's jump height, the central computing device can track and identify the user, determining the user's position on the user image before jumping (i.e., position 1) and position 2 on the user image when reaching the highest point of the jump, thereby determining the height difference D1 between position 1 and position 2 on the user image. Subsequently, based on the ratio D1 / D2 = H1 / H2, the user's jump height D2 is determined. Here, H1 is the height of the virtual user on the user image, and H2 is an estimated value of the user's actual height. Compared to existing motion guidance systems, the system provided in this embodiment has stronger rapid motion recognition and analysis capabilities.

[0119] After determining the scores for each user action, the central computing device sends the scores to the central display device for display.

[0120] In some embodiments, the large-screen device can display action evaluation information based on the overall stage score of the user's actions at preset time intervals. For example, when the user's overall stage score is higher than 90 points, the large-screen device displays action evaluation information such as "Perfect" or "Very good". When the user's overall stage score is between 80 and 90 points, it intelligently displays action evaluation information such as "Keep it up" or "Keep trying".

[0121] In other embodiments, see Figure 14As shown, after the end of a movement item, the smart TV can display the comprehensive score of the user for the movement item, the overall total score of each dimension (such as the total score of the front, the total score of the left side, and the total score of the right side), and the local total score of each side of the user (such as the total score of the head, the total score of the arm, the total score of the waist, and the total score of the legs, etc.). Through the total score of different parts, the user can determine the movement of each part of the body during the movement, and can more detailedly analyze the fitness movement and the movement state. Through the comprehensive score, the user can understand the comprehensive movement state.

[0122] (2) Posture correction prompt information

[0123] In a possible implementation, when the central computing device detects that a certain local stage score (for example, the arm stage score) of the user movement is lower than a preset value, the position difference between the virtual user arm and the demonstrator arm is determined, and posture correction information is displayed according to the position difference to prompt the user to adjust the movement posture. For example, when the virtual user's arm is lower than the position of the demonstrator's arm, the central computing device can determine the posture correction prompt information as: "Please raise the arm". After determining the posture correction prompt information, the central computing device can send it to the central display device for display.

[0124] In another possible implementation, the central computing device also compares the distance between the user's two feet with the standard distance between the two feet corresponding to the current movement to determine whether the user's two feet movement is standard, for example, whether the distance between the user's two legs is too large or not enough, and displays corresponding prompt information to remind the user to adjust, etc.

[0125] In other embodiments, for some jumping movements (for example, skipping movements), the central computing device can compare the jumping frequency of the user with the standard jumping frequency of the current movement. When the jumping frequency of the user is higher than the preset value (for example, 15%) of the jumping frequency, the central computing device controls the central display device to prompt the user to reduce the jumping frequency to reduce the movement intensity. When the jumping frequency of the user is lower than the preset value (for example, 15%) of the jumping frequency, the central computing device controls the central display device to prompt the user to increase the jumping frequency to increase the movement intensity.

[0126] In addition, the central device can also correct the jumping height of the user. For example, when the jumping height of the user is higher than the preset value (for example, 15%) of the standard jumping height, the central computing device controls the central display device to prompt the user to reduce the jumping height to reduce the movement intensity and save the physical strength. When the jumping height of the user is higher than the preset value (for example, 15%) of the standard jumping height, the central computing device controls the central display device to prompt the user to increase the jumping height to increase the movement intensity.

[0127] (3) health prompt information

[0128] In this embodiment, the central computing device can determine the health prompt information according to the personal physical parameter collected by the wearable device, and the central display device displays the health prompt information to prompt the user whether the user can currently perform the exercise. For example, when the heart rate of the user is greater than 160 beats per minute, the user is prompted to "fast heart rate, please rest before exercise". Or, when the systolic pressure of the user is greater than 160 mmHg, or the diastolic pressure of the user is greater than 120 mmHg, the smart TV prompts the user to "high blood pressure, please rest before exercise".

[0129] In some other embodiments, the central computing device can determine the exercise intensity of the user according to the heart rate of the user. For example, when the heart rate of the user is below 60% HRmax, it is determined that the exercise intensity level of the user is "low intensity". When the heart rate of the user is between 60-80% HRmax, it is determined that the exercise intensity level of the user is "medium intensity". When the heart rate of the user is above 80% HRmax, it is determined that the exercise intensity level of the user is "high intensity". Wherein, HRmax is the maximum value of the heart rate of the user. Generally, HRmax = 220 - user age, wherein the user age is determined according to the user input value. For example, when the age of the user is 20 years old, the HRmax of the user is 220-20 = 200. When the electronic device detects that the exercise intensity of the user is equal to or higher than the intensity level of the exercise item, the user is reminded to reduce the exercise intensity.

[0130] In addition, when the smart TV or the mobile phone obtains a plurality of user images taken from different angles, the smart TV or the mobile phone can establish a three-dimensional exercise model for the user in combination with the user images from different angles. The three-dimensional exercise model can show part or all of the process of the user participating in the exercise item. When the electronic device displays the three-dimensional model, the electronic device can control the three-dimensional model to rotate according to the user instruction, so as to show the exercise posture of the user from different angles.

[0131] In summary, the exercise guidance system provided by the embodiments of the present application includes at least two electronic devices, which can collect multi-dimensional user data and guide the user to perform exercise according to the multi-dimensional user data. Compared with the exercise guidance of the user according to the single-dimensional user data collected by a single channel, the system provided by the embodiments can improve the accuracy of exercise guidance, for example, reduce the inaccuracy of action scoring.

[0132] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0133] The embodiment of the present application further provides an electronic device configured to execute the user information determination method provided by the embodiment of the present application. The electronic device provided by the embodiment of the present application compares the user image with the second preset image corresponding to the deflection angle of the user image after determining the deflection angle of the user image, and determines the motion guidance information of the user, thereby improving the accuracy of user action recognition. Therefore, the user does not need to be limited by the deflection angle during the process of performing the motion exercise, and can select the deflection angle relative to the electronic device according to the user's own preference, thereby having a good user experience.

[0134] Figure 15 A structural schematic diagram of an electronic device is shown. The electronic device 200 can include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charge management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a loudspeaker 270A, a receiver 270B, a microphone 270C, a headset interface 270D, a sensor module 280, a key 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc.

[0135] It can be understood that the structure shown in the embodiment of the present application does not constitute a specific limitation on the electronic device 200. In other embodiments of the present application, the electronic device 200 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0136] For example, when the electronic device 200 is a mobile phone or a tablet computer, it can include all the components shown in the figure, or only include part of the components shown in the figure.

[0137] For example, when the electronic device 200 is a smart television, a smart screen, or other large-screen devices, it can also include all the components shown in the figure, or only include part of the components shown in the figure.

[0138] The processor 210 can include one or more processing units, for example: the processor 210 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0139] The controller can be the nerve center and command center of the electronic device 200. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0140] The memory can also be provided in the processor 210, for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory can save instructions or data that have just been used or are recycled by the processor 210. If the processor 210 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 210, thus improving the efficiency of the system.

[0141] In some embodiments, the processor 210 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0142] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 210 can include multiple sets of I2C buses. The processor 210 can be coupled to a charger, a flash, a camera 293, etc. through different I2C bus interfaces, respectively.

[0143] The I2S interface can be used for audio communication. In some embodiments, the processor 210 can include multiple sets of I2S buses. The processor 210 can be coupled to the audio module 270 through an I2S bus, to achieve communication between the processor 210 and the audio module 270. In some embodiments, the audio module 270 can deliver audio signals to the wireless communication module 260 through the I2S interface.

[0144] The PCM interface can also be used for audio communication, to sample, quantize and encode analog signals. In some embodiments, the audio module 270 and the wireless communication module 260 can be coupled through a PCM bus interface.

[0145] In some embodiments, the audio module 270 can also deliver audio signals to the wireless communication module 260 through the PCM interface. Both the I2S interface and the PCM interface can be used for audio communication.

[0146] The UART interface is a universal serial data bus, used for asynchronous communication. The bus can be a bidirectional communication bus. It converts data to be transmitted between serial communication and parallel communication.

[0147] In some embodiments, the UART interface is usually used to connect the processor 210 and the wireless communication module 260. For example, the processor 210 communicates with the Bluetooth module in the wireless communication module 260 through the UART interface, to achieve Bluetooth functionality. In some embodiments, the audio module 270 can deliver audio signals to the wireless communication module 260 through the UART interface, to achieve the function of playing music through Bluetooth earphones.

[0148] The MIPI interface can be used to connect the processor 210 and peripheral devices such as the display screen 294 and the camera 293. The MIPI interface includes the camera serial interface (CSI), the display serial interface (DSI), etc. In some embodiments, the processor 210 and the camera 293 communicate through the CSI interface, to achieve the shooting function of the electronic device 200. The processor 210 and the display screen 294 communicate through the DSI interface, to achieve the display function of the electronic device 200.

[0149] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 210 to the camera 293, the display 294, the wireless communication module 260, the audio module 270, the sensor module 280, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, an MIPI interface, etc.

[0150] The USB interface 230 is an interface that complies with the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 230 can be used to connect a charger to charge the electronic device 200, and can also be used to transmit data between the electronic device 200 and a peripheral device. It can also be used to connect a headset to play audio through the headset. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0151] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation of the electronic device 200. In some other embodiments of the present application, the electronic device 200 can also use different interface connection methods or combinations of multiple interface connection methods.

[0152] The charging management module 240 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 240 can receive charging input from a wired charger through the USB interface 230. In some wireless charging embodiments, the charging management module 240 can receive wireless charging input through a wireless charging coil of the electronic device 200. The charging management module 240 can charge the battery 242 while also providing power to the electronic device through the power management module 241.

[0153] The power management module 241 is used to connect the battery 242, the charging management module 240, and the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240 to provide power to the processor 210, the internal memory 221, the external memory, the display 294, the camera 293, and the wireless communication module 260, etc. The power management module 241 can also be used to monitor parameters such as battery capacity, battery cycle count, battery health status (leakage, impedance), etc.

[0154] In some other embodiments, the power management module 241 can also be disposed in the processor 210. In some other embodiments, the power management module 241 and the charging management module 240 can also be disposed in the same device.

[0155] The wireless communication function of the electronic device 200 can be implemented by the antenna 1, the antenna 2, the mobile communication module 250, the wireless communication module 260, the modem processor, and the baseband processor, etc.

[0156] The antenna 1 and the antenna 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 200 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In some other embodiments, the antennas can be used in combination with a tuning switch.

[0157] The mobile communication module 250 can provide a solution including 2G / 3G / 4G / 5G wireless communication applied to the electronic device 200. The mobile communication module 250 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 250 can receive electromagnetic waves by the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 250 can also amplify the signals modulated by the modem processor, and convert the signals into electromagnetic waves radiated by the antenna 1.

[0158] In some embodiments, at least part of the functional modules of the mobile communication module 250 can be arranged in the processor 210. In some embodiments, at least part of the functional modules of the mobile communication module 250 and at least part of the modules of the processor 210 can be arranged in the same device.

[0159] The modem processor can include a modulator and a demodulator. The modulator is used to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the loudspeaker 270A, the microphone 270B, etc.), or displays an image or a video through the display screen 294. In some embodiments, the modem processor can be an independent device. In some other embodiments, the modem processor can be independent of the processor 210, and arranged in the same device as the mobile communication module 250 or other functional modules.

[0160] The wireless communication module 260 can provide a solution for wireless communication, including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc., which are applied to the electronic device 200. The wireless communication module 260 can be one or more devices that integrate at least one communication processing module. The wireless communication module 260 receives electromagnetic waves via the antenna 2, frequency-modulates and filters the electromagnetic wave signals, and transmits the processed signals to the processor 210. The wireless communication module 260 can also receive signals to be transmitted from the processor 210, frequency-modulate them, amplify them, and radiate them as electromagnetic waves via the antenna 2.

[0161] In some embodiments, the antenna 1 and the mobile communication module 250 of the electronic device 200 are coupled, and the antenna 2 and the wireless communication module 260 are coupled, so that the electronic device 200 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidu navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and / or a satellite based augmentation systems (SBAS).

[0162] The electronic device 200 implements a display function through a GPU, a display screen 294, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 can include one or more GPUs, which execute program instructions to generate or change display information.

[0163] The display screen 294 is configured to display images, videos, etc. For example, the teaching videos and user action video in the embodiments of the present application, the display screen 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diodes (QLED), etc. In some embodiments, the electronic device 200 can include one or N display screens 294, where N is a positive integer greater than 1.

[0164] The electronic device 200 can implement the photographing function through the ISP, the camera 293, the video codec, the GPU, the display screen 294, and the application processor, etc.

[0165] The ISP is configured to process the data fed back by the camera 293. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also optimize the algorithm for the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, etc. of the shooting scene. In some embodiments, the ISP can be arranged in the camera 293.

[0166] The camera 293 is configured to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into a standard RGB, YUV, etc. format image signal. In some embodiments, the electronic device 200 can include one or N cameras 293, where N is a positive integer greater than 1.

[0167] The digital signal processor is used to process digital signals, in addition to being able to process digital image signals, it can also process other digital signals. For example, when the electronic device 200 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0168] The video codec is used to compress or decompress digital video. The electronic device 200 can support one or more video codecs. In this way, the electronic device 200 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0169] The NPU is a neural-network (NN) calculation processor, which can quickly process input information by drawing on the structure of a biological neural network, such as drawing on the transmission mode between human brain neurons, and can also constantly self-learn. Through the NPU, the electronic device 200 can realize intelligent cognition and other applications, such as: image recognition, face recognition, speech recognition, text understanding, etc.

[0170] In the embodiments of the present application, the NPU or other processor can be used to analyze and process images in the video stored by the electronic device 200, etc.

[0171] The external memory interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 200. The external memory card communicates with the processor 210 through the external memory interface 220 to realize data storage functions. For example, music, video, etc. Files are saved in the external memory card.

[0172] The internal memory 221 can be used to store computer executable program codes, which include instructions. The processor 210 executes various function applications and data processing of the electronic device 200 by running the instructions stored in the internal memory 221. The internal memory 221 can include a program storage area and a data storage area. The program storage area can store an operating system and at least one application program required by a function (such as a sound playing function, an image playing function, etc.). The data storage area can store data created during the use of the electronic device 200 (such as audio data, a phone book, etc.).

[0173] In addition, the internal memory 221 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0174] The electronic device 200 can implement audio functions through an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, and an application processor, etc.

[0175] The audio module 270 is configured to convert a digital audio signal into an analog audio signal for output, and to convert an analog audio input into a digital audio signal. The audio module 270 can also be configured to encode and decode audio signals. In some embodiments, the audio module 270 can be disposed in the processor 210, or some functional modules of the audio module 270 can be disposed in the processor 210.

[0176] The speaker 270A, also referred to as a "loudspeaker", is configured to convert an audio electrical signal into a sound signal. The electronic device 200 can listen to music or listen to a hands-free call through the speaker 270A, for example, the speaker can play the comparison and analysis result provided by the embodiments of the present application.

[0177] The receiver 270B, also referred to as a "earpiece", is configured to convert an audio electrical signal into a sound signal. When the electronic device 200 is on a call or receiving a voice message, the receiver 270B can be placed close to the ear of a person to receive the voice.

[0178] The microphone 270C, also referred to as a "microphone" or "sound transducer", is configured to convert a sound signal into an electrical signal. When making a call or sending a voice message, a user can speak into the microphone 270C close to the mouth to input a sound signal into the microphone 270C. The electronic device 200 can be provided with at least one microphone 270C. In other embodiments, the electronic device 200 can be provided with two microphones 270C, in addition to collecting sound signals, the electronic device 200 can also implement a noise reduction function. In other embodiments, the electronic device 200 can also be provided with three, four or more microphones 270C, to collect sound signals, reduce noise, and also identify the source of the sound, implement directional recording functions, etc.

[0179] The earphone interface 270D is configured to connect a wired earphone. The earphone interface 270D can be a USB interface 230, or a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0180] The keys 290 include a power key, a volume key, etc. The keys 290 can be mechanical keys. Alternatively, the keys 290 can be touch keys. The electronic device 200 can receive key inputs, and generate key signal inputs related to user settings and function control of the electronic device 200.

[0181] The motor 291 can generate a vibration prompt. The motor 291 can be used for incoming call vibration prompt, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. The motor 291 can also correspond to different vibration feedback effects for touch operations acting on different regions of the display screen 294. Different application scenarios (such as time reminders, received messages, alarms, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0182] The indicator 292 can be an indicator light, which can be used to indicate a charging state, a power change, and can also be used to indicate messages, missed calls, notifications, etc.

[0183] The SIM card interface 295 is used to connect a SIM card. The SIM card can be inserted into or pulled out of the SIM card interface 295 to realize contact and separation with the electronic device 200. The electronic device 200 can support one or N SIM card interfaces, and N is a positive integer greater than 1. The SIM card interface 295 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. The same SIM card interface 295 can simultaneously insert multiple cards. The types of the multiple cards can be the same or different. The SIM card interface 295 can also be compatible with different types of SIM cards. The SIM card interface 295 can also be compatible with external storage cards. The electronic device 200 interacts with a network through the SIM card to realize functions such as calling and data communication. In some embodiments, the electronic device 200 uses an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 200 and cannot be separated from the electronic device 200.

[0184] Embodiments of the present application provide a chip system, which includes a memory and a processor. The processor executes a computer program stored in the memory to implement the motion guidance information determination method provided in each of the above embodiments. The chip system can be a single chip or a chip module composed of multiple chips.

[0185] Embodiments of the present application provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the motion guidance information determination method provided in each of the above embodiments.

[0186] It should be appreciated that the processor referred to in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0187] It should also be understood that the memory referred to in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM) and direct rambus RAM (DR RAM).

[0188] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0189] In the description of the embodiments of the present application, unless otherwise specified, " / " means the meaning of or, for example, A / B can mean A or B; "and / or" herein is only a description of the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can mean that A exists alone, A and B exist together, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0190] The above terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments, unless otherwise specified, the meaning of "multiple" is two or more than two.

[0191] In the description of the present application, the reference to "one embodiment" or "some embodiments" means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in other some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0192] The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for determining motion guidance information, applied to electronic devices, characterized in that, The method includes: Display the first preset image and the captured user image; Identify the deflection angle of the virtual user's entire body relative to a reference position in the user image; A second preset image corresponding to the deflection angle is determined from the set of demonstration actions, wherein the set of demonstration actions stores preset images corresponding to different deflection angles for each demonstration action; Based on the user image and the second preset image, determine the user's motion guidance information; The first preset image and the second preset image present the same or different deflection angles of the demonstration action. The first preset image and the second preset image correspond to the same demonstration action. The deflection angle refers to the angle required to rotate from the reference position to the current position.

2. The method according to claim 1, characterized in that, The method further includes: Display the second preset image while simultaneously displaying the first preset image; or, Replace the first preset image with the second preset image.

3. The method according to claim 1 or 2, characterized in that, The exercise guidance information includes at least one of the following: movement score, movement comments, movement evaluation icon, posture correction information, calories burned, and exercise duration.

4. An electronic device, characterized in that, The electronic device is configured to: Display the first preset image and the captured user image; Identify the deflection angle of the virtual user's entire body relative to a reference position in the user image; A second preset image corresponding to the deflection angle is determined from the set of demonstration actions, wherein the set of demonstration actions stores preset images corresponding to different deflection angles for each demonstration action; Based on the user image and the second preset image, determine the user's motion guidance information; Wherein, the first preset image and the second preset image present the same or different deflection angles of the demonstration action, the first preset image and the second preset image correspond to the same demonstration action, and the deflection angle refers to the angle required to rotate from the reference position to the current position.

5. The electronic device according to claim 4, characterized in that, The electronic device is also configured to: Display the second preset image while simultaneously displaying the first preset image; or, Replace the first preset image with the second preset image.

6. The electronic device according to claim 4 or 5, characterized in that, The exercise guidance information includes at least one of the following: movement score, movement comments, movement evaluation icon, posture correction information, calories burned, and exercise duration.

7. A motion guidance system, characterized in that, It includes a first electronic device and at least one second electronic device; the first electronic device is configured to perform the method according to any one of claims 1-3; The first electronic device determines the user's exercise guidance information based on locally collected user data and user data collected by at least one second electronic device. The at least one second electronic device may be the same or different.

8. The system according to claim 7, characterized in that, The user data collected by the first electronic device is a first user image, and the user data collected by the second electronic device is a second user image; the deflection angle of the virtual user in the first user image is different from that in the second user image.

9. The system according to claim 8, characterized in that, The user data collected by the second electronic device is body fat information; The first electronic device recommends corresponding sports activities to the user based on the body fat information.

10. The system according to claim 8, characterized in that, The user data collected by the second electronic device is personal vital sign parameters; The first electronic device displays health tips based on the individual's vital signs parameters, and the health tips are used to prompt the user whether they are able to engage in exercise.

11. The system according to claim 10, characterized in that, The personal vital signs parameters include at least one of the following: heart rate, blood pressure, body temperature, and blood oxygen saturation.

12. The system according to claim 11, characterized in that, The user data collected by the second electronic device includes the user's jump height and jump frequency.

13. The system according to any one of claims 8-12, characterized in that, The first electronic device and the at least one second electronic device are connected via far-field communication technology, near-field communication technology, or a physical entity.

14. A chip system, characterized in that, The chip system includes a memory and a processor, the processor executing a computer program stored in the memory to implement the method as described in any one of claims 1-3.

15. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-3.

Images