A method and device for judging fitness venue

Abstract

The present application relates to the field of intelligent fitness technology, and specifically to a method, device and equipment for determining a fitness venue, a posture collection method. The method includes: starting a first application and selecting a first fitness project of the first application; when the venue within the perspective of the first device does not meet the venue conditions of the first fitness project, a second device equipped with a camera can be connected, and the second device collects a first image; based on the first image, it can be determined that the venue within the perspective of the second device meets the venue conditions; and the application interface of the first fitness project is presented. In the present application, when the venue within the perspective of the device running the fitness application does not meet the venue conditions of the fitness project, the venue within the perspective of other devices can be used as the fitness venue of the fitness project, so that the user can exercise, thereby limiting the user to the perspective of a specific device and choosing a fitness venue more freely.

Description

Technical Field

[0001] The present application relates to the field of intelligent fitness technology, and in particular to a method and device for determining a fitness venue. Background Art

[0002] As people's living standards improve, they pay more and more attention to their physical health. Fitness exercises are increasingly recognized as an effective means to enhance physical fitness, and fitness-related products are also welcomed and loved by more and more consumers.

[0003] Usually, if users want to do more professional fitness, they need to go to professional institutions such as gyms and rehabilitation centers to receive evaluation or guidance from fitness professionals. Going to professional institutions to do fitness takes a lot of time for users, making it difficult to do fitness anytime and anywhere.

[0004] As the group of fitness enthusiasts grows year by year, fitness time is also fragmented, and people hope to be able to exercise at home. When users exercise at home, they can use the TV to collect images of the user's fitness movements, and then evaluate or guide the user's fitness movements, so that users can perform more professional fitness activities at home. Usually, the field of view (FOV) of the camera of a TV is limited, and the position and posture of the TV are inconvenient to adjust. Therefore, when the space in front of the smart TV is limited (for example, the living room is small, and there are coffee tables, sofas and other furniture in front of the TV), the user needs to perform fitness movements away from the lens perspective of the smart TV. In this case, it is difficult for the TV to collect images of the user's fitness movements. Therefore, there is an urgent need for a solution that allows users to choose a fitness venue more freely without being limited to the lens perspective of the TV. Summary of the invention

[0005] The embodiments of the present application provide a method and device for determining a fitness venue, which can enable users to choose a fitness venue more freely without being limited to the lens viewing angle of a specific device.

[0006] In a first aspect, an embodiment of the present application provides a method for determining a fitness venue, which can be applied to a first device equipped with a camera; the method includes: a first application can be started, and a first fitness project of the first application can be selected; when the venue within the field of view of the first device does not meet the venue conditions of the first fitness project, a second device equipped with a camera can be connected, and the second device captures a first image; based on the first image, it can be determined that the venue within the field of view of the second device meets the venue conditions; and an application interface of the first fitness project is presented.

[0007] That is to say, through the fitness venue judgment method provided in the embodiment of the present application, when the venue within the perspective of the device running the fitness application does not meet the venue conditions of the fitness project, the venue within the perspective of other devices can be used as the fitness venue of the fitness project to facilitate users to perform fitness, thereby limiting users to the perspective of specific devices and allowing them to choose a fitness venue more freely.

[0008] In a possible implementation, before the second device captures the first image, the method further includes: the first device receives a second image from the second device, where the second image is an image captured by the second device when the first user is located in the first venue; the first device determines, based on the position of the image of the first user in the second image, that the viewing angle of the second device does not completely cover the posture of the first user when located in the first venue; and the first device provides first prompt information, where the first prompt information is used to prompt the second device to adjust its posture direction.

[0009] That is to say, in this implementation, the first device can determine whether the viewing angle of the second device covers the user's posture. If it does not cover the user's posture, it can provide prompt information to prompt the user how to adjust the posture of the second device so that the viewing angle of the second device can cover the user's posture.

[0010] In a possible implementation, the venue conditions include a first size, and the size of the first venue is greater than or equal to the first size; before the second device captures the first image, the method also includes: the first device receives a third image from the first device, and the third image is an image captured by the second device when the first user is located in the first venue; the first device determines, based on the position of the image of the first user in the third image, that the size of an area in the first venue that falls within the field of view of the second device is smaller than the first size; and the first device provides second prompt information, and the second prompt information is used to prompt the second device to adjust the posture direction.

[0011] That is to say, in this implementation, the first device can determine whether the venue within the field of view of the second device's lens meets the venue size required for the fitness project. If not, it can provide prompt information for how to adjust the posture of the second device so that the venue within the field of view of the second device can meet the venue size required for the fitness project.

[0012] In a possible implementation, the method also includes: when the venue within the field of view of the first device meets the venue conditions, the first device provides third prompt information, and the third prompt information is used to prompt the first user to perform a first fitness item in a first area, and the first area is an area in the venue within the field of view of the first device.

[0013] That is to say, in this implementation, when the venue within the field of view of the first device meets the venue conditions, the user can be prompted with the specific location of the fitness program action in the venue within the field of view of the first device to facilitate the user to better perform fitness.

[0014] In a possible implementation, the method also includes: when the first fitness project corresponds to multiple shooting perspectives, the first device is connected to a third device equipped with a camera, and the direction facing the camera of the third device is different from the direction facing the camera of the first device; receiving a fourth image from the third device, the fourth image is an image captured by the third device when the second user performs the first fitness project; the first device determines that the capture time of the fifth image is the same as the capture time of the fourth image, and the fifth image is an image captured by the first device when the second user performs the first fitness project; the first device displays the fourth image and the fifth image side by side.

[0015] That is to say, in this implementation, when it is necessary to capture the user's fitness posture from multiple angles, the first device can be connected to the third device, and the cameras of the two devices face different directions, so that the user's fitness posture can be captured from two angles, and the first device can simultaneously display the images captured by the first device and the second device at the same time, so that the user can conveniently watch the fitness posture captured from different angles at the same time.

[0016] In a possible implementation, the method also includes: when the first fitness item corresponds to fitness equipment, and the objects within the field of view of the first device do not include fitness equipment, the first device determines a first object used to replace the fitness equipment from the objects within the field of view of the first device; the first device provides fourth prompt information, and the fourth prompt information is used to prompt the user to use the first object to perform the first fitness item.

[0017] That is to say, for fitness programs that require fitness equipment, if the fitness equipment is not within the field of view of the first device, a substitute object for the fitness equipment can be selected from the existing objects within the field of view of the first device, and a user can be provided with a prompt to use the substitute object to perform the fitness program. In this way, in the absence of fitness equipment, substitute objects for fitness equipment can be automatically recommended to the user, thereby improving the user's fitness experience.

[0018] In a possible implementation, the method also includes: when the first fitness item corresponds to fitness equipment, and the objects within the field of view of the first device do not include fitness equipment, the first device determines the second fitness item corresponding to the second object, and the second object is one or more objects within the field of view of the first device; the first device provides fifth prompt information, and the fifth prompt information is used to prompt the user to use the second object to perform the second fitness item.

[0019] That is to say, in this implementation, for fitness items that require fitness equipment, if the fitness equipment is not within the field of view of the first device, other fitness items can be selected based on the existing objects within the field of view of the first device, and the user can be prompted to use the used objects to perform the other fitness items. In this way, in the absence of fitness equipment, fitness items can be automatically recommended to users, thereby improving the user's fitness experience.

[0020] In a possible implementation, the method also includes: connecting a second device configured with a camera includes: the first device provides a sixth prompt message, and the sixth prompt message is used to prompt the user to turn on the wireless connection function of the first device; the first device responds to the user-initiated operation of turning on the wireless connection function, and turns on the wireless connection function, so as to connect to the second device through the wireless connection function.

[0021] That is, in this implementation, when the second device needs to be connected, the first device can prompt the user to turn on the wireless connection function of the first device so as to connect to the first device through the wireless connection function.

[0022] In a possible implementation, the method further includes: the sixth prompt information corresponds to a first function area displayed by the first device, and the operation of turning on the wireless connection function is an operation acting on the first function area.

[0023] That is to say, in this implementation, the user can turn on the wireless connection function by directly touching the function area displayed on the first device, thereby improving the user operation experience.

[0024] In a possible implementation, the method also includes: before the second device captures the first image, the method also includes: sending an application start request to the second device to trigger the second device to start the second application or display a prompt message for prompting the user to start the second application, the second application corresponds to the first application, and the first image is an image captured by the second application by calling the camera of the second device.

[0025] That is to say, the first device can actively request the second device to open the relevant application, so that the relevant application can call the camera of the second device to collect images to determine whether the venue within the field of view of the second device meets the fitness venue required by the fitness project.

[0026] In the second aspect, an embodiment of the present application provides a first device, including: a processor, a memory, and a transceiver; the memory is used to store computer instructions; when the first device is running, the processor executes the computer instructions, so that the first device executes: starting a first application, selecting a first fitness project of the first application; when the venue within the field of view of the first device does not meet the venue conditions of the first fitness project, connecting a second device equipped with a camera, and the second device captures a first image; based on the first image, determining that the venue within the field of view of the second device meets the venue conditions; and presenting the application interface of the first fitness project.

[0027] In one possible implementation, before the second device captures the first image, the processor executes computer instructions so that the first device further executes: receiving a second image from the second device, where the second image is an image captured by the second device when the first user is located in the first venue; determining, based on the position of the image of the first user in the second image, that the viewing angle of the second device does not completely cover the posture of the first user when located in the first venue; and providing first prompt information, where the first prompt information is used to prompt the second device to adjust its posture direction.

[0028] In one possible implementation, the venue conditions include a first size, and the size of the first venue is greater than or equal to the first size; before the second device captures the first image, the processor executes computer instructions so that the first device further executes: receiving a third image from the first device, the third image being an image captured by the second device when the first user is located in the first venue; determining, based on the position of the image of the first user in the third image, that the size of an area in the first venue that falls within the field of view of the second device is smaller than the first size; and providing second prompt information, the second prompt information being used to prompt the second device in a posture adjustment direction.

[0029] In one possible implementation, the processor executes computer instructions so that the first device further executes: when the venue within the field of view of the first device meets the venue conditions, providing third prompt information, the third prompt information is used to prompt the first user to perform a first fitness item in a first area, and the first area is an area in the venue within the field of view of the first device.

[0030] In one possible implementation, the processor executes computer instructions so that the first device further executes: when the first fitness project corresponds to multiple shooting perspectives, connecting a third device equipped with a camera, the direction facing the camera of the third device is different from the direction facing the camera of the first device; receiving a fourth image from the third device, the fourth image being an image captured by the third device when the second user performs the first fitness project; the first device determining that the capture time of the fifth image is the same as the capture time of the fourth image, the fifth image being an image captured by the first device when the second user performs the first fitness project; and displaying the fourth image and the fifth image side by side.

[0031] In one possible implementation, the processor executes computer instructions so that the first device further executes: when the first fitness item corresponds to fitness equipment and the objects within the field of view of the first device do not include fitness equipment, determining a first object used to replace the fitness equipment from the objects within the field of view of the first device; and providing fourth prompt information, wherein the fourth prompt information is used to prompt the user to use the first object to perform the first fitness item.

[0032] In one possible implementation, the processor executes computer instructions so that the first device further executes: when the first fitness item corresponds to fitness equipment and the objects within the field of view of the first device do not include fitness equipment, determining a second fitness item corresponding to the second object, where the second object is one or more objects within the field of view of the first device; and providing fifth prompt information, where the fifth prompt information is used to prompt the user to use the second object to perform the second fitness item.

[0033] In one possible implementation, the processor executes computer instructions so that the first device also performs: providing a sixth prompt message, where the sixth prompt message is used to prompt the user to turn on the wireless connection function of the first device; in response to a user-initiated operation of turning on the wireless connection function, turning on the wireless connection function so as to connect to the second device through the wireless connection function.

[0034] In a possible implementation, the sixth prompt information corresponds to the first function area displayed by the first device, and the operation of turning on the wireless connection function is an operation acting on the first function area.

[0035] In one possible implementation, before the second device captures the first image, the processor executes computer instructions so that the first device also executes: sending an application start request to the second device to trigger the second device to start the second application or display a prompt message for prompting the user to start the second application, the second application corresponds to the first application, and the first image is an image captured by the second application by calling the camera of the second device.

[0036] It can be understood that the device provided in the second aspect is used to execute the method provided in the first aspect, and therefore, the beneficial effects that can be achieved can refer to the corresponding beneficial effects mentioned above.

[0037] In a third aspect, an embodiment of the present application provides a device for determining a fitness venue, which is configured on a first device; the determination device includes: a starting unit, used to start a first application and select a first fitness project of the first application; a connecting unit, used to connect a second device equipped with a camera when the venue within the field of view of the first device does not meet the venue conditions of the first fitness project, and the second device captures a first image; a determining unit, used to determine, based on the first image, that the venue within the field of view of the second device meets the venue conditions; and a presenting unit, used to present the application interface of the first fitness project.

[0038] In a possible implementation, the judging device also includes a providing unit; before the second device captures the first image, the connecting unit is also used to receive a second image from the second device, where the second image is an image captured by the second device when the first user is located in the first venue; the determining unit is also used to determine, based on the position of the image of the first user in the second image, that the viewing angle of the second device does not completely cover the posture of the first user when located in the first venue; the providing unit is used to provide first prompt information, and the first prompt information is used to prompt the second device to adjust its posture direction.

[0039] In a possible implementation, the venue conditions include a first size, and the size of the first venue is greater than or equal to the first size; the judgment device also includes a providing unit; before the second device captures the first image, the connection unit is also used to receive a third image from the first device, and the third image is an image captured by the second device when the first user is located in the first venue; the determination unit is also used to determine, based on the position of the image of the first user in the third image, that the size of an area in the first venue that falls within the field of view of the second device is smaller than the first size; the providing unit is used to provide second prompt information, and the second prompt information is used to prompt the second device to adjust the posture direction.

[0040] In a possible implementation, the judgment device also includes a providing unit; the providing unit is used to provide third prompt information when the venue within the field of view of the first device meets the venue conditions, and the third prompt information is used to prompt the first user to perform a first fitness item in a first area, and the first area is an area in the venue within the field of view of the first device.

[0041] In one possible implementation, the connection unit is also used to connect a third device equipped with a camera when the first fitness project corresponds to multiple shooting perspectives, and the direction facing the camera of the third device is different from the direction facing the camera of the first device; the connection unit is also used to receive a fourth image from the third device, and the fourth image is an image captured by the third device when the second user performs the first fitness project; the determination unit is also used to determine that the capture time of the fifth image is the same as the capture time of the fourth image, and the fifth image is an image captured by the first device when the second user performs the first fitness project; the presentation unit is also used to display the fourth image and the fifth image side by side.

[0042] In a possible implementation, the judgment device also includes a providing unit; the determination unit is also used to determine a first object used to replace the fitness equipment from the objects within the perspective of the first device when the first fitness item corresponds to fitness equipment and the objects within the perspective of the first device do not include fitness equipment; the providing unit is used to provide fourth prompt information, and the fourth prompt information is used to prompt the user to use the first object to perform the first fitness item.

[0043] In a possible implementation, the judgment device also includes a providing unit; the determination unit is also used to determine the second fitness item corresponding to the second object when the first fitness item corresponds to fitness equipment and the objects within the field of view of the first device do not include fitness equipment, and the second object is one or more objects within the field of view of the first device; the providing unit is used to provide fifth prompt information, and the fifth prompt information is used to prompt the user to use the second object to perform the second fitness item.

[0044] In one possible implementation, the judgment device also includes a providing unit and an enabling unit; the providing unit is used to provide a sixth prompt information, and the sixth prompt information is used to prompt the user to enable the wireless connection function of the first device; the enabling unit is used to respond to the user-initiated operation of enabling the wireless connection function, and enable the wireless connection function so as to connect to the second device through the wireless connection function.

[0045] In a possible implementation, the sixth prompt information corresponds to the first function area displayed by the first device, and the operation of turning on the wireless connection function is an operation acting on the first function area.

[0046] In one possible implementation, before the second device captures the first image, the connection unit is also used to send an application start request to the second device to trigger the second device to start the second application or display a prompt message for prompting the user to start the second application. The second application corresponds to the first application, and the first image is an image captured by the second application by calling the camera of the second device.

[0047] It can be understood that the judgment device provided in the third aspect is used to execute the method provided in the first aspect. Therefore, the beneficial effects that can be achieved can refer to the corresponding beneficial effects mentioned above.

[0048] In a fourth aspect, an embodiment of the present specification provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method provided in the first aspect.

[0049] In a sixth aspect, an embodiment of the present application provides a computer program product, which, when the program code contained in the computer program product is executed by a processor in an electronic device, implements the method provided in the second aspect.

[0050] The method, device and apparatus for determining a fitness venue provided in the embodiments of the present application can, when the venue within the viewing angle of the device running the fitness application does not meet the venue conditions of the fitness project, use the venue within the viewing angle of other devices as the fitness venue for the fitness project, so that the user can exercise, thereby limiting the user to the viewing angle of a specific device and choosing a fitness venue more freely. In addition, the device running the fitness application can be a large-screen device, and the other device can be a portable device. In the embodiments of the present application, the large-screen device can be used to adjust the posture of the portable device so that the posture of the user when performing the fitness action falls into the lens viewing angle of the portable device, so that the posture of the user when performing the fitness action can be photographed by the portable device, so that when the user exercises in different venues, the posture of the portable device can be quickly adjusted to capture the posture of the user when performing the fitness action. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of the structure of an intelligent fitness system provided in an embodiment of the present application;

[0052] Figure 2 A schematic diagram of the structure of a large-screen device provided in an embodiment of the present application;

[0053] Figure 3A A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0054] Figure 3B A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0055] Figure 4A A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0056] Figure 4B A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0057] Figure 5A A schematic diagram of a fitness scene provided in an embodiment of the present application;

[0058] Figure 5B A schematic diagram of a fitness scene provided in an embodiment of the present application;

[0059] Figure 5C A schematic diagram of a fitness scene provided in an embodiment of the present application;

[0060] Fig. 6A A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0061] Figure 6B A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0062] Figure 6C A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0063] Fig.6D A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0064] Fig. 6E A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0065] Fig. 6F A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0066] Figure 6G A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0067] Fig. 7A A schematic diagram of information interaction between a large-screen device and a portable device provided in an embodiment of the present application;

[0068] Figure 7B A schematic diagram of a user interface of a portable device provided in an embodiment of the present application;

[0069] Fig. 8A A schematic diagram of displaying an image on a portable device for viewing by a user;

[0070] Figure 8B A schematic diagram of images displayed on a large-screen device for users to watch;

[0071] Fig. 9A A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0072] Fig. 9B A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0073] Fig. 9C A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0074] Fig.9D A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0075] Fig.9E A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0076] Fig.9F A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0077] Fig. 10AA schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0078] Fig. 10B A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0079] Fig.11 A schematic diagram of a fitness scene provided in an embodiment of the present application;

[0080] Fig.12 A schematic diagram of a user interface of a large-screen device provided in an embodiment of the present application;

[0081] Fig.13A A schematic diagram of the skeleton nodes of a user exercising at a certain shooting angle provided in an embodiment of the present application;

[0082] Fig. 13B A schematic diagram of the skeleton nodes of a user exercising obtained from another shooting angle provided in an embodiment of the present application;

[0083] Fig.14 A flowchart of a large-screen smart fitness method assisted by a mobile phone provided in an embodiment of the present application;

[0084] FIG15A is a user's fitness posture image collected by a mobile phone according to an embodiment of the present application;

[0085] FIG15B is a schematic diagram of a fitness scene provided in an embodiment of the present application;

[0086] FIG15C is an image of a user's fitness posture captured by a television according to an embodiment of the present application;

[0087] Fig.16 A schematic block diagram of a fitness venue determination device provided in an embodiment of the present application;

[0088] Fig.17 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0089] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0090] In the description of this specification, "one embodiment" or "some embodiments" etc. means that one or more embodiments of this specification include a specific feature, structure or characteristic described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0091] In the description of this specification, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this specification, "multiple" means two or more than two.

[0092] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0093] Generally, large-screen devices such as smart TVs have built-in cameras that can be used to capture images of users' fitness movements. Combined with relevant image processing algorithms, they can identify users' fitness movements in real time, and then evaluate or guide users' fitness movements, allowing users to perform more professional fitness activities at home.

[0094] For families, TVs are usually placed in the living room. The field of view (FOV) of the camera of a smart TV is limited, and the position and posture of the smart TV are inconvenient to adjust. Therefore, when the space in front of the smart TV is limited (for example, the living room is small, and there are furniture such as coffee tables and sofas in front of the TV), users need to perform fitness movements out of the camera angle of the smart TV. In this case, it is difficult for the smart TV to collect images of the user's fitness movements. In addition, for fitness movements that require the use of special terrain (such as wall stretching, corridor pull-ups, etc.), the execution site is often out of the camera angle of the smart TV, making it impossible for the smart TV to collect this type of fitness movement. In addition, for fitness movements such as squats, lunges and turns, it is necessary to collect three-dimensional spatial information for movement evaluation. However, it is often difficult for smart TVs to collect three-dimensional spatial information. Therefore, there are great limitations in collecting user fitness movements through smart TVs and evaluating or guiding fitness movements.

[0095] The embodiment of the present application provides a method for determining a fitness venue. In this method, an electronic device, such as a large-screen device such as a smart TV, Huawei R smart screen, etc., can receive and display an image captured by a portable device, so that the user can observe from a distance (for example, 3 meters, 5 meters, etc.) whether the lens angle of the portable device covers the posture of the user when he is in a specific fitness venue based on the image displayed on the large screen. If the lens angle of the portable device does not cover the posture of the user when he is in a specific fitness venue, the user can further adjust the posture of the portable device so that the lens angle of the portable device covers the posture of the user when he is in the fitness venue. If the lens angle of the portable device covers the posture of the user when he is in the fitness venue, the posture of the user when he is exercising in the fitness venue can be collected by the portable device to evaluate or guide the fitness movements. The method for determining a fitness venue provided in the embodiment of the present application allows users to perform fitness movements without being limited to the venue under the lens angle of the large-screen device, and can choose a fitness venue more freely.

[0096] Next, an intelligent fitness system provided by an embodiment of the present application is introduced.

[0097] See also Figure 1 The smart fitness system provided in the embodiment of the present application includes a portable device 100 and an electronic device 200.

[0098] The portable device 100 may be a portable electronic device with an image capture function, such as a mobile phone, a tablet computer, a digital camera, a personal digital assistant (PDA), a wearable device, a laptop computer, etc. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices equipped with iOS, Android, Windows or other operating systems. The above-mentioned portable electronic device may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). The embodiment of the present application does not specifically limit the type of portable electronic device.

[0099] The electronic device 200 may be an electronic device equipped with a larger display screen, such as a smart TV, Huawei R Smart Screen, etc.

[0100] The electronic device 200 can be connected to the portable device 100 through a network. For example, the network can be a local area network (LAN) or a wide area network (WAN) (such as the Internet). The network between the electronic device 200 and the portable device 100 can be implemented using any known network communication protocol, and the above network communication protocol can be various wired or wireless communication protocols, such as Ethernet, universal serial bus (USB), Firewire, Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), New Radio (NR), Bluetooth, Wireless Fidelity (Wi-Fi) and other communication protocols.

[0101] Next, combine Figure 2 , the structure of the electronic device 200 is introduced as an example.

[0102] like Figure 2As shown, the electronic device 200 may include a processor 210, a memory 220, a wireless communication module 230, a display screen 240, a camera 250, an audio module 260, a wired communication module 270, a power switch 280, and the like.

[0103] The processor 210 can be used to read and execute computer-readable instructions. In a specific implementation, the processor 210 may mainly include a controller, an operator, and a register. Among them, the controller is mainly responsible for decoding instructions and issuing control signals for operations corresponding to the instructions. The operator is mainly responsible for saving register operands and intermediate operation results temporarily stored during the execution of instructions. In some embodiments, the processor 210 can be used to parse signals received by the wireless communication module 230 and / or the wired communication module 270, such as parsing image data, obtaining images, and analyzing the user posture in the image (for example, determining the angle between joints), and evaluating the user posture, etc. In an embodiment of the present application, the image data may include video data. It can be understood that a video is composed of multiple frames of images, so video data can also be referred to as image data.

[0104] The processor 210 may be used to perform corresponding processing operations according to the analysis results, such as playing a video or displaying an image through the display screen 240. For another example, the evaluation results may be displayed through the display screen 240.

[0105] The memory 220 is coupled to the processor 210 and is used to store various software programs and / or multiple sets of instructions.

[0106] The wireless communication module 230 may include a Bluetooth module 231 and a Wi-Fi module 232. The Bluetooth module 231 and / or the Wi-Fi module 232 may interact with the portable device 100 for data, such as receiving image data from the portable device 100. In a specific implementation, the Bluetooth module 231 may provide a classic Bluetooth (e.g., Bluetooth 2.1) module and / or a Bluetooth low energy (BLE) module. The Wi-Fi module 232 may include one or more of a Wi-Fi direct module, a Wi-Fi LAN module, or a Wi-Fi softAP module. In some embodiments, the Bluetooth module 231 and / or the Wi-Fi module 232 may transmit signals, such as broadcast Bluetooth signals, beacon signals, etc., so that other devices (e.g., the portable device 100) can discover the electronic device 200, and a wireless communication connection between the other device and the electronic device 200 can be established to interact with data. In some embodiments, the electronic device 200 can access the Internet through the Wi-Fi module 232, thereby establishing a communication connection with a server on the Internet (e.g., a video playback website server). In some embodiments, the wireless communication module 230 may further include an infrared communication module (not shown). The infrared communication module may communicate with a remote controller or other device through infrared remote control technology.

[0107] The camera 250 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format.

[0108] The display screen 240 can be used to display images, videos, etc. The display screen 240 can be a liquid crystal display (LCD), an organic light-emitting diode (OLED) display screen, an active-matrix organic light emitting diode (AMOLED) display screen, a flexible light-emitting diode (FLED) display screen, a quantum dot light-emitting diode (QLED) display screen, etc. In some embodiments, the display screen 240 can be a screen larger than 40 inches, for example, a 65-inch screen.

[0109] The audio module 260 may include a speaker 261. The processor 210 may convert a digital audio signal into an analog audio signal, and the speaker 261 may convert an analog audio signal into a sound signal. The audio module 260 may also include a microphone 262. The microphone 262 may be used to convert a sound signal into an electrical signal. When sending a voice control instruction, the user may issue a control voice, and the sound signal of the control voice may be input into the microphone 262 to be converted into an electrical signal. The processor 210 may parse the electrical signal to obtain the control instruction.

[0110] The wired communication module 270 may include a USB module 271, so that the electronic device 200 can communicate with other devices (such as the portable device 100) via a USB data line. The wired communication module 270 may also include a high definition multimedia interface (HDMI) (not shown). The electronic device 200 can communicate with a device such as a set top box (STB) via HDMI.

[0111] The power switch 280 may be used to control the supply of power to the large-screen device 200 .

[0112] Next, an example is given to introduce a method for determining a fitness venue provided in an embodiment of the present application.

[0113] In some embodiments, the method may be performed by the electronic device 200. Figure 3A As shown, the electronic device 200 may be installed with applications such as video, photo album, and fitness. The electronic device 200 may respond to a user-initiated operation for opening the fitness application and open the fitness application. The operation for opening the fitness application may be a touch operation, a voice operation command, or a control command of a remote controller.

[0114] like Figure 3B As shown, the fitness application may include one or more fitness items, such as wall stretching, wall handstand push-ups, pull-ups, squats, aerobics, etc. Exemplarily, one or more fitness items have posture information, and the posture information may include a single execution duration requirement of one or more postures and / or a standard value of the activity amplitude of a key bone node. The posture information is used to indicate the execution requirements or execution standards of one or more postures required or specified by the fitness item, such as the execution duration requirement of the posture, the standard value of the activity amplitude of the key bone node corresponding to the posture, etc. Exemplarily, the standard value of the activity amplitude may include a standard angle of the key bone node, etc. The standard angle of the key bone node may be the angle of the key bone node under the standard posture. For example, the key bone node of the squat may include a knee joint, and the standard angle of the knee joint may be the angle of the knee joint under the standard squat posture, such as 90°.

[0115] In the embodiment of the present application, posture may refer to the body shape when the body performs an action.

[0116] The electronic device 200 can respond to the user's operation of selecting a fitness project and determine the current fitness project. When or after the current fitness project is determined, the electronic device 200 can enter the image acquisition initialization state. In the image acquisition initialization state, the electronic device 200 can start its camera 250 to collect images. It can be understood that, usually for a fitness project, the user needs to perform the fitness project in a suitable venue, that is, the fitness project generally has specific venue requirements, which can be called venue conditions. Exemplarily, venue conditions can be associated with each fitness project in advance. In other words, each fitness project can have or correspond to preset venue conditions. The electronic device 200 can analyze whether the venue under the lens angle of view of its camera 250 is suitable for the venue requirements of the current fitness project based on the image captured by its camera 250. The current fitness project can also be called a fitness project selected by the user.

[0117] In some embodiments, see Figure 4A When the venue under the lens angle of the camera 250 meets the venue requirements of the current fitness program, the electronic device 200 can display information for prompting the user to start fitness, start the fitness program, and enter the fitness posture collection and analysis state. Figure 4A As shown, the information used to prompt the user to start fitness can be "Please start fitness".

[0118] In some embodiments, see Figure 4BAlthough the venue under the lens angle of the camera 250 meets the venue requirements of the current fitness program, the lens angle of the camera 250 may include objects such as furniture 301, which may affect the user's fitness movements. The electronic device 200 can select a suitable area within its lens angle of view so that the user can perform fitness movements. For example, Figure 4B As shown, the electronic device 200 can display area 302 to prompt the user to perform a fitness program in area 302. Area 302 can be a partial area of ​​the venue within the lens angle of view of the electronic device 200. The distance between the partial area and the furniture 301 is greater than the distance threshold. Exemplarily, the distance threshold can be determined by the electronic device 200 according to the venue requirements of the current fitness program. For example, the venue requirements include venue size requirements. If the size requirement is not less than 2m×2m, then the distance threshold is 2m.

[0119] In some embodiments, the venue under the lens angle of view of the camera 250 may not meet the venue requirements of the current fitness program.

[0120] In an illustrative example, the current fitness program is aerobics. It is understandable that the user needs to perform the body movements in aerobics in a relatively open field. Exemplarily, the field conditions for aerobics may include: the area of ​​the field is greater than or equal to the threshold value Y1, and there are no obstacles or shielding objects in the field. The threshold value Y1 may be a more empirical or experimental preset value, such as 2 meters × 2 meters, etc. Figure 5A , the venue under the lens angle of view of the camera 250 of the electronic device 200 is usually placed with furniture such as a coffee table 310 and a sofa 320, making it difficult for the user 400 to perform aerobics under the lens angle of view of the camera 250. In other words, the venue under the lens angle of view of the camera 250 of the electronic device 200 does not meet the venue conditions for aerobics. In specific implementation, the electronic device 200 can use a target recognition algorithm to analyze the image captured by the camera 250, and can identify the furniture such as a coffee table and a sofa included in the venue under the lens angle of view of the camera 250 to determine that the venue does not meet the venue conditions for aerobics. The target recognition algorithm can use a regional convolution neural network (RCNN) algorithm, a deep convolution neural network (DCNN) algorithm, etc.

[0121] In an illustrative example, the current fitness program is wall stretching. Exemplarily, the venue conditions for wall stretching may include: there is a wall in the venue that meets a preset height (for example, 2 meters), and there are no obstacles or shielding objects in front of the wall. Figure 5B, the camera 250 may include furniture such as a sofa 320. The sofa 320 is usually placed against a wall 330, so it is difficult for the user 400 to use the wall 330 under the camera 250's lens angle of view to perform the wall stretching fitness project. In other words, the venue under the camera 250's lens angle of view does not meet the venue conditions for wall stretching.

[0122] In an illustrative example, the current fitness program is pull-ups. Exemplarily, the venue conditions for pull-ups may include: there is a horizontal bar or other equipment that can achieve pull-ups in the venue. Figure 5C , the horizontal bar 340 does not fall within the viewing angle of the camera 250 , and therefore, the venue under the viewing angle of the camera 250 does not meet the venue conditions for pull-ups.

[0123] In some embodiments, if the venue under the lens angle of view of the camera 250 does not meet the venue requirements of the fitness program selected by the user, the electronic device 200 may directly refuse to start the fitness program. Fig. 6A , the electronic device 200 may display “The fitness program is not suitable, please exit”.

[0124] In some embodiments, when the user has not selected a fitness program, or when the venue under the viewing angle of the camera 250 does not meet the fitness program selected by the user, the electronic device 200 can recommend a fitness program to the user based on the venue under the viewing angle of the camera 250. Next, an example is given for illustration.

[0125] For an illustrative example, see Figure 6B , when the camera 250 includes the sofa 601 in the lens angle of view, the electronic device 200 can recommend to the user a fitness program suitable for being performed on the sofa, such as a hip bridge exercise. Figure 6B As shown, the electronic device 200 may display “Recommend hip bridge exercise on the sofa” to suggest or instruct the user to perform hip bridge exercise on the sofa.

[0126] For an illustrative example, see Figure 6C , when the camera 250 includes the table 602 in the lens angle of view, the electronic device 200 can recommend to the user a fitness program suitable for using or relying on the table, such as leg press exercise. Figure 6C As shown, the electronic device 200 may display “Recommend leg stretching exercise using a table” to suggest or instruct the user to use a table to perform leg stretching exercise.

[0127] Understandably, Figure 6B and Figure 6CThis is only used as an example to illustrate that the electronic device 200 recommends fitness programs according to the venue under the lens angle of the camera 250, and is not a limitation. The electronic device 200 can recommend different fitness programs according to different venues under the lens angle of the camera 250. For example, when the lens angle of the camera 250 includes dumbbells, the electronic device 200 can recommend dumbbell-related exercises (such as dumbbell shoulder press, dumbbell upright rowing, etc.). For another example, when the lens angle of the camera 250 includes a barbell, the electronic device 200 can recommend barbell-related exercises (such as lifting a barbell, etc.). And so on, I will not list them one by one here.

[0128] It should be noted that the sofa 601, table 602, dumbbells, barbells, etc. can be called fitness project identification objects. Specifically, the sofa 601 can be called an identification object for hip bridge exercise, the table 602 can be called an identification object for leg press exercise, the dumbbells can be called an identification object for dumbbell-related exercise, and the barbell can be called an identification object for barbell-related exercise. In specific implementation, fitness projects and their corresponding identification objects can be associated in advance to obtain associated data. The associated data can be stored in the electronic device 200 so that the electronic device 200 can recommend fitness projects to the user by identifying objects in the venue under the lens angle of the camera 250.

[0129] In some embodiments, when the fitness project selected by the user is a fitness project that requires the use of equipment, when there is no professional equipment required for the fitness project under the lens angle of the camera 250, the electronic device 200 can select an object similar to the professional equipment from the objects under the lens angle of the camera 250 to practice the corresponding action. In other words, the electronic device 200 can determine the equipment required for the fitness project from the objects under the lens angle of the camera 250 according to the fitness project. In the embodiment of the present application, the object similar to the professional equipment can also be referred to as an object that replaces the professional equipment, which can refer to an object that can play the same or similar function as the professional equipment, and the object can enable the user to complete the relevant action in the absence of professional equipment. The large-screen device can prompt the user to use the object similar to the professional equipment to perform the fitness project.

[0130] In an illustrative example, the fitness program selected by the user may be hip bridge exercise. It is understood that the professional equipment required for hip bridge exercise is usually a yoga mat or exercise mat. Fig.6D In the case where there is no yoga mat or exercise mat in the camera 250 lens angle of view, when the camera 250 lens angle of view includes objects such as sofa 603 and table 604, the electronic device 200 can prompt the user to perform hip bridge exercise on the sofa 603. For example, the electronic device 200 can display "recommend hip bridge exercise on the sofa" to suggest or instruct the user to perform hip bridge exercise on the sofa.

[0131] In another illustrative example, the fitness program selected by the user may be leg press exercise. It is understood that the professional equipment required for leg press exercise is usually a leg press machine, etc. Fig. 6E In the case where there is no leg stretcher in the camera 250 lens angle of view, when the camera 250 lens angle of view includes objects such as a sofa 603 and a table 604, the electronic device 200 can prompt the user to do leg stretching exercises with the table 604. For example, the electronic device 200 can display "recommended to do leg stretching exercises with the table" to suggest or instruct the user to do leg stretching exercises with the table.

[0132] In some embodiments, when the venue under the lens angle of view of the camera 250 does not meet the venue requirements of the fitness project selected by the user, the electronic device 200 can be connected to the portable device 100 to collect the user's posture when doing fitness with the help of the portable device 100. Since the posture of the portable device 100 is easy to adjust, the user can be separated from the lens angle of view of the camera 250 and choose the fitness venue more freely. The user can adjust the posture of the portable device 100 so that the lens angle of view of the camera of the portable device 100 can cover the fitness venue selected by the user. In other words, the user can adjust the posture of the portable device so that the venue within the lens angle of view of the camera of the portable device 100 meets the venue requirements of the fitness project selected by the user. Thereby, the posture of the user when doing fitness can be collected by the portable device 100. Next, examples are introduced in different embodiments.

[0133] In some embodiments, if the electronic device 200 and the portable device 100 are in an unconnected state, when the electronic device 200 determines that the venue under the lens angle of its camera is not suitable for the venue requirements of the current fitness program, the electronic device 200 may request or prompt the user to connect the electronic device 200 and the portable device 100.

[0134] For example, Fig. 6F As shown, the electronic device 200 may display “Please connect the portable device” to request the user to perform an operation to connect the electronic device 200 and the portable device 100. In one example, the area where “Please connect the portable device” is displayed may be a functional area, and the electronic device 200 may receive an operation initiated by the user in the functional area, turn on the wireless function (Bluetooth or Wi-FI), and broadcast a wireless signal (e.g., broadcast a Bluetooth signal or a Wi-Fi signal) so that the portable device 100 can find the electronic device 200 and realize a wireless connection between the electronic device 200 and the portable device 100. In one example, the electronic device 200 may receive a connection operation in the form of a voice operation instruction or a control instruction of a remote controller, etc., while displaying “Please connect the portable device”, and in response to the connection operation, turn on the wireless function and broadcast a wireless signal.

[0135] Exemplarily, the electronic device 200 may provide prompt information for directly prompting the user to turn on the wireless function. Figure 6G As shown, the electronic device 200 may display “Please turn on Bluetooth or Wi-Fi” so that the user can turn on Bluetooth or Wi-Fi of the electronic device 200. For example, the user can turn on Bluetooth or Wi-Fi in the “Settings” interface, “Device Connection” interface, or shortcut function interface of the electronic device 200 by manually enabling.

[0136] Exemplarily, the electronic device 200 may also prompt the user to connect the electronic device 200 and the portable device 100 in a wired manner.

[0137] In some embodiments, if the large-screen device 200 and the portable device 100 are in an unconnected state, when the electronic device 200 determines that the venue under the lens angle of its camera is not suitable for the venue requirements of the current fitness program, the electronic device 200 can automatically broadcast a Bluetooth signal or a beacon signal so that the portable device 100 can discover the electronic device 200 and then establish a network connection.

[0138] In some embodiments, the electronic device 200 may request or prompt the user to start a fitness application on the portable device 100. Fig. 6F , the electronic device 200 may display "Please open the fitness application on the portable device" to request or prompt the user to perform relevant operations. The fitness application on the portable device 100 and the fitness application on the electronic device 200 may be related applications, and the two may be used together. When there is a network connection between the portable device 100 and the electronic device 200, the fitness application on the portable device 100 and the fitness application on the electronic device 200 may exchange data.

[0139] In some embodiments, when or after the network connection is established between the electronic device 200 and the portable device 100, the electronic device 200 may send a fitness application start request to the portable device 100. Fig. 7A As shown, the electronic device 200 may display “Portable device connected” and “Requesting to open the fitness application on the portable device”.

[0140] In one example, the portable device 100 can respond to the fitness application start request and automatically start the fitness application. After the fitness application is started, the camera of the portable device 100 can be called to capture images or record videos.

[0141] In one example, the portable device 100 may, in response to the fitness application start request, request the user to start the fitness application, for example, Figure 7BAs shown, the portable device 100 may display “Please open the fitness application” to request or prompt the user to open the fitness application on the portable device 100. After the fitness application is opened, the camera of the portable device 100 may be called to capture images or record videos.

[0142] Next, for the convenience of description, the fitness application on the electronic device 200 may be referred to as a fitness application A1, and the fitness application on the portable device 100 may be referred to as a fitness application A2.

[0143] Exemplarily, the fitness application A2 can call the camera of the portable device 100 to capture images. It can be understood that a video is composed of multiple frames of images, and video capture or video recording can be referred to as image capture. Photos are usually single-frame images, and photo shooting can also be referred to as image capture. In other words, in the embodiment of the present application, image capture can refer to video capture or video recording, or it can refer to photo shooting. Correspondingly, the captured image can refer to an image in a video, or it can refer to a photo.

[0144] The portable device 100 may send the image captured by its camera to the electronic device 200. The electronic device 200 may display the image captured by the portable device 100.

[0145] It is understood that the user can freely select a fitness venue to perform the fitness program of his / her choice. The user can adjust the posture of the portable device 100 so that the fitness venue of his / her choice falls within the coverage of the lens angle of view of the portable device 100. The posture may refer to a posture and / or position in a three-dimensional space.

[0146] See also Fig. 8A Also see Figure 5A-Figure 5C When using the portable device 100 to collect user fitness images, the portable device 100 and the user need to maintain a certain distance (for example, 3 meters, 5 meters, etc.) so that the collected fitness images include the complete user posture. In the process of adjusting the posture of the portable device 100, the user can judge whether the user's posture at the fitness venue is completely within the lens angle of view of the portable device 100 by observing the image of the user at the fitness venue collected by the portable device 100, so as to determine the direction of further adjustment. Usually, the screen of the portable device 100 is small, and when the user is at the fitness venue of his choice, the distance between the user and the portable device 100 is far (for example, 3 meters, 5 meters, etc.). Therefore, it is inconvenient or difficult for the user to determine the direction of further adjustment by observing the image displayed on the screen of the portable device 100. In particular, it will be more inconvenient for users with poor eyesight (such as myopic users).

[0147] See also Figure 8B, the screen of the electronic device 200 is much larger than the screen of the portable device 100. In the embodiment of the present application, the screen of the electronic device 200 can be used to display the image captured by the portable device 100, so that the user can observe from a distance whether his posture in the fitness venue is completely within the lens angle of the portable device 100.

[0148] In some embodiments, the electronic device 200 may display the image captured by the portable device 100 in full screen.

[0149] In some embodiments, see Fig. 9A , the electronic device 200 can display the image captured by the portable device 100 in the area 241 on its display screen. In one example, the area 241 has a preset size so as to display the image captured by the portable device 100 in a large size. Taking the display screen of the electronic device 200 as an example of 65 inches, the size of the area 241 can be 60cm×70cm. In one example, the size of the area 241 is in a preset ratio to the size of the display screen of the electronic device 200, for example, the length of the area 241 is half of the length of the display screen of the electronic device 200, and the width of the area 241 is half of the width of the display screen of the electronic device 200.

[0150] Exemplarily, the user can adjust the posture of the portable device 100 so that the camera of the portable device 100 can face the venue selected by the user. The portable device 100 can respond to the user's operation and start recording a video. When the portable device 100 faces the venue selected by the user, the user can move to the venue selected by the user during the video recording process, so that the video recorded by the portable device 100 can have an image including the image of the user. The image including the image of the user in the video recorded by the portable device 100 facing the venue selected by the user can be called an image of the user when the user is located at the fitness venue selected by the user. That is, the portable device 100 can capture the image of the user when the user is located at the fitness venue selected by the user.

[0151] The portable device 100 can send the video it has recorded to the electronic device 200. For example, the portable device 100 can send the video it has recorded to the electronic device 200 in real time in the form of a video stream. The electronic device 200 can play the video stream it has received from the portable device 100, that is, the electronic device 200 can display the image in the video recorded by the portable device 100. The video recorded by the portable device 100 includes an image P1. Image P1 is an image of the user when the user is located at the fitness venue selected by the user. The electronic device 200 can determine whether the lens angle of the portable device 100 completely covers the user's posture based on the position of the user's image in the image P1. If the user's posture is not completely covered, the electronic device 200 can provide prompt information so that the user can further adjust the posture of the portable device 100. Image P1 can be one or more frames of images in the video recorded by the portable device 100. The multiple images can be images that are adjacent in sequence in the video.

[0152] For an example, see Fig. 9A , the image P1 can be shown as shown in the display in the area 241. It can be seen that the position of the portable device 100 is biased to the left, and the position of the portable device 100 needs to be adjusted to the right. Fig. 9A An arrow pointing to the right is shown, so that the user can adjust the position of the portable device 100 according to the indication of the arrow.

[0153] For an example, see Fig. 9B , the image P1 can be shown as in the display in the area 241. It can be seen that the position of the portable device 100 is lower, and the position of the portable device 100 needs to be adjusted upward. Fig. 9B An arrow pointing upward is shown, so that the user can adjust the position of the portable device 100 according to the indication of the arrow.

[0154] In one example, referring to FIG9C , the image P1 may be shown in the display of area 241 . It can be seen that the position of the portable device 100 is relatively close to the position of the user, and the portable device 100 needs to be adjusted away from the user. Fig. 9C An arrow pointing in the rear direction is shown, so that the user can adjust the position of the portable device 100 according to the indication of the arrow. Fig.9D , the image P1 may be shown as in the display in the area 241. It can be seen that the posture of the portable device 100 is biased toward the lower right, and the posture of the portable device 100 needs to be adjusted toward the upper left. Fig.9D An arrow pointing to the upper left is shown, so that the user can adjust the position of the portable device 100 according to the indication of the arrow.

[0155] Exemplarily, as described above, each fitness item corresponds to a venue condition. The electronic device 200 can determine whether the venue that falls within the lens angle of view of the portable device 100 meets the venue condition of the current fitness item. For example, it can be determined whether the size or dimensions of the venue that falls within the lens angle of view of the portable device 100 meets the venue size or dimensions required for the current fitness item. Specifically, the video recorded by the portable device 100 includes image P2. Image P2 is an image when the user is located at a fitness venue selected by the user. The electronic device 200 can determine whether the area in the venue where the user is located that falls within the lens angle of view of the portable device 100 meets the venue size required for the current fitness item based on the position of the user's image in image P2. Image P2 can be one or more frames of images in the video recorded by the portable device 100, and the multiple frames of images can be images that are adjacent in sequence in the video.

[0156] Taking the current fitness program as aerobics as an example, refer to Fig.9E , the image P2 can be shown as shown in the display of area 241, and the dotted box 701 shows the size of the venue corresponding to the aerobics. Fig.9E As shown, under the lens angle of the portable device 100, the size of the field where the user is located does not meet the field size corresponding to aerobics. In other words, the area of ​​the field where the user is located that falls within the lens angle of the portable device 100 does not meet the field size corresponding to aerobics, and the portable device 100 needs to be adjusted away from the user. Fig.9E An arrow pointing in the rear direction is shown, so that the user can adjust the position of the portable device 100 according to the indication of the arrow.

[0157] In some embodiments, as in 9A- Fig.9E As shown, the electronic device 200 can display an indication arrow or prompt information so that the user can adjust the position of the portable device 100 according to the indication arrow, or start fitness according to the prompt information. In one example, the indication arrow or prompt information can be determined by the electronic device 200 according to the position of the user's posture in the image. In another example, the indication arrow or prompt information can be determined by the portable device 100 according to the user's posture in the image, and the portable device 100 can send the indication arrow to the electronic device 200 so that the electronic device 200 displays the indication arrow.

[0158] In some embodiments, the electronic device 200 may play a voice prompt to remind the user to further adjust the direction of the portable device 100 or start exercising.

[0159] In some embodiments, the portable device 100 may play a voice prompt to remind the user to further adjust the direction of the portable device 100 or start exercising.

[0160] The image captured by the portable device 100 is displayed by the electronic device 200, so that the user can determine the further adjustment direction of the portable device 100 by observing the display screen of the large-screen device 100, thereby improving the user's operating experience of adjusting the posture of the portable device 100.

[0161] Through the above method, the portable device 100 can be adjusted to a suitable posture so as to collect the user's posture during fitness.

[0162] When the lens angle of view of the portable device 100 completely covers the user's posture, or when the lens angle of view of the portable device 100 completely covers the user's posture and the area of ​​the venue where the user is located that falls within the lens angle of view of the portable device 100 meets the venue size, the electronic device 200 can determine that the venue within the lens angle of view of the portable device 100 meets the venue conditions of the current fitness program.

[0163] Specifically, the video recorded by the portable device 100 includes an image P3. Image P3 is an image of the user at a fitness venue selected by the user. The electronic device 200 can determine whether the lens angle of view of the portable device 100 completely covers the user's posture based on the position of the user's image in the image P3. Alternatively, the electronic device 200 can determine whether the lens angle of view of the portable device 100 completely covers the user's posture based on the position of the user's image in the image P3, and determine whether the area in the venue where the user is located that falls within the lens angle of view of the portable device 100 meets the venue size required for the current fitness program. Image P3 can be one or more frames of images in the video recorded by the portable device 100, and the multiple frames of images can be images that are adjacent in sequence in the video.

[0164] In one example, see Fig.9F , image P3 can be shown as displayed in area 241, the lens angle of view of the portable device 100 completely covers the user's posture, and the area of ​​the venue where the user is located that falls within the lens angle of view of the portable device 100 meets the venue size. The electronic device 200 can determine that the venue within the lens angle of view of the portable device 100 meets the venue conditions of the current fitness project. In an example, the portable device 100 can determine that the venue within the lens angle of view of the portable device 100 meets the venue conditions of the current fitness project based on image P3, and send a notification message to the electronic device 200. The notification information can be used to notify the electronic device 200 that the venue within the lens angle of view of the portable device 100 meets the venue conditions of the current fitness project.

[0165] In some embodiments, when the electronic device 200 determines that the venue within the lens field of view of the portable device 100 meets the venue conditions of the current fitness project or receives a notification message from the portable device 100, the electronic device 200 can automatically enter the fitness posture collection and analysis state and present the application interface of the current fitness project.

[0166] In some embodiments, when the electronic device 200 determines that the venue within the lens angle of the portable device 100 meets the venue conditions of the current fitness program or receives a notification message from the portable device 100, a prompt message may be displayed to prompt the user to start fitness. Fig.9F As shown, the prompt information may be “The venue is suitable, please start exercising” to prompt the user to trigger the electronic device 200 to enter the fitness posture collection and analysis state, and present the application interface of the current fitness project.

[0167] In some embodiments, the user can trigger the electronic device 200 to enter the fitness posture collection and analysis state and present the application interface of the current fitness project through a preset fitness start posture. Fig. 10A As shown, the preset fitness start posture can be a posture of the user with both arms raised. In one example, the preset fitness start posture can be a posture of the user with both arms crossed. Etc., which will not be listed one by one here.

[0168] In some embodiments, the user can use voice commands to trigger the electronic device 200 to enter a fitness posture collection and analysis state and present an application interface for the current fitness project.

[0169] In some embodiments, the user can trigger the electronic device 200 to enter the fitness posture collection and analysis state and present the application interface of the current fitness project through remote control instructions.

[0170] In some embodiments, the user can trigger the electronic device 200 to start a fitness program through touch operation instructions, enter the fitness posture collection and analysis state, and present the application interface of the current fitness program.

[0171] In some embodiments, the application interface of the current fitness program can be Fig. 10B The fitness interface shown may include an area 242, which may be used to display images captured by the portable device 100. Exemplarily, the fitness interface may include an area 243, which may be used to display images including a standard posture.

[0172] In some embodiments, the portable device 100 can continuously collect images including the user's fitness posture, for example, record a video of the user's fitness posture, and send the video data to the electronic device 200. The electronic device 200 can play the video of the user's fitness posture, that is, display the image including the user's fitness posture. Exemplarily, when the electronic device 200 is in the fitness posture collection and analysis state, it can play an image including a standard posture of a fitness project to guide the user to correctly complete the movements specified in the fitness project. The electronic device 200 can display the image including the user's fitness posture and the image including the standard posture side by side. In one example, Fig. 10B As shown, the electronic device 200 can display a fitness interface when it is in the fitness posture collection and analysis state. The fitness interface may include area 242 and area 243. The large-screen device may display an image including the user's fitness posture in area 242, and may display an image including a standard posture in area 243.

[0173] The size of area 242 and the size of area 243 may be the same or different. Exemplarily, area 242 has a preset size so as to display the image captured by portable device 100 in a large size. Taking the display screen of electronic device 200 as an example of 65 inches, the size of area 242 may be 60 cm×70 cm. In one example, the size of area 242 is in a preset ratio to the size of the display screen of electronic device 200, for example, the length of area 242 is half the length of the display screen of electronic device 200, and the width of area 242 is half the width of the display screen of electronic device 200. Exemplarily, area 243 may be implemented with reference to area 242.

[0174] In some embodiments, the electronic device 200 or the portable device 100 can identify the user's fitness posture from an image including the user's fitness posture. For example, an RCNN algorithm or a DCNN algorithm can be used to identify the user's fitness posture. The electronic device 200 or the portable device 100 can analyze the user's fitness posture to determine whether the user has correctly completed the actions specified in the fitness program. If the analysis of the user's fitness posture is performed by the portable device 100, the portable device 100 can send the analysis result to the electronic device 200.

[0175] Exemplarily, as described above, the current fitness project may have posture information, and the posture information may include a single execution duration requirement of one or more postures of the current fitness project. The electronic device 200 or the portable device 100 may determine the execution duration of a single fitness posture performed by the user based on multiple frames of images including the user's fitness posture. If the execution duration of any fitness posture performed by the user is equal to or greater than the single execution duration requirement of the corresponding posture in the posture information, it can be determined that the fitness posture performed by the user is a valid fitness posture. If the current fitness project has a number of fitness posture requirements, each time the user performs an effective fitness posture, the electronic device 200 or the portable device 100 accumulates the effective fitness posture and the effective fitness posture performed previously. The effective fitness posture performed previously is the effective fitness posture performed by the user during this fitness process. Taking the "arms raised horizontally" fitness project as an example, it requires 50 arms raised horizontally, and the execution duration requirement of each arm raised horizontally is 5s. When a user performs the "Raise Arms" fitness program, if the execution time of raising arms horizontally reaches or exceeds 5 seconds, then this raising of arms horizontally is considered as a valid raising of arms horizontally. By accumulating each valid raising of arms horizontally during the user's "Raise Arms Vertically" fitness program, the number of valid raising of arms horizontally during the user's "Raise Arms Vertically" fitness program can be obtained.

[0176] For example, the image including the user's fitness posture captured by the portable device 100 may include a timestamp, so that the user can determine the execution time of the user performing a single fitness posture based on multiple frames of images including the user's fitness posture. The timestamp is the time when the image sensor of the portable device 100 captures the image including the user's fitness posture.

[0177] Exemplarily, as described above, the current fitness program has posture information, and the posture information may include the standard values ​​of the activity amplitude of the key nodes of one or more postures of the current fitness program. The electronic device 200 or the portable device 100 may determine whether the activity amplitude of the key nodes of any fitness posture of the user is equal to or close to the standard value of the activity amplitude. Exemplarily, the electronic device 200 or the portable device 100 may use a skeleton key node algorithm (also referred to as a human pose estimation algorithm) to identify the skeleton key nodes under the user's fitness posture in an image including the user's fitness posture, and determine the activity amplitude of the skeleton key nodes, and then determine whether the activity amplitude of the skeleton key nodes under the user's fitness posture is equal to or close to the standard value of the activity amplitude. In one example, the activity amplitude of the skeleton key nodes under the user's fitness posture may refer to the maximum activity amplitude of the skeleton key nodes under the fitness posture. Taking squat as an example, it can be understood that the user's posture before squatting is a standing posture, and the angle of the knee joint is close to 180°. When squatting, the smaller the angle of the knee joint is, the greater the range of the squat. Therefore, the minimum angle of the knee joint in the squat (maximum range of motion) can represent the knee joint angle (range of motion) in the squat. Exemplarily, the image corresponding to the maximum range of motion of the key bone nodes in any fitness posture can be called the key frame in the fitness posture.

[0178] Exemplarily, multiple amplitude floating ranges can be set based on the standard value of the activity amplitude. Different amplitude floating ranges correspond to different validity. Among them, the smaller the amplitude floating range, the higher the validity. The activity amplitude of the key skeletal nodes under the user's fitness posture falls into that amplitude floating range, and the validity corresponding to the amplitude floating range can be used as the validity of the user's fitness posture. In an example, taking the standard angle of the knee joint under a squat as an example, the standard angle can be set to 90°, and [85°, 95°], [80°, 100°], [75°, 105°] can be set, wherein [85°, 95°], [80°, 100°], [75°, 105°] correspond to high validity, medium validity, and low validity in turn. If the knee angle of a squat of a user is 92°, it can be determined that the squat is highly valid. If the knee angle of a squat of a user is 83°, it can be determined that the squat is medium valid. If the knee angle of a squat of the user is 89°, it can be determined that the squat is of low validity. If the knee angle of a squat of the user is not included in the above three intervals, it is determined that the squat is invalid.

[0179] The above counting of fitness postures and determining the effectiveness can be collectively referred to as evaluation of fitness postures.

[0180] Exemplarily, when the current fitness program or any section in the current fitness program ends, the electronic device 200 can display or play an image including an invalid fitness posture or a low-effective fitness posture, for example, a key frame of an invalid fitness posture or a low-effective fitness posture can be displayed. An invalid fitness posture or a low-effective fitness posture is a posture performed by a user when performing fitness according to the current fitness program or the section in the current fitness program. In one example, the electronic device 200 can display or play an image including an invalid fitness posture or a low-effective fitness posture in a first area on the display screen, and at the same time, display or play an image including a standard posture in a second area on the display screen.

[0181] Through the above scheme, when it is inconvenient to collect user fitness images through large-screen devices, the large-screen device can be used to assist the user in adjusting the posture of the portable device, so that the user's fitness images can be collected through the portable device, so that the user's fitness does not have to be limited to the place in front of the large-screen device, and the user can choose the fitness place more freely.

[0182] It is understandable that in order to achieve a more accurate evaluation of the user's fitness posture, it is necessary to collect the user's three-dimensional posture during fitness, that is, it is necessary to collect the three-dimensional spatial information of the user's fitness posture. The solution provided in the embodiment of the present application can use the portable device 100 and the electronic device 200 to collect the user's fitness posture from different angles, so as to obtain the user's three-dimensional posture during fitness. Next, the solution provided in the embodiment of the present application is introduced by example.

[0183] For example, see Fig.11 , the user can use the venue under the lens angle of view of the camera 250 of the electronic device 200 as the fitness venue, and can adjust the posture of the portable device 100 so that the lens angle of view of the camera of the portable device 100 can cover the fitness venue. The specific adjustment process can refer to the above introduction and will not be repeated here. In the embodiment of the present application, the posture may refer to the position and posture in the three-dimensional space, or may refer to the position in the three-dimensional space, or may refer to the posture in the three-dimensional space. Among them, the posture of the device in the three-dimensional space may refer to the angle of the device in the three directions of X, Y, and Z of the three-dimensional space coordinate system.

[0184] In some embodiments, the user can trigger the electronic device 200 to start the fitness program and enter the fitness posture collection and analysis state by starting the fitness program with a preset fitness program posture. Fig. 10A The description of the illustrated embodiment will not be repeated here.

[0185] In some embodiments, the user can use voice commands to trigger the electronic device 200 to start a fitness program and enter a fitness posture collection and analysis state.

[0186] In some embodiments, the user can trigger the electronic device 200 to start a fitness program and enter a fitness posture collection and analysis state through remote control instructions.

[0187] In some embodiments, the user can trigger the electronic device 200 to start a fitness program and enter a fitness posture collection and analysis state through touch operation instructions.

[0188] As described above, during the process of adjusting the posture of the portable device 100, the portable device 100 is in the state of collecting images. After the electronic device 200 enters the state of collecting and analyzing the fitness posture, the portable device 100 may be in the state of collecting images.

[0189] During the user's fitness process, the electronic device 200 and the portable device 100 can respectively collect images including the user's fitness posture from different angles. It can be understood that a video is composed of multiple frames of images, so the recorded video can be referred to as the collected image. In other words, in the embodiment of the present application, collecting the image including the user's fitness posture can include recording the video including the user's fitness posture, and the image including the user's fitness posture can also refer to the image in the video including the user's fitness posture.

[0190] For example, the portable device 100 may send the collected image including the user's fitness posture to the electronic device 200. The electronic device 200 may display the image collected by the portable device 100 or the image collected by the electronic device 200 itself. Fig.12 , the electronic device 200 can simultaneously display the image captured by the portable device 100 and the image captured by the electronic device 200 itself. For example, the electronic device 200 can display the image captured by the portable device 100 in area 244 on the display screen, and display the image captured by the electronic device 200 itself in area 245.

[0191] The size of area 244 and the size of area 245 may be the same or different. Exemplarily, area 244 has a preset size so as to display the image captured by portable device 100 in a large size. Taking the display screen of electronic device 200 as an example of 65 inches, the size of area 244 may be 60 cm×70 cm. In one example, the size of area 244 is in a preset ratio to the size of the display screen of electronic device 200, for example, the length of area 244 is half the length of the display screen of electronic device 200, and the width of area 244 is half the width of the display screen of electronic device 200. Exemplarily, area 245 may be implemented with reference to area 244.

[0192] In some embodiments, the image captured by the portable device 100 may include a timestamp, which is the time when the image sensor of the portable device 100 captures the image, so that the electronic device 200 can synchronously display the image captured by the portable device 100 and the image captured by the electronic device 200 according to the timestamp in the image captured by the portable device 100.

[0193] Exemplarily, the electronic device 200 and the portable device 100 can perform timestamp synchronization. For example, the timestamp synchronization of the electronic device 200 and the portable device 100 can be specifically that both the electronic device 200 and the portable device 100 can use universal time coordinated (UTC), which can be referred to as UTC timestamp synchronization. It can be understood that the electronic device 200 and the portable device 100 can access the Internet to achieve UTC timestamp synchronization.

[0194] Exemplarily, the image may be an image in a video, and the timestamp of the image may be included in a supplemental enhancement information (SEI) of the image.

[0195] Exemplarily, the image may be a photo, and the timestamp of the image may be included in the exchangeable image file (EXIF) information of the image.

[0196] In some embodiments, the electronic device 200 can evaluate the user's fitness posture based on the images collected by itself and the images collected by the portable device, for example, to determine the effectiveness of the fitness posture. Fig.13A and Fig. 13B Taking squat as an example, the electronic device 200 can use the skeletal key node algorithm to identify the skeletal joint nodes shown in 13A from the image collected by the portable device 100, including the knee joint. The electronic device 200 can use the skeletal key node algorithm to identify the skeletal joint nodes shown in 13B from the image collected by the electronic device 200, including the knee joint. As described above, the image collected by the portable device 100 includes a timestamp, and the electronic device 200 can also record the capture time of the image it collects. Therefore, the electronic device 200 can determine the skeletal key nodes of the user's same fitness posture at different angles, and then can more accurately determine the effectiveness of the fitness posture based on the skeletal key nodes at different angles of the fitness posture.

[0197] In some embodiments, the portable device 100 can evaluate the user's fitness posture based on the image including the user's fitness posture collected by it, for example, determine the effectiveness of the fitness posture based on the identified key skeletal nodes, and specifically refer to the above, which will not be repeated here. Exemplarily, the portable device 100 can adopt a key skeletal node algorithm to identify the key skeletal nodes under the user's fitness posture in the image including the user's fitness posture. In an example, the portable device 100 can send the key skeletal nodes it identifies to the electronic device 200 and the timestamp corresponding to the key skeletal node to the electronic device 200. The timestamp corresponding to the key skeletal node is the time when the image sensor of the portable device 100 captures the corresponding image of the key skeletal node. The electronic device 200 can identify the key skeletal nodes from the images collected by the electronic device 200.

[0198] Exemplarily, when the current fitness program or any section in the current fitness program ends, the electronic device 200 may display or play an image including an invalid fitness posture or a low-effective fitness posture, for example, a key frame of an invalid fitness posture or a low-effective fitness posture may be displayed. An invalid fitness posture or a low-effective fitness posture is a posture that the user performs when performing fitness according to the current fitness program or the section in the current fitness program.

[0199] In one example, the electronic device 200 may display or play an image including an invalid fitness posture or a low-effective fitness posture in a first area on the display screen, and at the same time, display or play an image including a standard posture in a second area on the display screen. In a specific example, the first area includes an area Q1 and an area Q2, and the electronic device 200 may display an image of an invalid fitness posture or a low-effective fitness posture collected by the portable device 100 in the area Q1, and display an image of an invalid fitness posture or a low-effective fitness posture collected by the electronic device 200 in the area Q2.

[0200] In one example, the portable device 100 can determine an invalid fitness posture or a low-effective fitness posture based on the image it has collected, and send a key frame corresponding to the invalid fitness posture or the low-effective fitness posture determined by it to the electronic device 200, and the key frame carries a timestamp. The electronic device 200 can determine the image B1 captured by the electronic device 200 at the capture time of the key frame based on the timestamp carried in the key frame. The electronic device 200 can simultaneously display or alternately display the key frame and the image B1 in the first area.

[0201] The method, device and apparatus for determining a fitness venue provided in the embodiments of the present application can, when the venue within the viewing angle of the device running the fitness application does not meet the venue conditions of the fitness project, use the venue within the viewing angle of other devices as the fitness venue for the fitness project, so that the user can exercise, thereby limiting the user to the viewing angle of a specific device and choosing a fitness venue more freely. In addition, the device running the fitness application can be a large-screen device, and the other device can be a portable device. In the embodiments of the present application, the large-screen device can be used to adjust the posture of the portable device so that the posture of the user when performing the fitness action falls into the lens viewing angle of the portable device, so that the posture of the user when performing the fitness action can be photographed by the portable device, so that when the user exercises in different venues, the posture of the portable device can be quickly adjusted to capture the posture of the user when performing the fitness action.

[0202] As people's living standards improve, they pay more and more attention to their physical health. Fitness exercises are increasingly recognized as an effective means to enhance physical fitness, and fitness-related products are also welcomed and loved by more and more consumers. The group of fitness enthusiasts is growing year by year, and fitness venues are no longer limited to specific places such as gyms. Fitness time is also fragmented, and fitness in the home environment is a market that needs to be opened up urgently. Based on large-screen devices such as living room TVs, which have a very high market share, intelligent fitness guidance using image processing has become a very potential solution: that is, through image recognition of users' fitness movements, and based on the key indicators of fitness movements, the quality of users' completion is evaluated, and wrong movements and improvement guidance are pointed out, so that users can exercise scientifically.

[0203] One solution is to use a smart TV with a camera in the living room to collect image data of users' fitness movements. Relying on the powerful processor computing power, image processing-related algorithms can be used to identify users' fitness movements in real time, and evaluate and guide them based on key movement features, thus meeting the professional fitness needs of ordinary family living rooms.

[0204] This solution has the following problems.

[0205] 1) The viewing angle of the TV camera is limited. For some small living rooms, it may not be possible to capture complete user image information.

[0206] 2) There is often a sofa and coffee table in front of the TV in the living room, and it is difficult for a general TV camera to flexibly adjust the FOV.

[0207] 3) Movements that require the use of special terrain (such as wall stretching and corridor pull-ups) require leaving the TV area and are not supported by ordinary TVs.

[0208] 4) Some fitness movements (such as squats, lunges and turns) require three-dimensional spatial information for complete indicator evaluation, which ordinary TVs cannot support.

[0209] The embodiment of the present application provides a technical solution for using a portable device such as a smart phone to assist a large-screen device in intelligent fitness guidance, which can achieve complete scene coverage and perfect indicator evaluation. In the solution provided in the embodiment of the present application, a mobile phone can be used to assist a large-screen device to provide intelligent fitness guidance for the entire scene, solving the problem that the large-screen device cannot fully capture user information in certain scenarios (small living room, furniture obstruction, out of TV FOV, etc.), and a mobile phone is used to assist a large-screen device to achieve complete capture of user information and provide an effective evaluation. In the solution provided in the embodiment of the present application, a mobile phone can be used to assist a large-screen device to achieve a multi-perspective comprehensive evaluation of fitness movements. Some fitness movements (such as squats, lunges and turns, etc.) require multi-perspective evaluation, and large-screen devices cannot obtain the user's multi-perspective information. The present invention can collect the user's multi-perspective information with the assistance of a mobile phone and comprehensively evaluate the user's fitness movements. The execution process of the solution provided in the embodiment of the present application can be as follows: Fig.14 shown.

[0210] In some embodiments, in scenarios where a large-screen device cannot obtain complete image information of the user (small living room, furniture obstruction, camera angle away from the TV, etc.), the user's fitness image information can be collected and processed using a mobile phone and transmitted back to the large-screen device for fitness guidance.

[0211] For an illustrative example, see Figure 5A , wherein the portable device 100 may be a mobile phone, and the electronic device 200 may be a television. Figure 5A As shown, in the scenario of no fitness equipment, a small living room and large furniture occupies a large space, resulting in a small space area for users to exercise, and the TV camera cannot fully capture the user's image.

[0212] For an illustrative example, see Figure 5B , wherein the portable device 100 may be a mobile phone, and the electronic device 200 may be a television. Figure 5B As shown in the figure, some fitness movements require the assistance of a wall, such as wall stretching and wall handstand push-ups. In this scenario, the user may be out of the camera angle of the TV, resulting in the TV being unable to fully capture the user's image;

[0213] For an illustrative example, see Figure 5C , wherein the portable device 100 may be a mobile phone, and the electronic device 200 may be a large-screen device (such as a television). Figure 5C As shown, some fitness movements require the use of auxiliary special equipment for training, such as using an indoor horizontal bar to do pull-ups and other exercises, which often need to be completed in a specific location in the room, resulting in the large-screen device camera being unable to fully capture the user's image.

[0214] In the above scenarios, there is a problem that the user cannot be fully captured by the camera of the large-screen device alone.

[0215] In some embodiments, in the solution provided by the embodiments of the present application, a mobile phone is used to assist a large-screen device in collecting user information, processing data in real time, transmitting user fitness information back, and guiding the user to complete training. The specific solution includes the following steps.

[0216] Step S1: Before the exercise begins, flexibly set the mobile phone viewing angle to collect the user's fitness data to ensure the integrity of the user in the image. Figure 5A , Figure 5B , Figure 5C In the picture, the mobile phone is placed.

[0217] Step S2: The mobile phone and the large-screen device are connected to each other for information transmission. The connection methods include but are not limited to wireless connection methods such as WLAN, Bluetooth, 5G, and wired methods such as USB.

[0218] Step S3: The user confirms to start fitness, and the coach's standard movements are played on the large-screen device to guide the user to complete.

[0219] Step S4: The mobile phone collects the user's fitness images, and transmits them back to the large-screen device together with the timestamp after processing.

[0220] Step S5: The large-screen device runs the skeleton key node algorithm, extracts the user's skeleton key nodes in real time, completes the counting and index evaluation of user actions, and gives the user real-time feedback.

[0221] Step S6: After each exercise segment, the large-screen device extracts the key frames of the incorrect movements, displays them to the user, and provides specific analysis guidance.

[0222] In the solution provided in the embodiment of the present application, when a large-screen device (such as a television) cannot fully capture the user's image information, a mobile phone is used to assist in achieving complete capture of the user's image. Therefore, the method of using a mobile phone to assist a large screen can achieve complete capture of the user's fitness data, conduct a comprehensive analysis of the user's fitness information, and guide the user to complete high-quality fitness activities. Compared with solutions that only support fitness guidance in more ideal scenarios, the solution provided in the embodiment of the application can achieve complete capture of user fitness information in more general scenarios, guide users to complete training, and improve the product's scope of application and user experience.

[0223] Understandably, some fitness movements require multi-angle images for comprehensive evaluation, such as squats, lunges and turns.

[0224] In some embodiments, the captured images of the squat action shown in Figures 15A, 15B and 15C can be referred to, wherein Figure 15A is an image taken from the perspective of a mobile phone, and Figure 15C is an image taken from the perspective of a TV. For fitness actions that require multi-angle image information for judgment, a mobile phone is used to collect multi-angle user fitness image information, which is transmitted to a large-screen device (such as a TV) after real-time processing by a skeleton node algorithm. The large-screen device combines the locally collected user fitness image information and the processing results for comprehensive evaluation, and provides users with accurate and comprehensive fitness guidance. Specifically, the solution provided in the embodiment of the present application may include the following steps.

[0225] Step 151: Set the mobile phone viewing angle to collect user fitness image data from the side, while ensuring the integrity of the user in both the large-screen device and the mobile phone. FIG. 5A to FIG. 5C The placement of the mobile phone in the middle.

[0226] Step 1502: The mobile phone and the large-screen device are connected to each other for information transmission. The connection method includes but is not limited to wireless connection methods such as WLAN, Bluetooth, 5G, and wired methods such as USB.

[0227] Step 1503: The large-screen device guides the user to perform a specified initial action, and the large-screen device and the mobile phone simultaneously collect and recognize the user's action.

[0228] Step 1504: The mobile phone returns the action features and timestamp to the TV, and the large-screen device matches the initial action features to ensure the synchronization of the information returned by the mobile phone and the data of the large-screen device.

[0229] Step 1505: The user confirms to start exercising, and the coach's standard movements are played on the large-screen device to guide the user to complete the exercise.

[0230] Step 1506: The mobile phone collects the user's fitness images, runs the skeleton key node algorithm locally, extracts the user's skeleton key nodes in real time, and returns them to the large-screen device.

[0231] Step 1507: The large-screen device integrates the local recognition skeleton nodes and the skeleton nodes recognized by the mobile phone to complete the completion count and indicator evaluation of the user action.

[0232] Step 1508: After each exercise segment is completed, the mobile phone extracts the key frames of the erroneous action and transmits them to the large-screen device. The large-screen device combines the erroneous key frames extracted locally, displays the user's erroneous action key frames and gives specific analysis suggestions.

[0233] In the solution provided in the embodiment of the present application, for specific fitness movements, such as squats, lunges and turns, a mobile phone is used to assist in taking multi-angle pictures to fully evaluate the user's fitness movements. In the solution provided in the embodiment of the present application, specific fitness movements need to be evaluated from multiple angles, such as squats. Compared with the solution that can only evaluate the user's fitness movements from one angle on a large screen, the solution provided in the embodiment of the present application uses a mobile phone to assist in a comprehensive and integrated evaluation of fitness movements from multiple angles, which improves the accuracy and effectiveness of the evaluation.

[0234] The solution provided in the embodiment of the present application utilizes a mobile phone to assist a large-screen device, thereby solving the problem that a complete user image cannot be captured using only a large-screen device, thereby achieving complete capture of the user image; and adopts a mobile phone multi-view image information processing method to perform a comprehensive and effective evaluation of specific fitness movements, make a more professional evaluation, and improve the user experience.

[0235] See also Fig.16 The embodiment of the present application provides a fitness venue determination device 1600, which can be configured in an electronic device 200. The device 1600 may include:

[0236] The starting unit 1610 is used to start a first application and select a first fitness item of the first application;

[0237] The connection unit 1620 is used to connect the portable device 100 to collect the first image when the venue within the field of view of the electronic device 200 does not meet the venue condition of the first fitness project;

[0238] A determination unit 1630, configured to determine, based on the first image, whether a site within the viewing angle of the portable device 100 satisfies a site condition;

[0239] The presenting unit 1640 is configured to present an application interface of the first fitness item.

[0240] The above mainly introduces the device provided by the embodiment of the present application from the perspective of the method flow. It is understandable that, in order to realize the above functions, each electronic device includes a hardware structure and / or software module corresponding to each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0241] The present application embodiment can be based on Figure 1-Figure 12The functions of the electronic device 200 in each method embodiment shown divide the electronic device into functional modules. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0242] See also Fig.17 The present application embodiment provides an electronic device 1700, which can execute the above Figure 1-Figure 12 The electronic device 1700 may include a processor 1710, a storage 1720, a transceiver 1730, and a display screen 1740. The storage 1720 stores instructions that can be executed by the processor 1710. When the instructions are executed by the processor 1710, the electronic device 1700 may perform the above Figure 1-Figure 12 The operations performed by the electronic device 200 in each method embodiment are shown. Specifically, the processor 1710 can perform data processing operations, the transceiver 1730 can perform data sending and / or receiving operations, and the display screen 1740 can display data.

[0243] It is understandable that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0244] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0245] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0246] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

Claims

1. A method for determining a fitness venue, characterized in that: Applied to a first device equipped with a camera; the method comprises: Launch a first application and select a first fitness item of the first application; When the venue within the viewing angle of the first device does not meet the venue condition of the first fitness item, a second device equipped with a camera is connected, and the second device captures the first image; Determining, based on the first image, that a venue within the perspective of the second device satisfies the venue condition; wherein the venue within the perspective of the second device serves as a fitness venue for the first fitness item; The application interface of the first fitness project is presented; wherein the first device uses the second device to collect the user's posture when performing fitness; wherein, The screen of the first device is larger than the screen of the second device; The site condition includes a first size, and the size of the first site is greater than or equal to the first size; Before the second device acquires the first image, the method further includes: The first device receives a third image, where the third image is an image captured by the second device when the first user is located at the first venue; The first device determines, according to the position of the image of the first user in the third image, that a size of an area in the first venue that falls within the viewing angle of the second device is smaller than the first size; The first device provides second prompt information, and the second prompt information is used to prompt the posture adjustment direction of the second device.

2. The method according to claim 1, characterized in that Before the second device acquires the first image, the method further includes: The first device receives a second image from the second device, where the second image is an image captured by the second device when the first user is located at the first venue; The first device determines, according to the position of the image of the first user in the second image, that the viewing angle of the second device does not completely cover the posture of the first user when the user is in the first venue; The first device provides first prompt information, and the first prompt information is used to prompt the second device for posture adjustment direction.

3. The method according to claim 1, characterized in that The method further comprises: When the venue within the field of view of the first device meets the venue condition, the first device provides third prompt information, and the third prompt information is used to prompt the first user to perform the first fitness item in a first area, and the first area is an area in the venue within the field of view of the first device.

4. The method according to claim 1, characterized in that The method further comprises: When the first fitness item corresponds to multiple shooting angles, the first device is connected to a third device equipped with a camera, and the direction facing the camera of the third device is different from the direction facing the camera of the first device; The first device receives a fourth image from the third device, where the fourth image is an image captured by the third device when the second user performs the first fitness program; The first device determines that the acquisition time of the fifth image is the same as the acquisition time of the fourth image, and the fifth image is an image acquired by the first device when the second user performs the first fitness program; The first device displays the fourth image and the fifth image side by side.

5. The method according to claim 1, characterized in that The method further comprises: When the first fitness item corresponds to fitness equipment, and the objects within the field of view of the first device do not include the fitness equipment, the first device determines a first object used to replace the fitness equipment from the objects within the field of view of the first device; The first device provides fourth prompt information, where the fourth prompt information is used to prompt the user to use the first object to perform the first fitness item.

6. The method according to claim 1, characterized in that The method further comprises: When the first fitness item corresponds to fitness equipment, and the objects within the field of view of the first device do not include the fitness equipment, the first device determines a second fitness item corresponding to a second object, where the second object is one or more objects within the field of view of the first device; The first device provides fifth prompt information, where the fifth prompt information is used to prompt the user to use the second object to perform the second fitness item.

7. The method according to claim 1, characterized in that The second device connected with a camera includes: The first device provides sixth prompt information, where the sixth prompt information is used to prompt a user to enable a wireless connection function of the first device; The first device turns on the wireless connection function in response to a user-initiated operation of turning on the wireless connection function, so as to connect to the second device through the wireless connection function.

8. The method according to claim 7, characterized in that The sixth prompt information corresponds to the first function area displayed by the first device, and the operation of turning on the wireless connection function is an operation acting on the first function area.

9. The method according to claim 1, characterized in that: Before the second device acquires the first image, the method further includes: The first device sends an application opening request to the second device to trigger the second device to start the second application or display prompt information for prompting the user to open the second application. The second application corresponds to the first application, and the first image is an image captured by the second application by calling the camera of the second device.

10. A first device, characterized in that: include: Processor, memory, transceiver; The memory is used to store computer instructions; When the first device is running, the processor executes the computer instructions so that the first device performs: Launch a first application and select a first fitness item of the first application; When the venue within the viewing angle of the first device does not meet the venue condition of the first fitness item, a second device equipped with a camera is connected, and the second device captures the first image; Determining, based on the first image, that a venue within the perspective of the second device satisfies the venue condition; wherein the venue within the perspective of the second device serves as a fitness venue for the first fitness item; The application interface of the first fitness project is presented; wherein the first device uses the second device to collect the user's posture when performing fitness; wherein, The screen of the first device is larger than the screen of the second device; The site condition includes a first size, and the size of the first site is greater than or equal to the first size; Before the second device acquires the first image, the processor executes the computer instruction so that the first device further executes: receiving a third image, where the third image is an image captured by the second device when the first user is located at the first venue; Determining, according to the position of the image of the first user in the third image, that a size of an area in the first venue that falls within the viewing angle of the second device is smaller than the first size; Provide second prompt information, where the second prompt information is used to prompt the posture adjustment direction of the second device.

11. The first device according to claim 10, characterized in that Before the second device captures the first image, the processor executes the computer instructions so that the first device further executes: receiving a second image from the second device, where the second image is an image captured by the second device when the first user is located at the first venue; determining, according to a position of the image of the first user in the second image, that the viewing angle of the second device does not completely cover the posture of the first user when the user is in the first venue; Provide first prompt information, where the first prompt information is used to prompt the posture adjustment direction of the second device.

12. The first device according to claim 10, characterized in that The processor executes the computer instructions so that the first device further executes: When the venue within the field of view of the first device meets the venue condition, third prompt information is provided, and the third prompt information is used to prompt the first user to perform the first fitness item in a first area, and the first area is an area in the venue within the field of view of the first device.

13. The first device according to claim 10, characterized in that The processor executes the computer instructions so that the first device further executes: When the first fitness item corresponds to multiple shooting angles, a third device equipped with a camera is connected, and the direction facing the camera of the third device is different from the direction facing the camera of the first device; receiving a fourth image from the third device, where the fourth image is an image captured by the third device when the second user performs the first fitness program; Determine that the acquisition time of the fifth image is the same as the acquisition time of the fourth image, and the fifth image is an image acquired by the first device when the second user performs the first fitness program; The fourth image and the fifth image are displayed in parallel.

14. The first device according to claim 10, characterized in that The processor executes the computer instructions so that the first device further executes: When the first fitness item corresponds to fitness equipment, and the objects within the viewing angle of the first device do not include the fitness equipment, determining a first object for replacing the fitness equipment from the objects within the viewing angle of the first device; Fourth prompt information is provided, where the fourth prompt information is used to prompt the user to use the first object to perform the first fitness item.

15. The first device according to claim 10, characterized in that The processor executes the computer instructions so that the first device further executes: When the first fitness item corresponds to fitness equipment, and the objects within the field of view of the first device do not include the fitness equipment, determining a second fitness item corresponding to a second object, where the second object is one or more objects within the field of view of the first device; Fifth prompt information is provided, where the fifth prompt information is used to prompt the user to use the second object to perform the second fitness item.

16. The first device according to claim 10, characterized in that The processor executes the computer instructions so that the first device further executes: Providing sixth prompt information, where the sixth prompt information is used to prompt the user to enable the wireless connection function of the first device; In response to a user-initiated operation of starting the wireless connection function, the wireless connection function is started so as to connect to the second device through the wireless connection function.

17. The first device according to claim 16, characterized in that The sixth prompt information corresponds to the first function area displayed by the first device, and the operation of turning on the wireless connection function is an operation acting on the first function area.

18. The first device according to claim 10, characterized in that Before the second device acquires the first image, the processor executes the computer instruction so that the first device further executes: An application opening request is sent to the second device to trigger the second device to start the second application or display a prompt message for prompting the user to open the second application, the second application corresponds to the first application, and the first image is an image captured by the second application by calling the camera of the second device.

19. A computer storage medium, characterized in that: The computer storage medium includes computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 9.

Images