Motion information interaction method and device in screen projection scene

Abstract

The invention provides a motion information interaction method and device in a screen projection scene, and relates to the field of fitness sports, and the method comprises the steps: obtaining an action video of a first user; the action video of the first user is processed to obtain action information, the action information is display information of a virtual image of the first user, and action features of the virtual image of the user correspond to action features of the user; determining the biological data information of the first user, processing the biological data information of the first user and / or the second user according to a processing rule, generating a video picture in combination with the display information of the virtual image of the first user and the fitness guidance video, and projecting the video picture to a display device for display.

Description

Technical Field

[0001] The present invention relates to the field of sports and fitness, and in particular to a method and device for interacting with sports information in a screen projection scenario. Background Art

[0002] In the existing screen projection scenario of watching sports and fitness through mobile phone applications, the sports video displayed on the user's mobile phone is usually projected, or the recorded sports video from the server is projected as a video stream.

[0003] With the development of intelligent posture recognition technology and the improvement of the scientific application concept of user biological data during exercise, how to better present the local user's intelligent recognition results and physiological data of exercise to the user in the screen projection scenario has become a new problem that needs to be solved. Summary of the invention

[0004] The present application provides a method and device for interacting with motion information in a screen projection scenario, which increases the interactivity between users and facilitates users to adjust their own motion rhythm to achieve the motion effect desired by the users.

[0005] According to one aspect of the present application, a method for interacting with motion information in a screen projection scenario is provided, the method comprising:

[0006] Obtaining the action video of the first user;

[0007] Processing the action video of the first user to obtain action information, where the action information is display information of the virtual image of the first user, and the action characteristics of the virtual image of the user correspond to the action characteristics of the user;

[0008] Determining biometric information of the first user, the biometric information including heart rate information of the user;

[0009] receiving biometric data information of a second user, where the second user is one or more users other than the first user;

[0010] Processing the biometric information of the first user and / or the second user according to the processing rules, and combining the display information of the virtual image of the first user and the fitness guidance video to generate a video screen, wherein the video screen includes the screen of the fitness guidance video, the biometric information of the first user and / or the second user displayed according to the processing rules, and the virtual image of the first user;

[0011] Project the video screen onto a display device for display.

[0012] Through screen projection, users can clearly see the exercise video and their own virtual image movements. In addition, through the interactive processing of other users' physiological data information and screen projection processing, the interactivity between users is further increased; through the display and interval division of heart rate data and integration into the fitness video screen for screen projection, users can more directly understand their exercise status, determine the current fat loss level of exercise, etc., which is convenient for users to adjust their exercise rhythm and achieve the desired exercise effect.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, processing the biometric data information of the first user and / or the second user according to the processing rule includes:

[0014] The heart rate information is processed according to the duration of different heart rate intervals, and the processed information is marked with different colors or symbols.

[0015] In combination with the first aspect, in certain implementations of the first aspect, the heart rate data information of the first user and / or the second user is counted and compared with a preset target heart rate value, and the compared result is displayed.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, processing the biometric data information of the first user and / or the second user according to the processing rule includes:

[0017] Grouping the first and second users according to user characteristics of the first and second users, wherein the user characteristics include at least one of geographic location, age, gender, athletic ability level, team selection requirements, and athletic score;

[0018] The exercise performance of each group is evaluated based on at least the heart rate data of the users in each group, and the evaluation results between different groups are compared.

[0019] In combination with the first aspect, in certain implementations of the first aspect, the video screen also includes feedback prompt content on the first user's action, and the feedback prompt content is generated by the terminal after performing action recognition based on the action video of the first user; or the feedback prompt content is feedback from the remote coach end after the display information of the virtual image is transmitted to the remote coach end.

[0020] In combination with the first aspect, in some implementations of the first aspect, the latest heart rate data of the first user and / or the second user is dynamically displayed in the video screen, the heart rate data of the first user and / or the second user is continuously received, and the distribution of durations of different heart rate levels is dynamically counted and displayed in the video screen;

[0021] When the heart rate of the first user does not reach the set target heart rate, prompt content is displayed in the video screen, and the prompt content is used to instruct the first user to adjust the action in order to adjust the heart rate.

[0022] In combination with the first aspect, in certain implementations of the first aspect, calorie consumption is calculated based on the user's heart rate data, and the calorie value consumed by the user is displayed in the video screen.

[0023] In combination with the first aspect, in some implementations of the first aspect, all users are sorted according to the heart rate data of the first user and the second user, and the sorting rules include sorting by heart rate value, or sorting by duration of a specified heart rate interval.

[0024] In combination with the first aspect, in some implementations of the first aspect, a virtual image of at least one second user is received, and the virtual image of the at least one second user is displayed in the video screen.

[0025] In combination with the first aspect, in certain implementations of the first aspect, the heart rate data is an estimated heart rate set according to the exercise intensity of the current course.

[0026] In a second aspect, a terminal device is provided, which can specifically execute each step in the above-mentioned first aspect and any possible implementation method thereof.

[0027] In a third aspect, a computer program product is provided, which includes: a computer program (also referred to as code, or instruction), which, when executed, enables a computer to execute the method in the first aspect and any possible implementation thereof.

[0028] In a fourth aspect, a computer-readable medium is provided, which stores a computer program (also referred to as code or instruction) which, when executed on a computer, enables the computer to execute the method in the first aspect and any possible implementation thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of a flow chart of a motion information interaction method in a screen projection scenario provided in an embodiment of the present application;

[0030] Figure 2 A schematic diagram of a terminal device provided in an embodiment of the present application;

[0031] Figure 3 A schematic diagram of another terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0033] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those within the scope of this description. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0034] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0035] The terminal device referred to in this application is not limited to any intelligent electronic product that can interact with the user (for example, through a touchpad), such as a smart phone, a tablet computer, etc. The mobile electronic product can use any operating system, such as an Android operating system, an iOS operating system, etc., or a dedicated fitness smart box, which is connected to an external display device by wired or wireless means; or a smart fitness mirror, a hardware terminal device with a camera and a display screen. The smart fitness mirror may include a processor that partially controls the operation of various subcomponents in the smart fitness mirror and manages the data flow (for example, video content, audio from the user, biometric feedback analysis) flowing into / out of the smart fitness mirror. The smart fitness mirror may include a display for displaying video content, a graphical user interface (GUI) with which the user can interact and control the smart fitness mirror, biometric feedback data and / or other visual content. The sensor may be coupled to the processor to collect user-related data. The antenna may be coupled to the processor to provide data transmission between the smart fitness mirror and another device (for example, a remote control device, a biometric sensor, a wireless router). The antenna may include multiple transmitters and receivers, each transmitter and receiver being customized for a specific frequency and / or wireless standard (e.g., Bluetooth, 802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 2G, 3G, 4G, 4GLTE, 5G). The amplifier may be coupled to the processor to receive an audio signal from the processor so as to output subsequent sound through the left speaker and / or the right speaker. The network device includes an electronic device capable of automatically performing numerical calculations and information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, a dedicated integrated circuit, a programmable logic device, a field programmable gate array, a digital signal processor, an embedded device, etc. The network device includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets, or a cloud consisting of a plurality of servers; here, the cloud is composed of a large number of computers or network servers based on cloud computing, wherein cloud computing is a type of distributed computing, a virtual supercomputer consisting of a group of loosely coupled computer sets. The network includes but is not limited to the Internet, wide area network, metropolitan area network, local area network, VPN network, wireless ad hoc network, etc. Preferably, the device can also be a program running on the user device, network device, or a device formed by integrating the user device and network device, network device, touch terminal, or network device and touch terminal through a network. Other display devices in this application include but are not limited to televisions, projectors, independent displays, computer display screens, other fitness devices with displays, smart virtual reality glasses, and other smart electronic products.

[0036] According to one aspect of the present application, reference Figure 1 , provides a motion information interaction method in a screen projection scenario, the method comprising:

[0037] S101, obtaining an action video of a first user;

[0038] In an optional implementation, a terminal device equipped with a camera and an application installed on the terminal device is used to start a mobile phone application, and the camera is used to capture human body movements. The user's movement video is obtained through the camera to capture human body movements.

[0039] In another optional implementation, the terminal device and the camera used are not integrated, and the smart device may be connected to other cameras.

[0040] S102, processing the action video of the first user to obtain action information, where the action information is display information of the virtual image of the first user, and the action characteristics of the virtual image of the user correspond to the action characteristics of the user;

[0041] Optionally, the collected image and video data may be preprocessed, such as scaling, rotating, cropping, etc., to facilitate subsequent processing.

[0042] Computer vision algorithms, such as keypoint detection and optical flow tracking, are used to extract features related to human movements from images.

[0043] In one embodiment, human skeleton points are detected for each image frame to obtain the coordinates of the skeleton points of the user in each image frame. The detected skeleton points may include multiple skeleton points of the following: top of the head, nose tip, left ear, right ear, chin, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left metacarpophalangeal joint, right metacarpophalangeal joint, left fingertips, right fingertips, left hip, right hip, left knee, right knee, left ankle, right ankle, left toe tip, right toe tip, left heel, right heel, etc.; the straight-line distance between the top skeleton point and the pelvic skeleton point is obtained, and the unit length of the coordinate system is obtained according to the straight-line distance; the straight-line distance between the top skeleton point and the pelvic skeleton point is obtained, and the straight-line distance is converted into One sixth of the distance is taken as the unit length of the coordinate system, that is, the size of the unit length 1 of the coordinate system is one sixth of the straight line distance; the straight line between the top bone point and the pelvic bone point is taken as the ordinate, the pelvic bone point is taken as the origin, and a rectangular coordinate system is constructed with the obtained unit length; the bone point coordinates in the second content are obtained by scaling the user's bone point coordinates in proportion, and a virtual image is obtained according to the bone point coordinates. The virtual image can be a stick figure with bone points, or a human body model with muscle shape. The virtual image is obtained by identifying and processing each frame of fitness action, and the order of the dynamic virtual image generated by the terminal device is determined according to the generation time of the bone point. Through this recognition technology, the user's movement posture can be scored or corrected; the user's actions can also be counted. In addition, optionally, the terminal device sends the data of the virtual image to a remote server or a terminal device of another user. The action information of the virtual image of the user's bone point obtained according to the video image can be called display information, and the dynamic display of the user's virtual image can be realized through the display information. In addition, the action video obtained may not be all videos after the fitness action is completed, and action acquisition and image processing can be performed in real time.

[0044] In another implementation, the trained motion recognition model can be deployed in a mobile phone application. The application reads the image data captured by the mobile phone camera in real time, performs motion recognition through the model, and displays the results in real time.

[0045] S103, determining biological data information of the first user, where the biological data information includes heart rate information of the user;

[0046] In one embodiment, determining the biological data information of the first user is to receive the heart rate information from a device with a heart rate measurement function worn by the first user in a wireless or wired manner, or other types of biological data information of the user monitored, such as blood oxygen value, movement speed, etc. The wearable device can be a heart rate sticker, a heart rate belt, a bracelet or watch with a heart rate detection function, etc.

[0047] In another implementation, the heart rate data of the first user is determined according to the estimated heart rate set for the exercise intensity of the current course. In one example, the exercise intensity of different parts of the current course can be evaluated to determine the estimated heart rate of the exercise in the exercise of the part; the estimated heart rate of the user in the current exercise part can also be determined by further integrating one or more parameters such as the user's age, exercise ability, height and weight; or, the estimated heart rate of the first user is determined according to the heart rate level of the second user, for example, the average heart rate of multiple second users in the current exercise part is used as the estimated heart rate of the first user.

[0048] In addition, multiple implementations may also be used in combination. For example, during exercise, if the biological data sent by the user's wearable device cannot be received, or the received wearable data of the user's device is obviously erroneous, the heart rate can be estimated to determine the heart rate information of the user.

[0049] By estimating heart rate, users can use the full fitness functions normally and complete all fitness courses without wearable devices or without wanting to display real heart rate data and other biological information, without affecting the sports experience and team performance of other users in group training. In addition, due to the instability of wearable devices or their falling off during exercise, by switching to estimated heart rate, the integrity of user training can be guaranteed and the sports equipment can be improved.

[0050] In addition, the terminal device can send the biological data such as the heart rate information of the first user to the remote server, and then send it to the terminal devices of other users through the remote server, or directly send it to other terminal devices.

[0051] S104, receiving biometric data information of a second user, where the second user is one or more users other than the first user;

[0052] In one implementation, the terminal device receives the biometric data information from the terminal device of the second user; the remote server may also first receive the biometric data information from the terminal device of the second user and then send it to the terminal device of the first user;

[0053] S105, processing the biological data information of the first user and / or the second user according to the processing rules, and combining the display information of the virtual image of the first user and the fitness guidance video to generate a video screen; the video screen includes the screen of the fitness guidance video, the biological data information of the first user and / or the second user displayed according to the processing rules, and the virtual image of the first user;

[0054] Wherein, the biometric information of the first user and / or the second user is processed according to the processing rules, including processing the biometric information of the first user. Optionally, when the biometric information of the second user is obtained, the biometric information of the second user can be processed; or the obtained biometric information of the second user itself is already processed information. Wherein, in one implementation, the heart rate percentage of the user is obtained according to the heart rate information of the user, and the heart rate percentage = real-time heart rate / maximum heart rate. The numerical result of the calculated heart rate percentage can also be rounded off as the heart rate percentage; wherein the maximum heart rate can be 220 minus the age of the user. If there is no age information, it is processed as 30 years old by default, or if the age information is not in the range of 3-80 years old, it is also processed as 30 years old. In addition, different heart rate percentages can be marked with different colors or background colors;

[0055] In one implementation, the heart rate percentage can be displayed in different colors or background colors in different intervals, and a text description of the interval can be further displayed, such as when the heart rate percentage is above 90%, the text description is the limit heart rate; when the heart rate percentage is between 80% and 90%, the text description is anaerobic endurance; when the heart rate percentage is between 70% and 80%, the text description is optimal fat burning; when the heart rate percentage is between 60% and 70%, the text description is basic aerobic; when the heart rate percentage is below 60%, the text description is activation regeneration. In addition, when the user's heart rate percentage changes in different intervals, the display color or background color of the heart rate percentage also changes accordingly.

[0056] In another embodiment, the heart rate interval is displayed, and the position of the user's current heart rate level in the heart rate interval is calculated and identified. The heart rate interval can be represented by a line, and different colors on the line indicate different heart rate levels. The user's current heart rate level is indicated by a symbol on the line.

[0057] In another embodiment, the heart rate information is processed according to the duration of different heart rate intervals, and the processed information is marked with different colors or symbols. For example, in the heart rate percentage intervals in the previous embodiment, the heart rate is marked according to the proportion of time in a certain interval.

[0058] In another embodiment, based on the acquired user's action video, the user's actions within a certain period of time are counted, or the standard degree of the user's actions are scored; the counting result or evaluation result is displayed in the video screen; optionally, the counting result or evaluation result of other users can be further received and displayed in the video screen. Among them, to judge whether the user's fitness action is standard, the standard action refers to the fitness action demonstrated / performed by the demonstration object in the fitness video, and the standard action is used to provide a reference for the user so that the user can imitate the standard action to exercise. The fitness action and the standard action can be compared through the coordinates of the skeleton point, which is not specifically limited here. If the user's fitness action is non-standard, dangerous or standard, the user can be prompted for non-standard, dangerous or standard, and the prompt method includes displaying text or icons in the video screen.

[0059] The above-mentioned various embodiments may also be used in any combination. In addition, the terminal device may receive heart rate data information of other users, and the heart rate data of other users may be the original value of the user's biological data, such as heart rate value, or may be processed data information, such as heart rate percentage. All users may be further sorted according to heart rate percentage, or sorted according to the duration of a specified heart rate interval. And the sorting result is displayed in the video screen.

[0060] The heart rate data information of the first user and / or the second user is counted and compared with a preset target heart rate value, and the comparison result is displayed. A target heart rate value can be preset for a certain course or action, and the heart rate data information of users who simultaneously participate in the course or complete the action is obtained, and the performance of the users is ranked or evaluated at least according to the situation of reaching the target heart rate value.

[0061] In addition, for the achievement of the target heart rate value, users can also be grouped and compared in groups. For example, a queue is generated according to the course entry time, and it is evenly divided into two or more groups. After the grouping starts, only online users are counted. Users who join later will be arranged to join a group with fewer people according to the existing grouping with the principle of keeping the number of people in each group as equal as possible. Among them, the comparison in groups can be the total number of completed actions of the members in the group, or the total realization evaluation score of the completed course, or the total duration of a specific heart rate percentage in the aforementioned embodiment, or the overall evaluation of the group of other evaluation results of the heart rate.

[0062] In another embodiment, all users who take the same course at the same time can be preset to complete the task requirements of the course together. The task requirement can be the total number of times all users complete one or more specific actions within a certain period of time, or the total duration of a specific heart rate percentage in the aforementioned embodiment, or the overall evaluation of all users of other evaluation results of heart rate. The terminal device or remote server processes the motion data of all users and compares it with the task requirements to determine whether the motion performance of all users meets the preset task requirements. The progress of the completion of the motion requirements or the overall motion data of all users can be dynamically displayed in the video screen to prompt all users of the gap from the task goal.

[0063] The user's virtual image displayed in the video screen may be a matchstick virtual image generated according to the first user's action information in step S101 and step S102. The virtual image may be a matchstick figure composed of bone point connections or a human model with muscle lines. It may also be other types of virtual images, such as cartoon task images or animal images. In addition, at least one second user's virtual image may be further displayed in the video screen. The terminal device may receive the virtual image information from the second user, or the server may process the virtual information of the second user and send the virtual images of other second users to the terminal device for screen projection display. In addition, a random algorithm may be used to determine the virtual image of the second user displayed on the display device of other users, or one or more second users with the best sports performance may be determined, and the virtual image of the best performing user may be obtained and displayed.

[0064] In the present application, on the one hand, the terminal device receives a sports video or image, processes the biometric information of the first user and / or the second user according to the processing rules, and combines the display information of the virtual image of the first user and the fitness guidance video to generate a video screen. The video screen can be generated by a terminal device; optionally, it can also be processed and generated by a remote server. Other display screens other than the sports video may not be in the same layer as the sports video, and a complete image or video screen is formed by layer superposition; or different display areas are set, and other screen information is displayed in the preset area to form a complete display image / video. The above steps for the generation method of the video are only examples, including but not limited to the use of any other AI technology, video editing technology, etc. for processing.

[0065] Optionally, the video screen also includes the user's current calorie consumption value, system reward value, etc.

[0066] S106, projecting the video image onto a display device for display.

[0067] In one implementation, the terminal device of the first user connects the terminal device that needs to be projected to other display devices through mirroring screen projection technology, such as using other display devices that support miracast, AirPlay, Google Cast and other protocols. The connection can be made wirelessly (Wi-Fi) or wired (such as HDMI). Find and turn on the screen projection function of the device on the source device (such as the terminal device). Usually set to enter the screen projection option. Once the connection is successfully completed, the video screen of the source device will be automatically mirrored to other display devices.

[0068] In another implementation, a remote server processes and generates the video images to be displayed, and after the user's terminal device initiates screen projection, the screen projection is performed through DLNA screen projection technology.

[0069] Through screen projection, users can clearly see the exercise video and their own virtual image movements. In addition, through the interactive processing of other users' physiological data information and screen projection processing, the interactivity between users is further increased; through the display and interval division of heart rate data and integration into the fitness video screen for screen projection, users can more directly understand their exercise status, determine the current fat loss level of exercise, etc., which is convenient for users to adjust their exercise rhythm and achieve the desired exercise effect.

[0070] Figure 2 2 is a schematic block diagram of a terminal device 200 provided in an embodiment of the present application. As shown in the figure, the terminal device 200 may include: a transceiver unit 210, a processing unit 220, and a video acquisition unit 230.

[0071] It should be understood that the terminal device 200 may correspond to the terminal device in the method according to the embodiment of the present application, and the various units in the terminal device 200 and the above-mentioned other operations and / or functions are the corresponding processes of the above-mentioned method embodiments.

[0072] Wherein, when the terminal device 200 is used to execute Figure 1 In method 100, the video acquisition unit 230 acquires the user's action video, the transceiver unit 210 is used to receive the fitness video and send a request to the server to obtain the fitness video, as well as other steps involving the sending and receiving of data and / or information, and the processing unit 220 can be used to execute other steps in the method.

[0073] Figure 3 is a schematic block diagram of a terminal device 300 provided in an embodiment of the present application. Figure 3As shown, the terminal device 300 includes: at least one processor 310 and a communication interface 320. The processor 310 is coupled to a memory and is used to execute instructions stored in the memory to control the communication interface 320 to send and / or receive information and video files. Exemplarily, the terminal device 300 also includes a memory 330 for storing instructions.

[0074] It should be understood that the processor 310 and the memory 330 can be combined into one processing device, and the processor 310 is used to execute the program code stored in the memory 330 to implement the above functions. In specific implementation, the memory 330 can also be integrated into the processor 310, or independent of the processor 310.

[0075] In addition, an embodiment of the present application provides a fitness system, which includes a remote server and a terminal device installed with an application program for executing the above method. The fitness system may further include a remote coach terminal for viewing the user's fitness status. And the system may further include a display device for displaying a video screen to the user by means of screen projection.

[0076] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0077] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0078] It is understandable that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0079] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0080] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be an intelligent fitness mirror device, a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape) or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0081] The above embodiments may be implemented in a variety of ways. For example, the embodiments may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code may be executed on a suitable processor or set of processors, whether provided in a single computer or distributed among multiple computers.

[0082] Furthermore, the computer may be embodied in any of a variety of forms, such as a rack-mounted computer, a desktop computer, a laptop computer, or a tablet computer. In addition, the computer may be embedded in a device not generally regarded as a computer but having suitable processing capabilities, including a personal digital assistant (PDA), a smart phone, or any other suitable portable or fixed electronic device.

[0083] In addition, the computer may have one or more input and output devices. Among other things, these devices may also be used to present a user interface. Examples of output devices that may be used to provide a user interface include a printer or display screen for visual presentation of the output and a speaker or other sound-generating device for audible presentation of the output. Examples of input devices that may be used for a user interface include a keyboard and a pointing device, such as a mouse, a touch pad, and a digitized tablet computer. As another example, a computer may receive input information through speech recognition or other audible formats.

[0084] Such computers may be interconnected in a suitable form through one or more networks, including local area networks or wide area networks, such as enterprise networks, intelligent networks (IN) or the Internet. Such networks may be based on suitable technologies, may operate according to suitable protocols, and may include wireless networks, wired networks or fiber optic networks.

[0085] The various methods or processes outlined herein may be encoded as software that can be executed on one or more processors using any of a variety of operating systems or platforms. In addition, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may also be compiled into executable machine language code or intermediate code that is executed on a framework or virtual machine. Some embodiments may specifically employ one or more of a specific operating system or platform and a specific programming language and / or scripting tool to facilitate execution.

[0086] In the claims and the description above, all transitional phrases such as "includes," "comprising," "carrying," "having," "containing," "involving," "holding," "consisting of," etc. should be understood as open-ended, i.e., including but not limited to.

[0087] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A motion information interaction method in a screen projection scenario, characterized in that: The method includes: obtaining an action video of a first user; processing the action video of the first user to obtain action information, wherein the action information is display information of a virtual image of the first user, and the action characteristics of the virtual image of the user correspond to the action characteristics of the user; determining the biometric information of the first user, wherein the biometric information includes the user's heart rate information; receiving the biometric information of a second user, wherein the second user is one or more users different from the first user; processing the biometric information of the first user and / or the second user according to processing rules, and generating a video screen in combination with the display information of the virtual image of the first user and a fitness guidance video, wherein the video screen includes the screen of the fitness guidance video, the biometric information of the first user and / or the second user displayed according to the processing rules, and the virtual image of the first user; and projecting the video screen onto a display device for display.

2. The method according to claim 1, characterized in that: Processing the biometric information of the first user and / or the second user according to processing rules includes: processing the heart rate information according to the duration of different heart rate intervals, and marking the processed information with different colors or symbols.

3. The method according to claim 1 or 2, characterized in that: Processing the biological data information of the first user and / or the second user according to the processing rules includes: collecting statistics on the heart rate data information of the first user and / or the second user, comparing it with the preset target heart rate value, and displaying the compared result.

4. The method according to any one of claims 1 to 3, characterized in that: The biometric data information of the first user and / or the second user is processed according to processing rules, including: grouping the first and second users according to user characteristics of the first user and the second user, wherein the user characteristics include at least one of geographic location, age, gender, athletic ability level, team selection requirements, and athletic score; evaluating the athletic performance of each group based on at least the heart rate data of the users in each group, and comparing the evaluation results between different groups.

5. The method according to any one of claims 1 to 4, characterized in that: The video screen also includes feedback prompt content on the action of the first user, and the feedback prompt content is generated by the terminal after performing action recognition based on the action video of the first user; Alternatively, the feedback prompt content is feedback from the remote coach end after the display information of the virtual image is transmitted to the remote coach end.

6. The method according to any one of claims 2 to 5, characterized in that: Dynamically display the latest heart rate data of the first user and / or the second user in the video screen, continuously receive the heart rate data of the first user and / or the second user, dynamically count the distribution of durations of different heart rate levels and display them in the video screen; when the heart rate of the first user does not reach the set target heart rate, display prompt content in the video screen, and the displayed prompt content is used to instruct the first user to adjust the action in order to adjust the heart rate.

7. The method according to any one of claims 1 to 6, characterized in that: The calorie consumption is calculated based on the user's heart rate data, and the calorie value consumed by the user is displayed in the video screen.

8. The method according to any one of claims 1 to 7, characterized in that: All users are sorted according to the heart rate data of the first user and the second user, and the sorting rules include sorting by heart rate value or sorting by duration of a specified heart rate interval.

9. The method according to any one of claims 1 to 9, characterized in that: A virtual image of at least one second user is received, and the virtual image of the at least one second user is displayed in the video screen.

10. The method according to any one of claims 1 to 9, characterized in that: The heart rate data is an estimated heart rate set according to the exercise intensity of the current course.

11. A terminal device, characterized in that: The terminal device is used to execute the method according to any one of claims 1-11.

Images