Motion control method, device, related equipment and computer readable storage medium

Abstract

The embodiment of the present application discloses a motion control method, apparatus, related equipment and computer-readable storage medium. The method applied to a terminal includes: receiving an operation instruction initiated by a wearer of the terminal; in response to the operation instruction, decoding the video stream encoding information received from the first cloud node to obtain a motion posture key frame of a virtual object; the motion posture key frame of the virtual object is generated by the first cloud node based on motion model rendering; receiving a road surface recognition model sent by the first cloud node; recognizing the real-time road surface image taken based on the road surface recognition model to obtain corresponding road surface information; superimposing and displaying the road surface information and the motion posture key frame of the virtual object; and controlling the wearer of the terminal to follow the virtual object to move.

Description

Technical Field

[0001] The present application relates to the field of wireless communication technology, and in particular to a motion control method, apparatus, related equipment and computer-readable storage medium. Background Art

[0002] At present, during running training, data collection and processing are located on the local terminal side, resulting in limited types of collected data. In addition, due to the heavy computing load of the local terminal, problems such as device heating and short battery life occur, which not only affects the user experience, but also easily causes sports injuries to users. In addition, in the running training method of the related technology, the information displayed is mainly text or charts. If the user wants to view further detailed information, it needs to manually operate the device to achieve it, which greatly reduces the training effect.

[0003] Based on this, there is an urgent need for a running training method that ensures that the user's sports training effect is optimal while avoiding sports injuries to the user. Summary of the invention

[0004] In order to solve the technical problems existing in the related art, the embodiments of the present application provide a motion control method, an apparatus, related equipment and a computer-readable storage medium.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides a motion control method, which is applied to a terminal. The method includes:

[0007] Receiving an operation instruction initiated by a wearer of the terminal;

[0008] In response to the operation instruction, decoding the received video stream encoding information sent by the first cloud node to obtain a motion posture key frame of the virtual object; the motion posture key frame of the virtual object is generated by the first cloud node based on motion model rendering;

[0009] Receiving the road surface recognition model sent by the first cloud node;

[0010] Recognize the real-time road surface image based on the road surface recognition model to obtain corresponding road surface information;

[0011] Overlaying and displaying the road surface information and the key frames of the motion posture of the virtual object; and

[0012] The wearer of the terminal is controlled to follow the movement of the virtual object.

[0013] In the above scheme, the method further comprises:

[0014] When the road surface information and the motion posture key frame of the virtual object are superimposed and displayed, the virtual object is controlled to be displayed at a first position; and the first position is associated with the road surface information.

[0015] In the above scheme, the method further comprises:

[0016] Decoding the received video stream encoding information sent by the first cloud node to obtain key geographical location coordinates in the movement route of the virtual object;

[0017] The key geographical location coordinates in the movement route of the virtual object are compared with the local geographical location coordinates of the terminal to determine the key points in the movement route of the virtual object.

[0018] In the above scheme, the method further comprises:

[0019] After determining the key points in the movement route of the virtual object, collecting geographical scene pictures by a collection device;

[0020] The collected geographical scene image and the local geographical location coordinates of the terminal are sent to the first cloud node so that the first cloud node can determine the route information to be selected.

[0021] In the above scheme, the method further comprises:

[0022] receiving the route information to be selected sent by the first cloud node;

[0023] The route information to be selected is displayed on a display screen of the terminal; the route information to be selected includes at least the direction and key positions of the movement route of the virtual object.

[0024] The embodiment of the present application also provides a motion control method, which is applied to a first cloud node, and the method includes:

[0025] generating a motion model of a virtual object;

[0026] Based on the motion model of the virtual object, rendering generates motion posture key frames of the virtual object;

[0027] Encoding the motion posture key frame of the virtual object to obtain corresponding video stream encoding information, and sending the video stream encoding information to the terminal so that the terminal can obtain the motion posture key frame of the virtual object;

[0028] A road surface recognition model is generated by training, and the road surface recognition model is sent to the terminal so that the terminal can obtain road surface information.

[0029] In the above solution, generating a motion model of a virtual object includes:

[0030] generating a movement route and a movement frequency of the virtual object;

[0031] A motion model of the virtual object is generated based on the motion route and motion frequency of the virtual object.

[0032] In the above solution, generating the movement route of the virtual object includes:

[0033] Get geographic location information sent by external sensors;

[0034] Obtaining the user's physical information and historical movement information sent by the second cloud node;

[0035] Acquiring user physiological information and road image information sent by the terminal;

[0036] A movement route of the virtual object is generated based on the geographic location association information, the user body information, the historical movement information, the user physiological information and the road image information.

[0037] In the above solution, generating the movement frequency of the virtual object includes:

[0038] Sending a first request to the terminal; the first request is used to request to obtain user motion data;

[0039] Based on the first request, acquiring the user motion data sent by the terminal;

[0040] Based on the acquired user motion data, a motion frequency of the virtual object is generated.

[0041] In the above solution, the step of rendering and generating motion posture key frames of the virtual object based on the motion model of the virtual object includes:

[0042] Obtain user physiological information and standard motion posture model;

[0043] The motion model of the virtual object, the user's physiological information, and the standard motion posture model are combined to render and generate motion posture key frames of the virtual object.

[0044] In the above solution, the training and generating of the road surface recognition model includes:

[0045] Acquire road image information uploaded by the terminal;

[0046] The acquired road image information is matched and trained with a trained road surface recognition model to obtain a new trained road surface recognition model.

[0047] In the above scheme, the method further comprises:

[0048] Receiving the geographical scene picture sent by the terminal and the local geographical location coordinates of the terminal;

[0049] Based on the geographical scene image, the local geographical location coordinates of the terminal and a map information library, the route information to be selected is determined; the route information to be selected includes at least the direction and key positions of the movement route of the virtual object.

[0050] The present application also provides a motion control device, which is applied to a terminal. The device includes:

[0051] A first receiving unit, configured to receive an operation instruction initiated by a wearer of the terminal;

[0052] A decoding unit, configured to decode the video stream encoding information received from the first cloud node in response to the operation instruction to obtain a motion posture key frame of the virtual object; the motion posture key frame of the virtual object is generated by the first cloud node based on motion model rendering;

[0053] A second receiving unit, used to receive the road surface recognition model sent by the first cloud node;

[0054] An identification unit, used to identify the real-time road surface image captured based on the road surface identification model to obtain corresponding road surface information;

[0055] A display unit, used for superimposing and displaying the road surface information and the motion posture key frame of the virtual object;

[0056] A control unit is used to control the wearer of the terminal to move following the virtual object.

[0057] The present application embodiment further provides a motion control device, which is applied to a first cloud node, and the device includes:

[0058] A first generating unit, configured to generate a motion model of a virtual object;

[0059] A second generating unit, configured to render and generate motion posture key frames of the virtual object based on the motion model of the virtual object;

[0060] An encoding unit, used for encoding the motion posture key frames of the virtual object to obtain corresponding video stream encoding information;

[0061] A first sending unit, configured to send the video stream encoding information to a terminal, so that the terminal can obtain a motion posture key frame of the virtual object;

[0062] A third generating unit, used for training and generating a road surface recognition model;

[0063] The second sending unit is used to send the road surface recognition model to the terminal so that the terminal can obtain road surface information.

[0064] An embodiment of the present application further provides a terminal, the terminal comprising: a first processor and a first memory for storing a computer program that can be run on the first processor;

[0065] Wherein, the first processor is used to execute the steps of any of the above-mentioned terminal side methods when running the computer program.

[0066] The embodiment of the present application further provides a first cloud node device, the first cloud node device comprising: a second processor and a second memory for storing a computer program that can be run on the second processor;

[0067] Wherein, the second processor is used to execute the steps of any method on the first cloud node side when running the computer program.

[0068] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods on the terminal side are implemented, or the steps of any of the above-mentioned methods on the first cloud node side are implemented.

[0069] The motion control method, apparatus, related equipment and computer-readable storage medium provided in the embodiments of the present application, after the terminal receives an operation instruction initiated by the wearer of the terminal, in response to the operation instruction, decodes the video stream encoding information received from the first cloud node to obtain the motion posture key frame of the virtual object; the motion posture key frame of the virtual object is generated by the first cloud node based on motion model rendering; receives the road surface recognition model sent by the first cloud node; recognizes the real-time picture of the road surface taken based on the road surface recognition model to obtain corresponding road surface information; superimposes the road surface information with the motion posture key frame of the virtual object and displays it; and controls the wearer of the terminal to move with the virtual object.

[0070] By adopting the solution of the embodiment of the present application, the motion posture key frames of the virtual object are rendered and generated by training the road surface recognition model through the first cloud node. The terminal only needs to decode the video stream encoding information to obtain the motion posture key frames of the virtual object, and recognize the real-time picture of the road surface taken based on the road surface recognition model to obtain the corresponding road surface information. This can reduce the calculation amount of the terminal, increase the battery life of the device, and avoid the problem of low-temperature burns caused by long-term heating of the device. The terminal can also display the road surface information and the motion posture key frames of the virtual object at the same time. In this way, when the user wears the terminal device, he can see the virtual object moving, so that the user only needs to follow the virtual object to achieve the best sports training effect and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 A schematic diagram of a flow chart of a motion control method on a terminal side provided in an embodiment of the present application;

[0072] Figure 2 A schematic diagram of a flow chart of a motion control method on a first cloud node side provided in an embodiment of the present application;

[0073] Figure 3 A schematic diagram of a flow chart of a motion control method provided in an embodiment of the present application;

[0074] Figure 4 A schematic diagram of the architecture of a motion control system provided in an embodiment of the present application;

[0075] Figure 5 A schematic diagram of the structure of a wearable augmented reality sports glasses provided in an embodiment of the present application;

[0076] Figure 6 A schematic diagram of generating a motion route and a motion frequency provided in an embodiment of the present application;

[0077] Figure 7 An interactive schematic diagram of a motion control method provided in an embodiment of the present application;

[0078] Figure 8 A schematic diagram of the structure of a motion control device provided in an embodiment of the present application;

[0079] Fig. 9 A schematic diagram of the structure of another motion control device provided in an embodiment of the present application;

[0080] Fig.10 A schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0081] Fig.11 A schematic diagram of the structure of a first cloud node device provided in an embodiment of the present application;

[0082] Fig.12 A schematic diagram of the structure of a motion control system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0083] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0084] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments. However, it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments, and the technical solutions recorded in the embodiments of the present application may be combined with each other without conflict.

[0085] Before introducing the technical solutions of the embodiments of the present application, the related technologies are described below.

[0086] As computer technology matures and costs decrease, augmented reality technology will continue to change people's work and lifestyles. The high bandwidth and low latency characteristics of the fifth-generation mobile communication (5G) network and the rise of edge computing will accelerate the maturity of the industry and the popularization of terminals.

[0087] During amateur running training, each user's cardiopulmonary function and muscle strength are different. To achieve the best sports training effect, users need to accurately control and adjust their running posture, speed, time, distance, etc., otherwise it is easy to cause sports injuries to users. At present, mobile phones, pedometers and other products can monitor the user's sports status. The equipment that monitors the user's sports status can be roughly divided into two types: mechanical and electronic. Mechanical equipment is equipped with devices such as heavy hammers or vibration sensors. When the user vibrates while walking, the heavy hammers or vibration sensors trigger recording; electronic equipment will integrate low-cost circuit accelerometers and gyroscopes, and match them with global positioning systems (GPS) and heart rate sensors, etc., which can record speed, distance, heart rate, route and other information. At present, a large number of mobile phone applications (APP, Application) and bracelets and watches are based on this technical principle. It can be seen that all current running training systems rely on the equipment itself or cooperate with smart terminals to collect and process all information. Users can only see sports information based on text or charts. The display method is single and not intuitive, and it cannot provide guidance to users during exercise.

[0088] Several sports training programs in the related art are exemplified below.

[0089] Solution 1 in the related technology provides a pace speed recommendation method and device, which includes: obtaining the total target distance and total target time of the user, and dividing the total target distance and total target time into multiple time periods, then obtaining the route of the user and extracting the route information of the route, and recommending the pace speed for each time period based on the route information.

[0090] Solution 2 in the related technology proposes an augmented reality method, server and terminal, the method comprising: first, the server receives the current geographic location information of the terminal from the terminal, then the server receives a request message from the terminal, the request message is used to request to obtain geographic location associated data, then the server determines the geographic location associated data based on the current geographic location information of the terminal, and finally the server sends the geographic location associated data to the terminal, so that the terminal performs augmented reality processing based on the geographic location associated data.

[0091] Solution three in the related technology proposes a gait rehabilitation training method and system based on augmented reality, the method comprising: selecting a walking training environment for the lower limbs according to the assessment results of the walking function of the rehabilitation training patient; selecting a lower limb walking training mode; projecting the patient's footprints onto a treadmill conveyor belt to build an augmented reality environment for walking training; guiding the patient to perform walking training, judging the patient's foot stepping rate on the augmented reality footprints in the augmented reality environment for walking training within a preset time, and giving feedback; after the walking training is completed, processing the rehabilitation training data, and outputting an analysis of the reasons for loss of points in the walking training and a rehabilitation evaluation report.

[0092] However, the above-mentioned related art solutions have the following problems:

[0093] 1. The data collection and processing in the running training method of the related technology is located on the local terminal side, resulting in limited types of collected data; and because the local terminal has an excessive computing load, it will cause the device to heat up and have a short battery life; the device is large in size and weight, and inconvenient to wear, which will affect the user experience and other serious problems. For example, the second solution in the above-mentioned related technology is to upload the current geographic location information of the terminal to the server, and determine the geographic location associated data based on the current geographic location information on the server side, but still send the geographic location associated data to the terminal for augmented reality processing. It can be seen that in the running sports scenario, it is difficult to simultaneously meet the requirements of terminal size and weight, computing power, battery life and heating.

[0094] 2. In the running training methods of related technologies, the information displayed is mainly in the form of text or charts. If the user wants to view detailed information, he needs to manually operate the device and enter the secondary or even tertiary interface to see real-time statistical data. The viewing method is inconvenient and not intuitive. For users, the cold digital information lacks feedback, making training more boring, and it is difficult to help and guide the training, which greatly reduces the training effect.

[0095] It can be seen that the relevant technology lacks a running training method that ensures that the user's sports training effect is optimal while avoiding sports injuries to the user.

[0096] Based on this, in various embodiments of the present application, after the terminal receives an operation instruction initiated by the wearer of the terminal, the terminal decodes the video stream encoding information sent by the first cloud node in response to the operation instruction to obtain the motion posture key frame of the virtual object; the motion posture key frame of the virtual object is generated by the first cloud node based on the motion model rendering; the road surface recognition model sent by the first cloud node is received; the real-time picture of the road surface taken is recognized based on the road surface recognition model to obtain the corresponding road surface information; the road surface information is superimposed and displayed with the motion posture key frame of the virtual object; and the wearer of the terminal is controlled to follow the virtual object to move.

[0097] By adopting the solution of the embodiment of the present application, the motion posture key frames of the virtual object are rendered and generated by training the road surface recognition model through the first cloud node. The terminal only needs to decode the video stream encoding information to obtain the motion posture key frames of the virtual object, and recognize the real-time picture of the road surface taken based on the road surface recognition model to obtain the corresponding road surface information. This can reduce the calculation amount of the terminal, increase the battery life of the device, and avoid the problem of low-temperature burns caused by long-term heating of the device. The terminal can also display the road surface information and the motion posture key frames of the virtual object at the same time. In this way, when the user wears the terminal device, he can see the virtual object moving, so that the user only needs to follow the virtual object to achieve the best sports training effect and improve the user experience.

[0098] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0099] The present application embodiment provides a motion control method, which is applied to a terminal. Figure 1 A flow chart of a motion control method on a terminal side provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0100] Step 101, receiving an operation instruction initiated by a wearer of the terminal.

[0101] Step 102, in response to the operation instruction, decode the video stream encoding information received from the first cloud node to obtain the motion posture key frame of the virtual object.

[0102] In an embodiment of the present application, the motion posture key frames of the virtual object are generated by the first cloud node based on motion model rendering.

[0103] Step 103: Receive the road surface recognition model sent by the first cloud node.

[0104] Step 104 , recognize the real-time road surface image based on the road surface recognition model to obtain corresponding road surface information.

[0105] Step 105 , displaying the road surface information and the key frames of the motion posture of the virtual object in superposition; and controlling the wearer of the terminal to follow the movement of the virtual object.

[0106] In the embodiment of the present application, the terminal may be a wearable device, for example, the terminal is an augmented reality (AR) sports glasses. Here, the operation instruction is used to request to wear the terminal, such as requesting to wear AR sports glasses. In this way, when the user, i.e., the wearer of the terminal, wears the AR sports glasses, the user will see a virtual object moving on the display screen of the AR sports glasses, so that the user only needs to follow the movement state of the virtual object, such as movement frequency, movement route, movement posture, etc., to achieve the best sports training effect.

[0107] It should be noted that according to the characteristics of AR sports glasses themselves, superimposing virtual objects on the user's field of view will not affect the user's observation and perception of the surrounding environment, so there is no safety hazard.

[0108] In an embodiment of the present application, in response to a condition or state on which the executed operation depends, when the dependent condition or state is met, one or more operations executed may be in real time or have a set delay; unless otherwise specified, there is no restriction on the order of execution of the multiple operations executed.

[0109] Here, the first cloud node, such as an edge cloud node, has a low latency characteristic. The first cloud node generates a motion posture key frame of a virtual object through cloud rendering based on a motion model, and then performs joint video encoding on the rendered motion posture key frame of the virtual object and the key geographic location coordinates in the motion route of the pre-generated virtual object to obtain the corresponding video stream encoding information. In actual application, when the terminal receives the wearing operation instruction initiated by the user, the first cloud node sends the video stream encoding information to the terminal. After the terminal receives the video stream encoding information, it decodes the video stream encoding information, and can obtain not only the motion posture key frame of the virtual object, but also the key geographic location coordinates in the motion route of the virtual object.

[0110] In actual application, the terminal stores the obtained key geographical location coordinates in the movement route of the virtual object, and determines the key points in the movement route of the virtual object based on the key geographical location coordinates in the movement route of the virtual object.

[0111] Based on this, in some embodiments, the method further includes:

[0112] Decoding the received video stream encoding information sent by the first cloud node to obtain key geographical location coordinates in the movement route of the virtual object;

[0113] The key geographical location coordinates in the movement route of the virtual object are compared with the local geographical location coordinates of the terminal to determine the key points in the movement route of the virtual object.

[0114] Here, the key geographical location may be, for example, a turn, a fork in the road, or the like that appears in the movement route of the virtual object, and is not limited here.

[0115] In actual application, after the terminal determines the key points in the movement route of the virtual object, the first cloud node determines the route information to be selected.

[0116] Based on this, in some embodiments, the method further includes:

[0117] After determining the key points in the movement route of the virtual object, collecting geographical scene pictures by a collection device;

[0118] The collected geographical scene image and the local geographical location coordinates of the terminal are sent to the first cloud node so that the first cloud node can determine the route information to be selected.

[0119] Here, the acquisition device can be, for example, a camera installed on the terminal. Specifically, after the terminal determines the key points in the movement route of the virtual object, the terminal turns on the front camera and collects geographic scene pictures. The geographic scene pictures here can be continuous pictures or discontinuous pictures. Then, the terminal packages the collected geographic scene pictures and the local geographic location coordinates of the terminal and sends them to the first cloud node, and the first cloud node determines the route information to be selected based on this.

[0120] In actual application, after the first cloud node determines the route information to be selected, it will send the determined route information to be selected to the terminal. After the terminal receives the route information to be selected, it will clearly indicate the direction and key positions of the movement route on the terminal's display screen, so as to provide guidance for the user during the movement process.

[0121] Based on this, in some embodiments, the method further includes:

[0122] receiving the route information to be selected sent by the first cloud node;

[0123] The route information to be selected is displayed on a display screen of the terminal; the route information to be selected includes at least the direction and key positions of the movement route of the virtual object.

[0124] In actual application, in order to reduce the amount of calculation of the terminal, the first cloud node can train and generate a road surface recognition model. When the terminal receives an operation instruction initiated by the wearer of the terminal, it responds to the operation instruction, receives the road surface recognition model sent by the first cloud node, and recognizes the road surface information based on the road surface recognition model. Finally, the recognized road surface information and the key frames of the motion posture of the virtual object are superimposed and displayed on the display screen of the terminal.

[0125] It should be noted that in the embodiment of the present application, the display position of the virtual object is not arbitrary, but is calculated by reasoning of the road recognition model, thereby ensuring that the virtual object can be displayed at the appropriate position, and then controlling the wearer of the terminal to follow the virtual object in the first position to move, so as to achieve the best sports training effect.

[0126] Based on this, in some embodiments, the method further includes:

[0127] When the road surface information and the motion posture key frame of the virtual object are superimposed and displayed, the virtual object is controlled to be displayed at a first position; and the first position is associated with the road surface information.

[0128] Correspondingly, the embodiment of the present application further provides a motion control method, which is applied to the first cloud node. Figure 2 A flow chart of a motion control method on the first cloud node side provided in an embodiment of the present application, such as Figure 2 As shown, the method includes:

[0129] Step 201: Generate a motion model of a virtual object.

[0130] Step 202: Render and generate motion posture key frames of the virtual object based on the motion model of the virtual object.

[0131] Step 203 , encoding the motion posture key frames of the virtual object to obtain corresponding video stream encoding information, and sending the video stream encoding information to the terminal so that the terminal can obtain the motion posture key frames of the virtual object.

[0132] Step 204: train and generate a road surface recognition model, and send the road surface recognition model to the terminal so that the terminal can obtain road surface information.

[0133] In some embodiments, the generation of the motion model of the virtual object in step 201 may be implemented in the following manner:

[0134] generating a movement route and a movement frequency of the virtual object;

[0135] A motion model of the virtual object is generated based on the motion route and motion frequency of the virtual object.

[0136] The following is an explanation of the process of generating the movement route and movement frequency of the virtual object.

[0137] In some embodiments, generating a motion path of the virtual object includes:

[0138] Get geographic location information sent by external sensors;

[0139] Obtaining the user's physical information and historical movement information sent by the second cloud node;

[0140] Acquiring user physiological information and road image information sent by the terminal;

[0141] A movement route of the virtual object is generated based on the geographic location association information, the user body information, the historical movement information, the user physiological information and the road image information.

[0142] Specifically, the first cloud node sends a second request to the external sensor, the second request is used to request to obtain geographic location-related information, and the external sensor returns the obtained geographic location-related information result to the first cloud node. In actual implementation, the terminal can first upload geographic location data to the first cloud node, such as GPS data or Beidou data, and then the first cloud node sends a second request to the surrounding external sensors according to the received geographic location coordinates to request to obtain the surrounding geographic location-related information; wherein the geographic location-related information includes but is not limited to one of the following: temperature, humidity, wind direction, wind force, air pressure, air quality, altitude. The first cloud node sends a third request to the second cloud node (such as the central cloud node), the third request is used to request to obtain the user's physical information and historical motion information. Usually, the user's physical information and historical motion information are stored on the second cloud node. The first cloud node sends a request to the second cloud node, and the second cloud node returns the user's physical information and historical motion information to the first cloud node according to the third request; wherein the user's physical information includes but is not limited to one of the following: age, height, weight, foot type, medical history; historical motion information includes but is not limited to one of the following: historical running volume, pace, exercise duration, historical motion route. The first cloud node sends a fourth request to the terminal, and the fourth request is used to request to obtain the user's physiological information and road image information. In actual implementation, after receiving the fourth request sent by the first cloud node, the terminal can collect the user's physiological information through the optical heart rate sensor located at the temple, and collect the road image information through the camera device, and then send all the collected user's physiological information and road image information to the first cloud node; wherein the user's physiological information includes but is not limited to one of the following: heart rate, respiratory rate, oxygen uptake, blood oxygen level, blood pressure, blood perfusion, and the user's physiological information here is a digital signal. The first cloud node can calculate and generate the movement route of the virtual object based on the obtained geographical location association information, user's body information, historical movement information, user's physiological information and road image information.

[0143] It should be noted that the movement route of the virtual object can be generated in real time based on the terminal wearer's physical information, historical movement information, physiological information, geographic location information and road image information, or it can be generated by the above-mentioned relevant information of other users, which is not limited here.

[0144] In some embodiments, generating the movement frequency of the virtual object includes:

[0145] Sending a first request to the terminal; the first request is used to request to obtain user motion data;

[0146] Based on the first request, acquiring the user motion data sent by the terminal;

[0147] Based on the acquired user motion data, a motion frequency of the virtual object is generated.

[0148] Specifically, the first cloud node sends a first request to the terminal to request to obtain the user motion data. After receiving the first request, the terminal collects the user motion data and feeds the collected user motion data back to the first cloud node so that the first cloud node can calculate the motion frequency of the virtual object. The user motion data includes but is not limited to one of the following: step length, slope, and position information.

[0149] In some embodiments, for the motion model of the virtual object in step 202, rendering and generating the motion posture key frame of the virtual object can be implemented in the following manner:

[0150] Obtain user physiological information and standard motion posture model;

[0151] The motion model of the virtual object, the user's physiological information, and the standard motion posture model are combined to render and generate motion posture key frames of the virtual object.

[0152] Here, the user's physiological information includes but is not limited to one of the following: heart rate, respiratory rate, oxygen uptake, blood oxygen level, blood pressure, and blood perfusion; the first cloud node generates a first motion posture model based on the motion model of the virtual object and the user's physiological information, and then matches the first motion posture model with the motion standard posture model to render the motion posture key frame of the virtual object. In actual application, the first cloud node will focus on marking the key parts and actions in the motion posture key frame of the virtual object.

[0153] In an embodiment of the present application, the first cloud node, such as an edge cloud node, has a low latency characteristic. The first cloud node generates motion posture key frames of the virtual object through cloud rendering based on the motion model, and then performs joint video encoding on the rendered motion posture key frames of the virtual object and the key geographic location coordinates in the pre-generated motion route of the virtual object to obtain corresponding video stream encoding information.

[0154] In actual application, the first cloud node trains and generates a road surface recognition model, and transmits the road surface recognition model to the terminal, so that the terminal can recognize the real-time image of the road surface captured based on the road surface recognition model to obtain the corresponding road surface information.

[0155] Based on this, in some embodiments, the training and generation of the road surface recognition model in step 204 can be implemented in the following manner:

[0156] Acquire road image information uploaded by the terminal;

[0157] The acquired road image information is matched and trained with a trained road surface recognition model to obtain a new trained road surface recognition model.

[0158] Here, after the first cloud node obtains the road image information, the road image information can be stored in the road surface sample library, and the road surface recognition model can be trained using the existing model training method to obtain a new trained road surface recognition model, which will not be repeated here.

[0159] In some embodiments, the method further comprises:

[0160] Receiving the geographical scene picture sent by the terminal and the local geographical location coordinates of the terminal;

[0161] Based on the geographical scene image, the local geographical location coordinates of the terminal and a map information library, the route information to be selected is determined; the route information to be selected includes at least the direction and key positions of the movement route of the virtual object.

[0162] Here, the geographic scene picture can be a continuous picture or a discontinuous picture. After the terminal determines the key points in the movement route of the virtual object, the terminal turns on the front camera and collects the geographic scene picture, and packages the collected geographic scene picture and the local geographic location coordinates of the terminal to the first cloud node, and the first cloud node determines the route information to be selected based on this.

[0163] In actual application, after the first cloud node determines the route information to be selected, it will send the determined route information to be selected to the terminal. After the terminal receives the route information to be selected, it will clearly indicate the direction and key positions of the movement route on the terminal's display screen, so as to provide guidance for the user during the movement process.

[0164] The present application also provides a motion control method. Figure 3 A flow chart of a motion control method provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the method includes:

[0165] Step 301: The first cloud node generates a motion model of a virtual object, and based on the motion model of the virtual object, renders and generates motion posture key frames of the virtual object.

[0166] Step 302: The first cloud node encodes the motion posture key frames of the virtual object to obtain corresponding video stream encoding information.

[0167] Step 303: The first cloud node trains and generates a road surface recognition model.

[0168] Step 304, after receiving the operation instruction initiated by the wearer of the terminal, the terminal receives the video stream encoding information sent by the first cloud node in response to the operation instruction; and decodes the video stream encoding information to obtain the motion posture key frame of the virtual object.

[0169] Step 305: The terminal receives the road surface recognition model sent by the first cloud node, and recognizes the captured real-time road surface image based on the road surface recognition model to obtain corresponding road surface information.

[0170] Step 306: The terminal overlays and displays the road surface information and the motion posture key frame of the virtual object.

[0171] Step 307: The terminal controls the wearer of the terminal to move following the virtual object.

[0172] It should be noted that the specific processing process of the terminal and the first cloud node has been described in detail above and will not be repeated here.

[0173] The motion control method provided in the embodiment of the present application generates motion posture key frames of virtual objects and generates a road surface recognition model through training by rendering a first cloud node. The terminal only needs to decode the video stream encoding information to obtain the motion posture key frames of the virtual object, and recognize the real-time picture of the road surface taken based on the road surface recognition model to obtain the corresponding road surface information. This can reduce the amount of calculation of the terminal, increase the battery life of the device, and avoid the problem of low-temperature burns caused by long-term heating of the device. The terminal can also display the road surface information and the motion posture key frames of the virtual object at the same time. In this way, when the user wears the terminal device, he can see the virtual object moving, so that the user only needs to follow the virtual object to achieve the best sports training effect and improve the user experience.

[0174] The present application is further described in detail below in conjunction with application examples.

[0175] Taking the running scene as an example, in this application embodiment, after the user wears a terminal such as AR sports glasses, he can see a virtual object running in front of his field of vision. The user only needs to follow the movement state of the virtual object, such as movement frequency, movement route, movement posture, etc., to achieve the best running training effect. At the same time, according to the characteristics of AR sports glasses, superimposing the virtual object in the user's field of vision will not affect the user's observation and perception of the surrounding environment, so there is no safety hazard.

[0176] Figure 4 A schematic diagram of the architecture of a motion control system provided in an embodiment of the present application, such as Figure 4As shown, the motion control system of the embodiment of the present application includes the terminal side, the edge cloud (first cloud node), and the central cloud (second cloud point). In order to ensure the user's good wearing comfort, the terminal side needs to fully consider the weight, volume, battery life, heat generation and other issues of the terminal. Therefore, the terminal side only has the necessary functions such as the collection, display, and transmission of some information. The collection of other geographic location related information, the rendering of virtual objects, the training of road recognition models and other computing functions are placed on the edge cloud for processing. The terminal should try to avoid active and periodic data uploads.

[0177] In this application embodiment, the terminal in the architecture of the motion control system can be designed as wearable AR sports glasses. Figure 5 A schematic diagram of the structure of a wearable augmented reality (AR) sports glasses provided in an embodiment of the present application. The main functions of the wearable AR sports glasses include collecting and uploading user physiological information, geographic location information, image acquisition, audio and video decoding, display, etc. Figure 5 As shown, the structure of wearable AR sports glasses mainly includes a computing module (central processing unit (CPU) / graphics processing unit (GPU)), a communication module (such as cellular communication), a positioning module (GPS), an image acquisition module (using a spatial computing camera for image acquisition), a power (management) module, an (optical) heart rate acquisition module, an inertial measurement unit (IMU) (including an acceleration sensor, a gravity sensor, a distance sensor) module, a sound module, a display module, etc.

[0178] The edge cloud in the motion control system architecture is responsible for information processing and calculation, and calculates the movement route and movement frequency of the virtual object by obtaining geographic location-related information, user physiological information, user movement data, historical movement information, etc. In addition, the edge cloud is also responsible for rendering and encoding the key frames of the virtual object's movement posture, training the road recognition model, etc. Among them, the edge cloud system is composed of Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS). Among them, IaaS needs to have underlying functions such as image rendering and AI computing; PaaS needs to have key technical capabilities such as cloud rendering, cloud perception, and model training; SaaS needs to deploy business application modules connected to AR terminals. The central cloud in the motion control system architecture is responsible for user authentication, management and storage of user data, and storage of maps and running routes.

[0179] The implementation process of the motion control method of the embodiment of the present application is described in detail below.

[0180] Figure 6 A schematic diagram of generating a motion route and a motion frequency is provided in an embodiment of the present application, such as Figure 6 As shown, wearable AR sports glasses are referred to as AR glasses for short, and send geographic location data, such as GPS data or Beidou data, to the edge cloud. The edge cloud stores the geographic location data and sends a request to the surrounding external sensors based on the received geographic location data to request to obtain the surrounding geographic location-related information, wherein the geographic location-related information includes but is not limited to one of the following: temperature, humidity, wind direction, wind force, air pressure, air quality, and altitude; the edge cloud sends a request to the central cloud to request to obtain the user's physical information and historical motion information. After receiving the request, the central cloud returns the user's physical information and historical motion information to the edge cloud, wherein the user's physical information includes but is not limited to one of the following: age, height, weight, foot type, and medical history; the historical motion information includes but is not limited to one of the following: historical running volume, pace, exercise duration, and historical motion route; the edge cloud will also send a request to the AR glasses to request the AR glasses to collect the user's physiological information and road image information. After collecting the user's physiological information and road image information, the AR glasses return it to the edge cloud. In this way, the edge cloud generates the motion route of the virtual object based on the geographic location-related information, the user's physical information, the historical motion information, the user's physiological information, and the road image information. In addition, the edge cloud sends a request to the AR glasses, requesting the AR glasses to collect user motion data. After receiving the request, the AR glasses return the collected user motion data to the edge cloud, and then the edge cloud generates the motion frequency of the virtual object based on the acquired user motion data.

[0181] based on Figure 6 The interactive graph of movement routes and movement frequencies is shown. Figure 7 An interactive schematic diagram of a motion control method provided in an embodiment of the present application, such as Figure 7 As shown in the figure, before the edge cloud generates the motion route and motion frequency, the AR glasses send a user authentication request to the central cloud. When the central cloud successfully authenticates, it returns the user authentication result to the AR glasses. The central cloud sends a service initialization request to the edge cloud. After receiving the service initialization request, the edge cloud performs service initialization and returns the result of service initialization to the AR glasses, and then adopts the same method as in the example above. Figure 6 The same processing process generates the movement route and movement frequency, which will not be repeated here. Please refer to the above Figure 6Description. After generating the motion route and motion frequency, the edge cloud generates a motion model of the virtual object based on the motion route and motion frequency of the virtual object, and then renders and generates the motion posture key frame of the virtual object (virtual object rendering), and encodes the motion posture key frame of the virtual object to obtain the corresponding video stream encoding information, and sends the video stream encoding information to the AR glasses, and the AR glasses decode the video stream encoding information to obtain the motion posture key frame of the virtual object; at the same time, the edge cloud will also train and generate a road surface recognition model, and send the generated road surface recognition model to the AR glasses. The AR glasses recognize the real-time road surface image taken based on the road surface recognition model to obtain the corresponding road surface information, and then the road surface information and the motion posture key frame of the virtual object (virtual object) are superimposed on the display screen of the AR glasses. At this time, the AR glasses send a service termination request to the edge cloud. After receiving the service termination request, the edge cloud saves all user data in the central cloud, and the central cloud logs out after storing the user data; in addition, after receiving the service termination request, the edge cloud will also save the road surface recognition model. After the road surface recognition model is saved, the user logs out and releases the resources.

[0182] Considering that the pedometers and solutions in the related art cannot provide an intuitive and convenient information display method, the embodiment of the present application starts from the user's wearing experience, and places all modules except the user's physiological information, geographic information, motion information and display functions on the edge cloud node for processing, which effectively reduces the weight of the terminal device, increases the endurance of the terminal device, and avoids the problem of low-temperature burns caused by long-term wearing and heating. At the same time, the running route of the virtual object followed by the user is calculated and generated by the user's own physiological information, real-time environmental data and historical motion data; the motion posture of the virtual object is customized and generated by the user's real-time motion information, ensuring that the user can maximize the training effect while avoiding sports injuries; the display position of the virtual object is calculated by the road surface recognition model inference, and the training process of the road surface recognition model is processed by the edge cloud to ensure that the virtual object is displayed in the appropriate position, that is, displayed in the first position.

[0183] In order to implement the motion control method on the terminal side of the embodiment of the present application, the embodiment of the present application also provides a motion control device, which is arranged on the terminal. Figure 8 A schematic diagram of the structure of a motion control device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the device comprises:

[0184] A first receiving unit 81, configured to receive an operation instruction initiated by a wearer of the terminal;

[0185] The decoding unit 82 is used to decode the video stream encoding information received from the first cloud node in response to the operation instruction to obtain a motion posture key frame of the virtual object; the motion posture key frame of the virtual object is generated by the first cloud node based on motion model rendering;

[0186] A second receiving unit 83, configured to receive the road surface recognition model sent by the first cloud node;

[0187] The recognition unit 84 is used to recognize the real-time road surface image captured based on the road surface recognition model to obtain corresponding road surface information;

[0188] A display unit 85, used for superimposing and displaying the road surface information and the motion posture key frame of the virtual object;

[0189] The control unit 86 is used to control the wearer of the terminal to move along with the virtual object.

[0190] In some embodiments, the control unit 86 is further used to: when the display unit 85 superimposes and displays the road surface information and the motion posture key frame of the virtual object, control the virtual object to be displayed at a first position; the first position is associated with the road surface information.

[0191] In some embodiments, the decoding unit 82 is further used to: decode the video stream encoding information received from the first cloud node to obtain key geographical location coordinates in the movement route of the virtual object;

[0192] The device also includes: a first determining unit, which is used to compare the key geographical location coordinates in the movement route of the virtual object with the local geographical location coordinates of the terminal to determine the key points in the movement route of the virtual object.

[0193] In some embodiments, the apparatus further comprises:

[0194] A collection unit, configured to collect a geographical scene image through a collection device after the first determination unit determines the key point in the movement route of the virtual object;

[0195] The third sending unit is used to send the collected geographical scene image and the local geographical location coordinates of the terminal to the first cloud node, so that the first cloud node can determine the route information to be selected.

[0196] In some embodiments, the apparatus further comprises:

[0197] A third receiving unit, configured to receive the route information to be selected sent by the first cloud node;

[0198] The display unit is further used to display the route information to be selected on the display screen of the terminal; the route information to be selected includes at least the direction and key positions of the movement route of the virtual object.

[0199] In actual application, the decoding unit 82, the identification unit 84, the display unit 85, and the control unit 86 can be implemented by a processor in the motion control device, and the first receiving unit 81 and the second receiving unit 83 can be implemented by a communication interface in the motion control device.

[0200] In order to implement the motion control method on the first cloud node side of the embodiment of the present application, the embodiment of the present application also provides another motion control device, which is arranged on the first cloud node, such as an edge cloud node. Fig. 9 A schematic diagram of the structure of another motion control device provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown, the device comprises:

[0201] A first generating unit 91, configured to generate a motion model of a virtual object;

[0202] A second generating unit 92 is used for rendering and generating motion posture key frames of the virtual object based on the motion model of the virtual object;

[0203] The encoding unit 93 is used to encode the motion posture key frames of the virtual object to obtain corresponding video stream encoding information;

[0204] A first sending unit 94 is used to send the video stream encoding information to the terminal so that the terminal can obtain the motion posture key frame of the virtual object;

[0205] The third generating unit 95 is used for training and generating a road surface recognition model;

[0206] The second sending unit 96 is used to send the road surface recognition model to the terminal so that the terminal can obtain road surface information.

[0207] In some embodiments, the first generating unit 91 is specifically configured to:

[0208] generating a movement route and a movement frequency of the virtual object;

[0209] A motion model of the virtual object is generated based on the motion route and motion frequency of the virtual object.

[0210] Here, the first generating unit 91 is specifically used for:

[0211] Get geographic location information sent by external sensors;

[0212] Obtaining the user's physical information and historical movement information sent by the second cloud node;

[0213] Acquiring user physiological information and road image information sent by the terminal;

[0214] A movement route of the virtual object is generated based on the geographic location association information, the user body information, the historical movement information, the user physiological information and the road image information.

[0215] Here, the first generating unit 91 is specifically used for:

[0216] Sending a first request to the terminal; the first request is used to request to obtain user motion data;

[0217] Based on the first request, acquiring the user motion data sent by the terminal;

[0218] Based on the acquired user motion data, a motion frequency of the virtual object is generated.

[0219] In some embodiments, the second generating unit 92 is specifically configured to:

[0220] Obtain user physiological information and standard motion posture model;

[0221] The motion model of the virtual object, the user's physiological information, and the standard motion posture model are combined to render and generate motion posture key frames of the virtual object.

[0222] In some embodiments, the third generating unit 95 is specifically configured to:

[0223] Acquire road image information uploaded by the terminal;

[0224] The acquired road image information is matched and trained with a trained road surface recognition model to obtain a new trained road surface recognition model.

[0225] In some embodiments, the apparatus further comprises:

[0226] A fourth receiving unit, configured to receive the geographical scene picture sent by the terminal and the local geographical location coordinates of the terminal;

[0227] The second determination unit is used to determine the route information to be selected based on the geographical scene image, the local geographical location coordinates of the terminal and the map information library; the route information to be selected includes at least the direction and key positions of the movement route of the virtual object.

[0228] In actual application, the first generating unit 91, the second generating unit 92, the encoding unit 93, and the third generating unit 95 can be implemented by a processor in the motion control device, and the first sending unit 94 and the second sending unit 96 can be implemented by a communication interface in the motion control device.

[0229] It should be noted that the motion control device provided in the above embodiment only uses the division of the above program modules as an example when controlling the wearer of the terminal to follow the virtual object to move. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the motion control device provided in the above embodiment and the motion control method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0230] Based on the hardware implementation of the above program module, and in order to implement the method on the terminal side of the embodiment of the present application, the embodiment of the present application also provides a terminal, Fig.10 A schematic diagram of the structure of a terminal provided in an embodiment of the present application, such as Fig.10 As shown, the terminal 100 includes:

[0231] The first communication interface 1001 is capable of exchanging information with the first cloud node;

[0232] The first processor 1002 is connected to the first communication interface 1001 to implement information interaction with the first cloud node, and is used to execute the method provided by one or more technical solutions on the terminal side when running the computer program. The computer program is stored in the first memory 1003.

[0233] Specifically, the first communication interface 1001 is used to receive an operation instruction initiated by the wearer of the terminal; and receive a road surface recognition model sent by the first cloud node;

[0234] The first processor 1002 is used to decode the video stream encoding information received from the first cloud node in response to the operation instruction to obtain the motion posture key frame of the virtual object; the motion posture key frame of the virtual object is generated by the first cloud node based on the motion model rendering; the real-time picture of the road surface taken is recognized based on the road surface recognition model to obtain the corresponding road surface information; the road surface information is superimposed and displayed with the motion posture key frame of the virtual object; and the wearer of the terminal is controlled to follow the virtual object to move.

[0235] In some embodiments, the first processor 1002 is further configured to:

[0236] When the road surface information and the key frame of the motion posture of the virtual object are superimposed and displayed, the virtual object is controlled to be displayed at a first position; the first position is associated with the road surface information.

[0237] In some embodiments, the first processor 1002 is further configured to:

[0238] Decoding the received video stream encoding information sent by the first cloud node to obtain key geographical location coordinates in the movement route of the virtual object;

[0239] The key geographical location coordinates in the movement route of the virtual object are compared with the local geographical location coordinates of the terminal to determine the key points in the movement route of the virtual object.

[0240] In some embodiments, the first processor 1002 is further configured to:

[0241] After determining the key points in the movement route of the virtual object, collecting geographical scene pictures through a collection device;

[0242] The first communication interface 1001 is further used for:

[0243] The collected geographical scene image and the local geographical location coordinates of the terminal are sent to the first cloud node so that the first cloud node can determine the route information to be selected.

[0244] In some embodiments, the first communication interface 1001 is further used to: receive the route information to be selected sent by the first cloud node;

[0245] The first processor 1002 is further configured to: display the route information to be selected on a display screen of the terminal; the route information to be selected includes at least a direction and a key position of a movement route of the virtual object.

[0246] It should be noted that the specific processing process of the first communication interface 1001 and the first processor 1002 is detailed in the method embodiment and will not be repeated here.

[0247] Of course, in actual application, the various components in the terminal 100 are coupled together through the bus system 1004. It can be understood that the bus system 1004 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1004 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig.10 Various buses are labeled as bus system 1004 .

[0248] The first memory 1003 in the embodiment of the present application is used to store various types of data to support the operation of the terminal 100. Examples of such data include: any computer program used to operate on the terminal 100.

[0249] The method disclosed in the above embodiment of the present application can be applied to the first processor 1002, or implemented by the first processor 1002. The first processor 1002 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the first processor 1002 or an instruction in the form of software. The above-mentioned first processor 1002 may be a general-purpose processor, a digital signal processor (DSP, DigitalSignal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The first processor 1002 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the first memory 1003, and the first processor 1002 reads the information in the first memory 1003, and completes the steps of the aforementioned terminal side method in combination with its hardware.

[0250] In an exemplary embodiment, the terminal 100 may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor or other electronic components to execute the aforementioned terminal side method.

[0251] Based on the hardware implementation of the above program module, and in order to implement the method on the first cloud node side of the embodiment of the present application, the embodiment of the present application also provides a first cloud node device, Fig.11 A schematic diagram of the structure of a first cloud node device provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, the first cloud node device 110 includes:

[0252] The second communication interface 1101 is capable of exchanging information with the terminal;

[0253] The second processor 1102 is connected to the second communication interface 1101 to implement information interaction with the terminal, and is used to execute the method provided by one or more technical solutions of the first cloud node side when running the computer program. The computer program is stored in the second memory 1103.

[0254] Specifically, the second processor 1102 is used to generate a motion model of a virtual object; based on the motion model of the virtual object, render and generate motion posture key frames of the virtual object; encode the motion posture key frames of the virtual object to obtain corresponding video stream encoding information; and is also used to train and generate a road surface recognition model;

[0255] The second communication interface 1101 is used to send the video stream encoding information to the terminal so that the terminal can obtain the motion posture key frame of the virtual object; and is also used to send the road surface recognition model to the terminal so that the terminal can obtain road surface information.

[0256] In some embodiments, the second processor 1102 is specifically configured to:

[0257] generating a movement route and a movement frequency of the virtual object;

[0258] A motion model of the virtual object is generated based on the motion route and motion frequency of the virtual object.

[0259] In some embodiments, the second processor 1102 is specifically configured to:

[0260] Get geographic location information sent by external sensors;

[0261] Obtaining the user's physical information and historical movement information sent by the second cloud node;

[0262] Acquiring user physiological information and road image information sent by the terminal;

[0263] A movement route of the virtual object is generated based on the geographic location association information, the user body information, the historical movement information, the user physiological information and the road image information.

[0264] In some embodiments, the second processor 1102 is specifically configured to:

[0265] Sending a first request to the terminal; the first request is used to request to obtain user motion data;

[0266] Based on the first request, acquiring the user motion data sent by the terminal;

[0267] Based on the acquired user motion data, a motion frequency of the virtual object is generated.

[0268] In some embodiments, the second processor 1102 is specifically configured to:

[0269] Obtain user physiological information and standard motion posture model;

[0270] The motion model of the virtual object, the user's physiological information, and the standard motion posture model are combined to render and generate motion posture key frames of the virtual object.

[0271] In some embodiments, the second processor 1102 is specifically configured to:

[0272] Acquire road image information uploaded by the terminal;

[0273] The acquired road image information is matched and trained with a trained road surface recognition model to obtain a new trained road surface recognition model.

[0274] In some embodiments, the second communication interface 1101 is further used to: receive a geographical scene picture sent by the terminal and the local geographical location coordinates of the terminal;

[0275] The second processor 1102 is further used to determine the route information to be selected based on the geographic scene image, the local geographic location coordinates of the terminal and the map information library; the route information to be selected includes at least the direction and key positions of the movement route of the virtual object.

[0276] It should be noted that the specific processing process of the second communication interface 1101 and the second processor 1102 is detailed in the method embodiment and will not be repeated here.

[0277] Of course, in actual application, the various components in the first cloud node device 110 are coupled together through the bus system 1104. It can be understood that the bus system 1104 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1104 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig.11 Various buses are labeled as bus system 1104 .

[0278] The second memory 1103 in the embodiment of the present application is used to store various types of data to support the operation of the first cloud node device 110. Examples of such data include: any computer program used to operate on the first cloud node device 110.

[0279] The method disclosed in the above embodiment of the present application can be applied to the second processor 1102, or implemented by the second processor 1102. The second processor 1102 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the second processor 1102 or an instruction in software form. The above-mentioned second processor 1102 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The second processor 1102 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the second memory 1103. The second processor 1102 reads the information in the second memory 1103 and completes the steps of the above-mentioned first cloud node side method in combination with its hardware.

[0280] In an exemplary embodiment, the first cloud node device 110 may be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, Microprocessors, or other electronic components to execute the aforementioned method on the first cloud node side.

[0281] It can be understood that the memory (first memory 1003 and second memory 1103) of the embodiment of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory.

[0282] Volatile memory can be random access memory (RAM, Random Access Memory), which is used as external high-speed cache. By exemplary but not restrictive explanation, many forms of RAM are available, such as static random access memory (SRAM, Static Random Access Memory), synchronous static random access memory (SSRAM, Synchronous Static Random Access Memory), dynamic random access memory (DRAM, Dynamic Random Access Memory), synchronous dynamic random access memory (SDRAM, Synchronous Dynamic Random Access Memory), double data rate synchronous dynamic random access memory (DDRSDRAM, Double Data Rate Synchronous Dynamic Random Access Memory), enhanced synchronous dynamic random access memory (ESDRAM, Enhanced Synchronous Dynamic Random Access Memory), synchronous connection dynamic random access memory (SLDRAM, Sync Link Dynamic Random Access Memory), direct memory bus random access memory (DRRAM, Direct Rambus Random Access Memory). The memory described in the embodiment of the application is intended to include but is not limited to these and any other suitable type of memory.

[0283] In order to implement the method of the embodiment of the present application, the embodiment of the present application also provides a motion control system, Fig.12 A schematic diagram of the structure of the motion control system provided in the embodiment of the present application, such as Fig.12 As shown, the system includes:

[0284] The first cloud node 121 is used to generate a motion model of a virtual object; based on the motion model of the virtual object, render and generate motion posture key frames of the virtual object; encode the motion posture key frames of the virtual object to obtain corresponding video stream encoding information, and send the video stream encoding information to the terminal so that the terminal obtains the motion posture key frames of the virtual object; train and generate a road surface recognition model, and send the road surface recognition model to the terminal so that the terminal obtains road surface information;

[0285] Terminal 122 is used to receive an operation instruction initiated by the wearer of the terminal; in response to the operation instruction, receive the video stream encoding information sent by the first cloud node 121, and decode the video stream encoding information to obtain the motion posture key frame of the virtual object; it is also used to receive the road surface recognition model sent by the first cloud node 121, and recognize the real-time road surface image taken based on the road surface recognition model to obtain corresponding road surface information; it is also used to superimpose the road surface information with the motion posture key frame of the virtual object for display; and control the wearer of the terminal to follow the virtual object to move.

[0286] Here, the motion posture key frame of the virtual object is generated by the first cloud node based on motion model rendering.

[0287] It should be noted that the specific processing process of the first cloud node 121 and the terminal 122 has been described in detail above and will not be repeated here.

[0288] In an exemplary embodiment, the embodiment of the present application also provides a computer-readable storage medium, for example, including a first memory 1003 storing a computer program, and the computer program can be executed by the first processor 1002 of the terminal 100 to complete the steps described in the aforementioned terminal-side method. For another example, including a second memory 1103 storing a computer program, the computer program can be executed by the second processor 1102 of the first cloud node device 110 to complete the steps described in the aforementioned first cloud node-side method. Among them, the computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, FlashMemory, magnetic surface memory, optical disk or CD-ROM; it can also be various devices including one or any combination of the above memories.

[0289] In the embodiments of the present application, the terms "first", "second", etc. are merely used to distinguish similar objects and do not represent a specific order or sequence for the objects. It can be understood that "first", "second", etc. can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0290] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A motion control method, characterized in that: Applied to a terminal, the method comprises: Receiving an operation instruction initiated by a wearer of the terminal; In response to the operation instruction, decoding the received video stream encoding information sent by the first cloud node to obtain a motion posture key frame of the virtual object; the motion posture key frame of the virtual object is generated by the first cloud node based on motion model rendering; Receiving the road surface recognition model sent by the first cloud node; Recognize the real-time road surface image based on the road surface recognition model to obtain corresponding road surface information; Overlaying and displaying the road surface information and the key frames of the motion posture of the virtual object; and The wearer of the terminal is controlled to follow the movement of the virtual object.

2. The method according to claim 1, characterized in that The method further comprises: When the road surface information and the motion posture key frame of the virtual object are superimposed and displayed, the virtual object is controlled to be displayed at a first position; and the first position is associated with the road surface information.

3. The method according to claim 1, characterized in that The method further comprises: Decoding the received video stream encoding information sent by the first cloud node to obtain key geographical location coordinates in the movement route of the virtual object; The key geographical location coordinates in the movement route of the virtual object are compared with the local geographical location coordinates of the terminal to determine the key points in the movement route of the virtual object.

4. The method according to claim 3, characterized in that The method further comprises: After determining the key points in the movement route of the virtual object, collecting geographical scene pictures by a collection device; The collected geographical scene image and the local geographical location coordinates of the terminal are sent to the first cloud node so that the first cloud node can determine the route information to be selected.

5. The method according to claim 4, characterized in that The method further comprises: receiving the route information to be selected sent by the first cloud node; The route information to be selected is displayed on a display screen of the terminal; the route information to be selected includes at least the direction and key positions of the movement route of the virtual object.

6. A motion control method, characterized in that: Applied to a first cloud node, the method comprises: generating a motion model of a virtual object; Based on the motion model of the virtual object, rendering generates motion posture key frames of the virtual object; Encoding the motion posture key frame of the virtual object to obtain corresponding video stream encoding information, and sending the video stream encoding information to the terminal so that the terminal can obtain the motion posture key frame of the virtual object; A road surface recognition model is generated by training, and the road surface recognition model is sent to the terminal so that the terminal can obtain road surface information; wherein the motion posture key frames of the virtual object and the road surface information are superimposed and displayed by the terminal; and the wearer of the terminal is controlled by the terminal to follow the movement of the virtual object.

7. The method according to claim 6, characterized in that The generating of the motion model of the virtual object comprises: generating a movement route and a movement frequency of the virtual object; A motion model of the virtual object is generated based on the motion route and motion frequency of the virtual object.

8. The method according to claim 7, characterized in that The generating the movement route of the virtual object comprises: Get geographic location information sent by external sensors; Obtaining the user's physical information and historical movement information sent by the second cloud node; Acquiring user physiological information and road image information sent by the terminal; A movement route of the virtual object is generated based on the geographic location association information, the user body information, the historical movement information, the user physiological information and the road image information.

9. The method according to claim 7, characterized in that: The generating the movement frequency of the virtual object comprises: Sending a first request to the terminal; the first request is used to request to obtain user motion data; Based on the first request, acquiring the user motion data sent by the terminal; Based on the acquired user motion data, a motion frequency of the virtual object is generated.

10. The method according to claim 6, characterized in that The step of rendering and generating motion posture key frames of the virtual object based on the motion model of the virtual object comprises: Obtain user physiological information and standard motion posture model; The motion model of the virtual object, the user's physiological information, and the standard motion posture model are combined to render and generate motion posture key frames of the virtual object.

11. The method according to claim 6, characterized in that The training and generating of the road surface recognition model comprises: Acquire road image information uploaded by the terminal; The acquired road image information is matched and trained with a trained road surface recognition model to obtain a new trained road surface recognition model.

12. The method according to claim 6, characterized in that The method further comprises: Receiving the geographical scene picture sent by the terminal and the local geographical location coordinates of the terminal; Based on the geographical scene image, the local geographical location coordinates of the terminal and a map information library, the route information to be selected is determined; the route information to be selected includes at least the direction and key positions of the movement route of the virtual object.

13. A motion control device, characterized in that: Applied to a terminal, the device comprises: A first receiving unit, configured to receive an operation instruction initiated by a wearer of the terminal; A decoding unit, configured to decode the video stream encoding information received from the first cloud node in response to the operation instruction, to obtain a motion posture key frame of the virtual object; the motion posture key frame of the virtual object is generated by the first cloud node based on motion model rendering; A second receiving unit, used to receive the road surface recognition model sent by the first cloud node; An identification unit, used to identify the real-time road surface image captured based on the road surface identification model to obtain corresponding road surface information; A display unit, used for superimposing and displaying the road surface information and the motion posture key frame of the virtual object; A control unit is used to control the wearer of the terminal to move following the virtual object.

14. A motion control device, characterized in that: Applied to a first cloud node, the device comprises: A first generating unit, configured to generate a motion model of a virtual object; A second generating unit, configured to render and generate motion posture key frames of the virtual object based on the motion model of the virtual object; An encoding unit, used for encoding the motion posture key frames of the virtual object to obtain corresponding video stream encoding information; A first sending unit, configured to send the video stream encoding information to a terminal, so that the terminal can obtain a motion posture key frame of the virtual object; A third generating unit, used for training and generating a road surface recognition model; The second sending unit is used to send the road surface recognition model to the terminal so that the terminal can obtain road surface information; wherein the motion posture key frame of the virtual object and the road surface information are superimposed and displayed by the terminal; and the wearer of the terminal is controlled by the terminal to follow the virtual object to move.

15. A terminal, characterized in that: The terminal comprises: a first processor and a first memory for storing a computer program that can be run on the first processor; Wherein, when the first processor is used to run the computer program, the steps of the method described in any one of claims 1 to 5 are executed.

16. A first cloud node device, characterized in that: The first cloud node device comprises: a second processor and a second memory for storing a computer program that can be run on the second processor; Wherein, when the second processor is used to run the computer program, the steps of the method described in any one of claims 6 to 12 are executed.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 5, or implements the steps of the method according to any one of claims 6 to 12.

Images