Method, device, electronic device and storage medium for generating user motion trajectory

Abstract

The present invention provides a method, device, electronic device and storage medium for generating a user motion trajectory, belonging to the field of computer technology, the method comprising: generating static three-dimensional identity information based on an acquired user image; generating dynamic three-dimensional identity information based on the static three-dimensional identity information, the dynamic three-dimensional identity information being information describing a deformable three-dimensional human body model of the user; determining a three-dimensional trajectory route of a user's sampling point, and fusing the three-dimensional trajectory route with a preset map to generate a three-dimensional trajectory map; fusing the dynamic three-dimensional identity information with the three-dimensional trajectory route to generate a motion trajectory frame sequence, and fusing the motion trajectory frame sequence with the three-dimensional trajectory map to generate a user motion trajectory video. The user motion trajectory video of the present invention can more intuitively and vividly express the user's real image, and can obtain a better personalized image and personalized experience.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, electronic device and storage medium for generating a user motion trajectory. Background Art

[0002] As people's lives become more diversified, more and more people rely on electronic devices to record and share their life trajectories, such as running, cycling, etc. In the application of electronic devices, you can see an arrow or a dot representing the user on a flat map, and a line segment is used to record the user's running trajectory from the starting point to the end point, generating a trajectory animation video that can record the user's previous exercise process.

[0003] The user may be marked as a dot, an arrow, a 2D figure, or a 3D figure on the application, from one dimension to two dimensions and then to three dimensions, so that the user can more intuitively perceive the activity trajectory that he has moved. However, these user marks cannot express the user more realistically and vividly, and still need some auxiliary marks to more fully express the user. At the same time, it is not intuitive enough. In addition, the existing technology for displaying the user's movement trajectory is generally a recording type, without a real-time live broadcast function, so it lacks a personalized image and personalized experience. Summary of the invention

[0004] The present invention provides a method, device, electronic device and storage medium for generating a user motion trajectory, so as to solve the problem in the prior art that the user's existing marks cannot better express the user's life trajectory and the personalized experience is insufficient.

[0005] The present invention provides a method for generating a user motion trajectory, the method comprising:

[0006] generating static three-dimensional identity information based on the acquired user image;

[0007] Generating dynamic three-dimensional identity information according to the static three-dimensional identity information, wherein the dynamic three-dimensional identity information is information describing a deformable three-dimensional human body model of the user;

[0008] Determine a three-dimensional trajectory route of a user's sampling point, and merge the three-dimensional trajectory route with a preset map to generate a three-dimensional trajectory map;

[0009] The dynamic three-dimensional identity information is fused with the three-dimensional trajectory route to generate a motion trajectory frame sequence, and the motion trajectory frame sequence is fused with the three-dimensional trajectory map to generate a user motion trajectory video.

[0010] According to a method for generating a user motion trajectory provided by the present invention, after generating a user motion trajectory video, the method further includes:

[0011] Obtaining the user's motion trajectory video;

[0012] Rendering the user motion trajectory video, wherein the rendering includes adding one or more combinations of materials, effects, subtitles, icons, and music;

[0013] Display the rendered user motion trajectory video on the preset application.

[0014] According to a method for generating a user motion trajectory provided by the present invention, the rendering process of the user motion trajectory video includes:

[0015] Recalling one or more pre-stored static three-dimensional identity information;

[0016] Based on the pre-stored one or more static three-dimensional identity information, and according to calling the static three-dimensional identity information generated by the user image, a corresponding user motion trajectory video is generated.

[0017] According to a method for generating a user motion trajectory provided by the present invention, the rendering process of the user motion trajectory video includes:

[0018] Performing personalized modification on the static three-dimensional identity information;

[0019] Generate the corresponding user motion trajectory video based on the personalized modified static three-dimensional identity information.

[0020] According to a method for generating a user motion trajectory provided by the present invention, after generating a user motion trajectory video, the method further includes:

[0021] Live broadcast of the user's motion trajectory video in real time; or

[0022] The user's motion trajectory video is recorded and played.

[0023] According to a method for generating a user motion trajectory provided by the present invention, recording and playing the user motion trajectory video includes:

[0024] One or more combined operations of editing, forwarding, storing, and replaying are performed on the user motion trajectory video.

[0025] According to a method for generating a user motion trajectory provided by the present invention, the user image includes a first image and a second image, and the step of generating static three-dimensional identity information based on the acquired user image includes:

[0026] Acquire a first image and a second image of a user, wherein the first image includes user feature information, and the second image includes three-dimensional information corresponding to the first image;

[0027] Extracting user features from the first image and the second image respectively to generate corresponding user texture information and user three-dimensional information;

[0028] The user texture information is mapped to the user three-dimensional information to generate the static three-dimensional identity information.

[0029] According to a method for generating a user motion trajectory provided by the present invention, the user texture information is a combination of one or more of facial features, head features, and body features.

[0030] According to a method for generating a user motion trajectory provided by the present invention, the user three-dimensional information is one or more combinations of facial features, head features, and body features.

[0031] According to a method for generating a user motion trajectory provided by the present invention, the second image is acquired through any one of a 3D TOF camera, a 3D structured light camera, a binocular stereo vision camera, a coded structured light camera, and a three-dimensional scanner.

[0032] According to a method for generating a user motion trajectory provided by the present invention, the user image is one or more combinations of a real face model, a real head model, a real body model, and an edited real model of the user pre-stored in the system.

[0033] According to a method for generating a user motion trajectory provided by the present invention, the step of generating dynamic three-dimensional identity information according to the static three-dimensional identity information comprises:

[0034] The static three-dimensional identity information is subjected to displacement parameter adjustment to generate a three-dimensional human body model, wherein the displacement parameter is adjusted to a parameter value moving from a first coordinate system to a second coordinate system;

[0035] The three-dimensional human body model is subjected to motion parameter adjustment to generate the dynamic three-dimensional identity information, wherein the motion parameters include one or more combinations of parameters of center of mass motion, parameters of upper body motion, and parameters of lower body motion.

[0036] According to a method for generating a user motion trajectory provided by the present invention, the step of determining a three-dimensional trajectory route of a user's sampling point comprises:

[0037] The three-dimensional trajectory route is generated by system preset settings according to user needs, and the three-dimensional trajectory route describes the trajectory route that the user wants to move; or the three-dimensional trajectory route is generated according to the user's real-time movement, and the three-dimensional trajectory route describes the trajectory route generated by the user's real-time movement.

[0038] According to a method for generating a user motion trajectory provided by the present invention, the steps of determining a three-dimensional trajectory route of a user's sampling point and fusing the three-dimensional trajectory route with a preset map to generate a three-dimensional trajectory map include:

[0039] Obtaining geographic information of user sampling points, sequentially connecting the geographic information of the sampling points to generate a three-dimensional trajectory route, and marking the starting point and the end point of the three-dimensional trajectory route;

[0040] Mapping the data of all sampling points onto the preset map, and drawing a line from the starting point to the next adjacent point according to a preset time rhythm until the line reaches the end point;

[0041] The coordinate system of the sampling point and the coordinate system of the preset map are combined into the same world coordinate system to achieve the fusion of the three-dimensional trajectory route and the preset map.

[0042] According to a method for generating a user motion trajectory provided by the present invention, the step of fusing the dynamic three-dimensional identity information with the three-dimensional trajectory route to generate a motion trajectory frame sequence comprises:

[0043] Mapping the dynamic three-dimensional identity information to the sampling points to create real-life character image information of the user;

[0044] The real person image information is integrated with the three-dimensional trajectory route to generate the motion trajectory frame sequence.

[0045] According to a method for generating a user motion trajectory provided by the present invention, the step of fusing the motion trajectory frame sequence with the three-dimensional trajectory map to generate a user motion trajectory video comprises:

[0046] The motion trajectory frame sequence is synthesized with the three-dimensional trajectory map, wherein the synthesis process includes binding the coordinate data, motion data, and physical data between the user's real character image information and the three-dimensional trajectory map;

[0047] The real person image information and the three-dimensional trajectory map are scaled according to a preset ratio to ensure that the frame sequence of the real person image information is matched and adjusted with the time sequence of the three-dimensional trajectory map.

[0048] The present invention also provides a device for generating a user motion trajectory, the device comprising:

[0049] A user static identity information generation module, used to generate static three-dimensional identity information based on the acquired user image;

[0050] A user dynamic identity information generation module, used to generate dynamic three-dimensional identity information according to the static three-dimensional identity information, wherein the dynamic three-dimensional identity information is information describing a three-dimensional human body model of the user;

[0051] A three-dimensional trajectory map generation module is used to determine the three-dimensional trajectory route of the user's sampling point, and merge the three-dimensional trajectory route with a preset map to generate a three-dimensional trajectory map;

[0052] The user motion trajectory video generation module is used to fuse the dynamic three-dimensional identity information with the three-dimensional trajectory route to generate a motion trajectory frame sequence, and fuse the motion trajectory frame sequence with the three-dimensional trajectory map to generate a user motion trajectory video.

[0053] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned methods for generating a user motion trajectory are implemented.

[0054] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned methods for generating a user motion trajectory are implemented.

[0055] The present invention provides a method, device, electronic device and storage medium for generating a user motion trajectory, which obtains a user image, generates dynamic three-dimensional identity information of an augmented reality character phenomenon after processing the user image, generates a three-dimensional trajectory map according to the user's three-dimensional trajectory route, and generates a user motion trajectory video after the dynamic three-dimensional identity information and the three-dimensional trajectory map are merged. The user motion trajectory video can express the user's real image more intuitively and vividly, and can obtain a better personalized image and personalized experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A schematic diagram of the flow of a method for generating a user motion trajectory provided by the present invention;

[0058] Figure 2 A schematic diagram of a user motion trajectory video provided by the present invention;

[0059] Figure 3 A schematic diagram of a process for generating static three-dimensional identity information provided by the present invention;

[0060] Figure 4 A schematic diagram of a process for generating dynamic three-dimensional identity information provided by the present invention;

[0061] Figure 5 A schematic diagram of a skeleton tree provided by the present invention;

[0062] Figure 6 A schematic diagram of a process for generating a three-dimensional trajectory map provided by the present invention;

[0063] Figure 7 A schematic diagram of the process of generating a user motion trajectory video provided by the present invention;

[0064] Figure 8 A structural block diagram of a device for generating a user motion trajectory provided by the present invention;

[0065] Fig. 9 This is a structural block diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0068] Combine the following Figure 1-Figure 9 The invention describes a method, device, electronic device and storage medium for generating a user motion trajectory.

[0069] Figure 1 A schematic diagram of the flow chart of the method for generating a user motion trajectory provided by the present invention is shown in FIG. Figure 1 A method for generating a user motion trajectory, the method comprising:

[0070] Step 101: Generate static three-dimensional identity information based on the acquired user image.

[0071] Optionally, the user image can be obtained by real-time shooting with a camera, or by acquiring an existing user image. The user image can be an image of the user himself, which is more authentic. The real image of the user can be directly presented based on the user image without adding more logos, which is more personalized.

[0072] It is understandable that the user image described in the present invention is not limited to the real image of the user himself, and can also be other images, such as cartoon images, landscape images, etc., and the present invention does not make any limitation here.

[0073] Optionally, the static three-dimensional identity information can be generated in real time, or it can be one or more pre-stored static three-dimensional identity information, and the generated static three-dimensional identity information can be edited or otherwise personalized. For example, the static model corresponding to the static three-dimensional identity information can be modified, including but not limited to adding eyebrows, adding beards, wearing glasses, and other personalized decorations.

[0074] It should be noted that the purpose of generating the static three-dimensional identity information in step 101 is to ensure the integrity of the three-dimensional information of the user image and to better display the user's true personal image. It is understandable that the static three-dimensional identity information of the present invention can be generated in real time or can be static three-dimensional identity information pre-stored by the system, and the present invention is not limited thereto.

[0075] Step 102: Generate dynamic three-dimensional identity information according to the static three-dimensional identity information, wherein the dynamic three-dimensional identity information is information describing a deformable three-dimensional human body model of the user.

[0076] It should be noted that the static three-dimensional identity information ensures the integrity of the segmented three-dimensional information of the user, and then the dynamic three-dimensional identity information is generated based on the static three-dimensional identity information. The purpose of generating dynamic three-dimensional identity information is to enable the three-dimensional human body model described by the dynamic three-dimensional identity information to perform deformable movements, and the movement of the three-dimensional human body model includes one or more combinations of center of mass movement, upper body movement, and lower body movement. Among them, the center of mass motion theorem is one of the universal theorems of dynamics, which will not be described in detail here.

[0077] Step 103: determine the three-dimensional trajectory route of the user's sampling point, and merge the three-dimensional trajectory route with the preset map to generate a three-dimensional trajectory map.

[0078] Optionally, the three-dimensional trajectory route can be generated by a system preset setting according to user needs, and the three-dimensional trajectory route describes the trajectory route that the user wants to move. For example, when using the present invention, the user can pre-set the three-dimensional trajectory route that the user wants to move on the mobile phone app, and the user can also set the three-dimensional trajectory route that the user wants to move into a three-dimensional trajectory route of various shapes, such as a bull shape, a duck shape, etc., to meet the personalized needs of the user.

[0079] Optionally, the three-dimensional trajectory route may also be generated according to the real-time movement of the user, and the three-dimensional trajectory route describes the trajectory route generated by the real-time movement of the user.

[0080] Optionally, the user's route activity track from the starting point to the end point is connected to generate a three-dimensional track route through the geographic information of the sampling points according to the time sequence and the change of the spatial position. The three-dimensional track route reflects the actual activity track of the user.

[0081] Optionally, the three-dimensional trajectory route is integrated with a preset map. Specifically, the data of all the sampling points of the user can be mapped to the preset map, and the coordinate system of the sampling points and the coordinate system of the preset map are unified into the same world coordinate system to ensure the integration of the three-dimensional trajectory route and the preset map, so that the user's three-dimensional trajectory route can be presented in the preset map.

[0082] Step 104: fuse the dynamic three-dimensional identity information with the three-dimensional trajectory route to generate a motion trajectory frame sequence, and fuse the motion trajectory frame sequence with the three-dimensional trajectory map to generate a user motion trajectory video.

[0083] Optionally, a normal walking speed reference value of the augmented reality character image presented by the dynamic three-dimensional identity information can be set first, so that the moving speed of different motion types combined with the three-dimensional trajectory route is a positively correlated linear relationship with the normal walking speed reference value, so that a motion trajectory frame sequence can be obtained.

[0084] Optionally, the motion trajectory frame sequence is fused with the three-dimensional trajectory map to generate a user motion trajectory video. Specifically, the coordinate data, motion data, and physical data between the user's augmented reality character image and the three-dimensional trajectory map can be bound to ensure that the real character image information is matched and adjusted with the three-dimensional trajectory map.

[0085] It should be noted that the present invention generates a motion trajectory frame sequence by integrating the augmented reality character image presented by the dynamic three-dimensional identity information with the three-dimensional motion trajectory route, and then integrates the motion trajectory frame sequence with the three-dimensional trajectory map to generate a user motion trajectory video, which can effectively realize the user's augmented reality character image, make the three-dimensional motion trajectory and the three-dimensional trajectory map seamlessly connected, have a stronger sense of reality, a more personalized effect, and a better personalized experience.

[0086] In an embodiment of the present invention, the method for generating a user motion trajectory further includes the following steps:

[0087] The generated user motion trajectory video can be rendered and displayed on a preset application (such as Figure 2 shown). Figure 2 The map shown is a three-dimensional map, with a three-dimensional character image of the user in the middle and a darkened black line at the back being a three-dimensional motion trajectory of the user, which is a trajectory connecting the starting point to the current position.

[0088] Optionally, in order to provide a better personalized experience for the user motion trajectory video, rendering processing may be performed based on the generated user motion trajectory video, and the rendering processing includes adding one or more combinations of materials, effects, subtitles, icons, and music.

[0089] Specifically, after the user's motion trajectory video is generated, you can add effects, subtitles, icons, music, control logos, etc. to obtain more materials for display. If you add subtitles, the total mileage, time, average speed, altitude, climbing speed, descending speed, maximum elevation difference, and even the user's heart rate can all be displayed.

[0090] Furthermore, the rendering process of the user motion trajectory video includes:

[0091] One or more pre-stored static three-dimensional identity information is called, and based on the pre-stored one or more static three-dimensional identity information, a corresponding user motion trajectory video is generated according to the static three-dimensional identity information generated by calling the user image.

[0092] For example, for the generated user motion trajectory video, the user can also call to switch to generate user motion trajectory videos of other types of real-person models, that is, different static three-dimensional identity information can be pre-stored, and then the static three-dimensional identity information generated by the user image can be called to synthesize a new real-person model. For example: a new real-person model can be generated by combining "real face + virtual model", or a new real-person model can be generated by combining "real head + virtual model". The real-person model of the generated user motion trajectory video is replaced by the new real-person model to generate the corresponding user motion trajectory video, thereby realizing the rendering processing of the user motion trajectory video.

[0093] Furthermore, the user's motion trajectory video can be displayed not only in real-time live broadcast, but also in offline display, such as video playback. Live broadcast means that the user can watch it in real time, invite friends to participate in the interaction, or transmit it to another display terminal through the network for more users to watch in real time. Offline display is the video segment file after the user's exercise, and the user can view, edit, store, forward, share, etc. on the video segment file.

[0094] It should be noted that whether the user renders or does not render the generated user motion trajectory video may be determined according to the user's personalized settings, and the display method of the user motion trajectory video may also be determined according to the user's personalized settings, and the present invention is not limited thereto.

[0095] The above-mentioned Figure 1 The specific implementation method of steps 101 to 104 in the embodiment is further described.

[0096] Figure 3 A schematic diagram of the process of generating static three-dimensional identity information provided by the present invention, such as Figure 3 In step 101, the user image includes a first image and a second image, and the step of generating static three-dimensional identity information based on the acquired user image includes:

[0097] Step 301: Acquire a first image and a second image of a user, wherein the first image includes user feature information, and the second image includes three-dimensional information corresponding to the first image.

[0098] Optionally, the first image and the second image of the user can be captured by a camera, wherein the first image can be a color image and the second image can be a depth image. A color image refers to an RGB image captured by a color camera, which is a color image. Each pixel has a color, and the color on the pixel represents the texture information of the user image. A depth image refers to the value in the Z direction that constitutes each pixel, similar to a grayscale image, using a grayscale value from 0 to 255 to represent the distance value of each pixel point. The depth information obtained by shooting can be obtained by any of 3D TOF cameras, 3D structured light cameras, binocular stereo vision cameras, coded structured light cameras, and three-dimensional scanners. The depth image is colorless, and each pixel represents an X, Y, Z point coordinate, and the point coordinates constitute the surface position information corresponding to each target in space.

[0099] Optionally, the first image includes user feature information, which may be feature information of a face, head, or whole body. The second image includes three-dimensional information corresponding to the first image of the user, which refers to the position information of the X, Y, and Z point coordinates contained in each pixel.

[0100] Optionally, the user image is acquired in real time, or may be one or more combinations of a real face model, a real head model, a real body model, and an edited real model of the user pre-stored in the system.

[0101] Step 302: extract user features from the first image and the second image respectively to generate corresponding user texture information and user three-dimensional information.

[0102] Optionally, the user texture information may be one or more combinations of facial features, head features, and body features; the user three-dimensional information may be one or more combinations of facial features, head features, and body features.

[0103] Among them, the user texture information refers to the RGB image (color image) taken by a color camera, where each pixel has color, representing the texture information of the object; the user three-dimensional information refers to the position information of the xyz point coordinates contained in each pixel.

[0104] Optionally, at least one frame of image is required to extract user features from the first image. If one frame of image is captured at a fixed preset viewing angle, contour information of user features in the frame can be detected based on the captured image, and user texture information in the contour information can be extracted to obtain initial user texture information.

[0105] Furthermore, if several frames of images are captured at a fixed preset viewing angle, contour information of user features between frames can be detected based on the captured images, and the contour information can be extracted, and then the extracted texture information can be spliced ​​to obtain more realistic user texture information.

[0106] Optionally, user features are extracted from the second image. If a frame of image is captured at a fixed preset viewing angle, contour information of user features in the frame can be detected, and user depth information in the contour information can be segmented to obtain initial three-dimensional user information.

[0107] Furthermore, if several frames of images are captured at a fixed viewing angle, contour information of user features between frames can be detected based on the captured images, and the contour information can be segmented, and then the segmented contour information can be spliced ​​to obtain more realistic three-dimensional information of the user.

[0108] Optionally, several frames of images may be captured at different preset viewing angles to obtain user texture information and user three-dimensional information at a larger viewing angle, thereby achieving more realistic user texture information and user three-dimensional information.

[0109] It can be seen from this that the user texture information and the user three-dimensional information can be generated in a suitable manner according to the needs of the actual effect, such as a method of shooting a frame of image at a fixed preset angle, or a method of shooting multiple frames of image at a fixed preset angle, or a method of shooting multiple frames of image at different preset angles. The present invention does not limit the specific method.

[0110] Step 303: Map the user texture information to the user three-dimensional information to generate the static three-dimensional identity information.

[0111] Optionally, the user static identity information refers to a computer virtual character image with a three-dimensional model, which is static.

[0112] Optionally, after segmentation, the user's 3D information is subjected to 3D modeling processing to obtain a 3D model, and then the user's texture information is mapped onto the user's 3D model. At the same time, corresponding filling processing is performed according to the integrity of the user's 3D information and the category of the user's texture information to generate the user's static 3D identity information.

[0113] Specifically, after segmenting the user three-dimensional information, the steps of determining whether the user three-dimensional information is completed are as follows:

[0114] Step 1: determine whether the user's 3D information is complete after segmentation. If incomplete, fill in the incomplete user's 3D information by identifying the user's texture information and determining the specific category (for example, determining whether the specific category is a face, a head, or a body, etc.).

[0115] It should be noted that the purpose of judging whether the user's three-dimensional information is complete after segmentation is because data loss, the so-called holes, may occur during the acquisition of the three-dimensional signal. If data is missing, the three-dimensional model will be distorted, so it is necessary to determine whether the point cloud data of this pixel is lost based on the relationship between the point clouds of the same frame image, or the relationship between the adjacent upper frame images (time and space, etc.), or the synchronized image of another RGB camera, and then fill in the missing data to restore the point cloud in the hole. Among them, filling the holes of the three-dimensional point cloud before three-dimensional modeling is a basic step, which will not be described in detail here.

[0116] Step 2: After the user's three-dimensional information is complete, three-dimensional modeling can be performed to obtain the user's initial three-dimensional model.

[0117] Step three, map the user texture information onto the user's initial 3D model to generate the user's static 3D identity information.

[0118] It should be noted that the user static three-dimensional identity information includes the user's real identity information, and default pre-defined skin information can also be set. The real identity information refers to the information within the outline of the above-mentioned user image, and the non-real identity information refers to the information outside the outline of the above-mentioned user image, such as filled information. If the above-mentioned captured user image only contains the user's facial information, then all information except the face is non-real identity information; if the above-mentioned captured user image only contains head information, then all information except the head is non-real identity information; if the above-mentioned captured user image is full-body information, then there is no non-real identity information at this time.

[0119] Optionally, if non-real identity information needs to be set, for example, the skin information of the user image can be skinned, that is, the pre-defined skin information is adapted to the non-real identity information, overlapped with the non-real identity information, and spliced ​​with the real identity information.

[0120] Furthermore, the skin information may also include other defined skin information, which may be predefined or subsequently defined. By changing the skin information of the image, the user may have more personalized choices and a better personalized experience.

[0121] It should be noted that the above-mentioned skin information includes but is not limited to information such as eyebrows, hair, and clothes.

[0122] Figure 4 A schematic diagram of the process of generating dynamic three-dimensional identity information provided by the present invention, such as Figure 4 In step 102, the step of generating dynamic three-dimensional identity information according to the static three-dimensional identity information includes:

[0123] Step 401 , adjusting the static three-dimensional identity information by displacement parameters to generate a three-dimensional human body model, wherein the displacement parameters are adjusted to parameter values ​​for moving from a first coordinate system (ie, current coordinate system) to a second coordinate system (ie, next coordinate system).

[0124] In order to make the user image in the application look more realistic, it is necessary to give more activity parameters to the static three-dimensional identity information generated above, so as to make it more realistic.

[0125] Specifically, the step of generating the three-dimensional human body model comprises:

[0126] Step 1: skeletonize the static three-dimensional identity information to obtain a skeleton tree and active nodes. The skeleton tree divides the human body and limbs into different segments, and then connects them into a corresponding tree according to the active nodes (such as Figure 5 shown).

[0127] The skeleton tree is mainly composed of the head, torso, left arm, left forearm, right arm, right forearm, left thigh, left calf, right thigh, right calf and other basic parts. Each bone and node will change position according to the input local coordinate system, and the bone will change position with the parent bone. For example, when the torso rotates, the left arm will rotate with it, and the left forearm will also rotate with the left arm.

[0128] Step 2: Mark the relationships of the skeleton tree from top to bottom, from parent nodes to child nodes.

[0129] Optional, specific marking method, for example, if the torso is taken as the main body, the shoulders and arms are marked as the parent nodes, and the arms are marked as the child nodes; when the arms are marked as the parent nodes, the wrists are marked as the child nodes. That is, the child nodes move with the movement of the parent nodes, but the parent nodes may not move when the child nodes move.

[0130] Step three, assign transformation matrix parameters composed of translation components and rotation components to each active node to obtain a deformable three-dimensional human body model, and at the same time obtain a human body coordinate system and a local coordinate system of each joint.

[0131] The above-mentioned translation component is the moving position parameter of the active node on the x, y, and z axes of the corresponding local coordinate system, and is limited to a reasonable range according to actual conditions. When the value of the translation component is equal to the median value, it can be regarded as stationary.

[0132] Among them, the above-mentioned rotation component is the rotation quaternion parameter of the active node on the x, y, z, and w axes of the corresponding local coordinate system, and is limited to a reasonable range according to actual conditions. When the w value of the rotation component is equal to the median value, it can be regarded as no rotation.

[0133] Step 402, adjusting the motion parameters of the three-dimensional human body model to generate the dynamic three-dimensional identity information, wherein the motion parameters include one or more combinations of parameters of center of mass motion, parameters of upper body motion, and parameters of lower body motion.

[0134] For the human body model, the embodiment of the present invention can divide the whole body movement into center of mass movement, upper body movement and lower body movement. Since the movement of the human body is a relative movement between limbs, it can be regarded as a periodic, continuous, stable and coordinated movement. From the kinematic analysis, it can be seen that the center of mass trajectory of the human body in walking movement is approximately a smooth sine curve, but when specifically applied to the present invention, the present invention only needs to obtain the horizontal displacement and the vertical displacement. Based on this, the upper body movement and the lower body movement can be regarded as swinging, that is, each movement posture is a combination of the spatial angles between adjacent limbs.

[0135] Taking lower body movement as an example, the three most basic movements of walking, running and stepping are similar and can be simplified to two single steps equal to one total step, and the movement frequency f=2. The movement speed P is equal to the total step length SL divided by the movement frequency f. The time taken to complete a total step length SL (i.e., from one posture to the next identical posture) is the movement cycle TC. According to kinematic analysis, in straight-line walking, the swinging foot does not move at a uniform speed. The speed is the slowest at the beginning and end of the swing phase and the fastest in the middle time period. The smooth curve function that can be used to describe the swinging ankle movement trajectory by using a quadratic Bezier curve is as follows:

[0136]

[0137] Among them, u∈[0,1] represents the ratio of the control point to the curve, P0 and P2 are the starting point and end point of the curve, and P1 is the control point. The control point of the Bezier curve is obtained by inverting the coordinates of the starting point, end point and highest point during pedaling, and then the position coordinates of the ankle are obtained, that is, the parameters of the lower body movement are obtained.

[0138] In addition, due to the coordinated relationship between the swing of the hands and legs, the angle function of the forward and backward swing completed in one cycle is in is the forward and backward swing angle of the shoulder joint, h is the proportional factor, a is the forward and backward swing angle of the hip joint of the opposite side of the body, and θ is the offset, which are the parameters for upper body movement.

[0139] In summary, the dynamic three-dimensional identity information is generated by adjusting the motion parameters of the three-dimensional human body model, wherein the motion parameters can be obtained by the above method.

[0140] Figure 6 A schematic diagram of the process of generating a three-dimensional trajectory map provided by the present invention, such as Figure 6 In the above step 103, the step of determining the three-dimensional trajectory route of the user's sampling point and fusing the three-dimensional trajectory route with the preset map to generate a three-dimensional trajectory map includes:

[0141] Step 601, obtaining geographic information of user sampling points, sequentially connecting the geographic information of the sampling points to generate a three-dimensional trajectory route, and marking the starting point and the end point of the three-dimensional trajectory route; the specific steps include:

[0142] Step 1: Get the user's geographical information.

[0143] The embodiment of the present invention can communicate with one or more combinations of satellites, base stations, and other instruments through the device carried by the user (such as an electronic device, a mobile phone, etc.) to obtain the user's geographical information, which includes information such as longitude, latitude, or altitude.

[0144] Step 2: Establish a coordinate system with time and space.

[0145] When the user selects the start, the system records the start time, obtains the current geographic information as the geographic coordinate origin, establishes a coordinate system with time and space, and marks the coordinate origin as the starting point to obtain a starting coordinate (X0, Y0, Z0, T0); then, at intervals of a period of time or / and a longitude or / and latitude, more sampling point coordinates (Xn, Yn, Zn, Tn; where n=1, 2, ...k) are obtained in sequence. Once the user selects the end, the sampling point n=k is marked as the end coordinate and end time. Connect the n+1 coordinate points in sequence to obtain a three-dimensional trajectory route.

[0146] Step 3: Calculate the speed of any two sampling points.

[0147] The speed of any two sampling points can be calculated based on any two coordinates (Xn, Yn, Zn) and (Xn-1, Yn-1, Zn-1) to get the line distance Sn between the two sampling points, and then the speed Vn in the time period can be obtained by comparing it with the time difference between the two sampling points; finally, the total distance S of the trajectory can be obtained by adding up the line distances of all sampling points, and the total time t can be obtained by subtracting the start time from the end time; and the average speed v can be obtained by dividing the total distance S by the total time t.

[0148] Furthermore, sampling points can be obtained by time and space. The representation unit when time is used is seconds; when space is used, the representation unit defaults to meters or kilometers (or kilometers), so a spatial threshold is used to automatically switch the representation unit, that is, when the distance from the starting point to the end point is less than 1 kilometer, meters are used as units, otherwise kilometers are used as units. In order to reduce the huge data processing caused by the increase in sampling points, the sampling points can be reduced according to certain rules. Time sampling is very accurate, but there are large errors in spatial sampling, with small errors of tens of centimeters and large errors of tens of meters. Therefore, a flexible sampling strategy is needed. When the representation distance is less than ten meters, meter-level interval sampling is used. When the representation distance is less than 100 meters, ten-meter interval sampling is used. When the representation distance exceeds 100 meters, 100-meter interval sampling is used. In this way, a very accurate sampling data can be obtained.

[0149] Step 602, mapping the data of all sampling points onto the preset map, and drawing a line from the starting point to the next adjacent point according to a preset time rhythm until the line reaches the end point.

[0150] Specifically, after the start and end points are determined, the sampling points in the middle need to be supplemented and deleted according to the distance accuracy. If the distance accuracy is at the meter level, if there are adjacent points with a distance exceeding the meter level, the points are supplemented in combination with the relationship between the movement speed and the time interval; if the distance accuracy is at the ten-meter level, if there are adjacent points with a distance less than 10 meters, they are deleted, and if there are adjacent points with a distance exceeding ten meters, the points are supplemented in combination with the relationship between the movement speed and the time interval; if the distance accuracy is at the hundred-meter level, if there are adjacent points with a distance less than 100 meters, they are deleted, and if there are adjacent points with a distance exceeding one hundred meters, the points are supplemented in combination with the relationship between the movement speed and the time interval. This process is to optimize the redundant or missing sampling points, obtain equally spaced continuous adjacent points, and then connect all the sampling points from the start to the end point to obtain a three-dimensional trajectory route.

[0151] Step 603: The coordinate system of the sampling point and the coordinate system of the preset map are combined into the same world coordinate system to achieve the fusion of the three-dimensional trajectory route and the preset map.

[0152] Specifically, the data of all sampling points are mapped to a preset map, and a line is drawn from the starting point to the next adjacent point according to the time rhythm until the end point is reached, and then the coordinate system of the sampling point and the coordinate system of the preset map are unified into the same world coordinate system.

[0153] In summary, by unifying the coordinate system of the sampling point and the coordinate system of the preset map into the same world coordinate system, the fusion of the three-dimensional trajectory route and the preset map is achieved.

[0154] Figure 7 A schematic diagram of the process of generating a user motion trajectory video provided by the present invention, such as Figure 7 As shown, in the above step 104, the dynamic three-dimensional identity information is fused with the three-dimensional trajectory route to generate a motion trajectory frame sequence, and the motion trajectory frame sequence is fused with the three-dimensional trajectory map to generate a user motion trajectory video, including:

[0155] Step 701, mapping the dynamic three-dimensional identity information to the sampling points to create the user's real character image information.

[0156] Specifically, the centroid position of the user's 3D human model is mapped to the sampling point, and a relative offset is performed according to the body size of the 3D human model, and then the human coordinate system of the user's 3D human model is bound to the world coordinate system to create the user's real character image information. Among them, the human body is perpendicular to the ground surface, and the direction of the face is consistent with the direction of the line connecting the adjacent points.

[0157] It should be noted that the above-mentioned relative offset needs to be performed according to the body size of the three-dimensional human body model because if the center of mass of the human body model is positioned at the waist, and the point center of mass of the motion trajectory is at the heel, then there will be a center of mass that does not overlap. If it is forced to overlap, the lower body of the human body model will be sunk to the ground, which looks unreal. The sole of the human body model needs to be on the ground surface, and the human body needs to be away from the vertical ground wall surface. Therefore, the embodiment of the present invention is to set an offset parameter to change the intersection with different surfaces to achieve a more realistic creation of the user's real character image.

[0158] Step 702: Fusing the real person image information with the three-dimensional trajectory route to generate the motion trajectory frame sequence.

[0159] Specifically, assuming that the normal walking speed of a normal person is about 1.2 meters per second, the normal running speed is about 3 times the walking speed, and the normal cycling speed is about 4 times the walking speed. The speed of going uphill is half the walking speed, and the speed of going downhill is twice the walking speed.

[0160] Assuming that the normal walking speed of a normal person is used as a reference, the normal walking movement frequency of the user's augmented reality character image defaults to 4 small steps / second. Other different types of movement can be combined with the moving speed of the three-dimensional trajectory route, which is a positively correlated linear relationship with the normal walking movement frequency. For example, if the running movement speed is 4 times the normal walking speed, the running movement frequency is equal to 16 small steps / second, thereby obtaining a motion trajectory frame sequence.

[0161] It should be noted that the step frequency and linear relationship coefficient of any running type can be updated through static configuration, which will not be described in detail here.

[0162] Step 703, synthesizing the motion trajectory frame sequence with the three-dimensional trajectory map, wherein the synthesis includes binding the coordinate data, motion data, and physical data between the user's real character image information and the three-dimensional trajectory map.

[0163] Specifically, the motion trajectory frame sequence and the three-dimensional trajectory map are synthesized, including binding the coordinate data, motion data, physical data, etc. between the user's augmented reality character image information and the three-dimensional trajectory map, to ensure that the user's augmented reality character image stands on the ground and does not interfere.

[0164] Step 704: scaling the real person image information and the three-dimensional trajectory map according to a preset ratio to ensure that the frame sequence of the real person image information is matched and adjusted with the time sequence of the three-dimensional trajectory map.

[0165] Specifically, in terms of space, the user's augmented reality character image information and the three-dimensional trajectory map are scaled according to a certain size ratio, and a linkage relationship is generated when the map is scaled, and it increases when it increases and decreases when it decreases. In terms of time, the frame sequence of the user's augmented reality character image information is matched and adjusted with the time sequence of the three-dimensional trajectory map.

[0166] It should be noted that the spatial parameters are configured because the map can be enlarged or reduced. If the face model remains unchanged, it will produce a relatively large sense of distortion and disharmony. Therefore, through the spatial parameter configuration function, the appropriate ratio can be set for random following, and the user can also set the zoom-in or zoom-out ratio by himself.

[0167] Similar to the configuration of spatial parameters, the configuration of time parameters is because in the trajectory video, if it is a real-time live user motion trajectory video, its duration cannot be predicted. Therefore, in real time, a suitable ratio can be set to perform random follow-up motion swings to maintain a rhythmic sense of activity. However, when the user motion trajectory video ends, the duration is fixed. Some will be very long, and some will be very short. If all are played normally, there will be a relatively large sense of distortion and disobedience. Therefore, the user will be allowed to control the time to a shorter time (such as tens of seconds, or a few minutes, etc.) to represent the original actual time. In this way, the number of compressed trajectories will increase and the motion swing will accelerate; therefore, it is necessary to configure the time parameters to reduce the acquisition points and swing to a reasonable state.

[0168] The above-mentioned spatial parameters and time parameters can be updated through static configuration, which will not be described in detail here.

[0169] The device for generating a user motion trajectory provided by the present invention is described below. The device for generating a user motion trajectory described below and the method for generating a user motion trajectory described above can refer to each other.

[0170] Figure 8 The structural block diagram of the device for generating the user motion trajectory provided by the present invention is as follows: Figure 8 A device 800 for generating a user motion trajectory includes a user static identity information generating module 801, a user dynamic identity information generating module 802, a three-dimensional trajectory map generating module 803, and a user motion trajectory video generating module 804.

[0171] The user static identity information generating module 801 is used to generate static three-dimensional identity information based on the acquired user image.

[0172] Specifically, the static identity information generation module 801 is used to obtain a first image and a second image of a user, wherein the first image includes user feature information, and the second image includes three-dimensional information corresponding to the first image, and the user image includes the first image and the second image; extract user features from the first image and the second image respectively to generate corresponding user texture information and user three-dimensional information; map the user texture information to the user three-dimensional information to generate the static three-dimensional identity information.

[0173] Furthermore, the user static identity information generation module 801 is used to obtain the user static identity information after photographing the user's identity image information and processing it. The user static identity information is an augmented reality character image information obtained by synthesizing the user's real identity information and the identity information of the virtual character. Among them, the user's own real identity information includes at least one or more of the face, head, and body, so that people can clearly distinguish who the augmented reality character image is from a sense, without adding other identifications to supplement it. The user's real identity information can be collected by photographing the user himself to obtain a color image and a depth map, and then the color image and the depth map are processed separately to obtain the user's augmented reality three-dimensional model information, and then merged with the three-dimensional model information of the virtual character to obtain a combination of real character information and virtual character information.

[0174] The user dynamic identity information generating module 802 is used to generate dynamic three-dimensional identity information according to the static three-dimensional identity information, where the dynamic three-dimensional identity information is information describing the three-dimensional human body model of the user.

[0175] Specifically, the user dynamic identity information generation module 802 is used to adjust the static three-dimensional identity information by displacement parameters to generate a three-dimensional human body model, wherein the displacement parameters are adjusted to parameter values ​​for moving from a first coordinate system to a second coordinate system; and to adjust the motion parameters of the three-dimensional human body model to generate the dynamic three-dimensional identity information, wherein the motion parameters include one or more combinations of parameters for center of mass motion, parameters for upper body motion, and parameters for lower body motion.

[0176] Furthermore, the user dynamic identity information generation module 802 is used to process the user's augmented reality character image with motion parameter data to obtain a motion mechanism with a real person. By skeletalizing the user's augmented reality character image information according to character kinematics, a skeleton tree mainly composed of the head, body trunk, left arm, left forearm, right arm, right forearm, left thigh, left calf, right thigh, right calf, etc. is obtained, and then the bones and nodes are marked, and a coordinate system related to the human body is established. Finally, the corresponding motion parameters are determined according to the connection between them, so as to obtain an augmented reality character image of the user with motion ability.

[0177] The three-dimensional trajectory map generation module 803 is used to determine the three-dimensional trajectory route of the user's sampling point, and merge the three-dimensional trajectory route with the preset map to generate a three-dimensional trajectory map.

[0178] Specifically, the three-dimensional trajectory map generation module 803 is used to obtain the geographic information of the user's sampling points, connect the geographic information of the sampling points in sequence to generate a three-dimensional trajectory route, and mark the starting point and the end point of the three-dimensional trajectory route; map the data of all sampling points to the preset map, and draw a line from the starting point to the next adjacent point according to a preset time beat until the line reaches the end point; and make the coordinate system of the sampling points and the coordinate system of the preset map the same world coordinate system to achieve the fusion of the three-dimensional trajectory route and the preset map.

[0179] Furthermore, the three-dimensional trajectory map generation module 803 is used to obtain a three-dimensional trajectory route after simultaneously collecting and processing the time information and space information of the user during the exercise. When the user is exercising, the electronic device carried by the user can collect time information and space information at the same time, complete the collection of the user's points from the starting point to the end point and the adjacent points in the middle according to a certain strategy, and delete the redundant sampling points, or fill in the missing sampling points, so that each sampling point has time and space information and is equivalent to the movement of the user; finally, these collection points are connected in time order to obtain a three-dimensional trajectory route.

[0180] Furthermore, the three-dimensional trajectory map generation module 803 is used to synthesize the three-dimensional trajectory line and the user identity information to obtain the user's motion trajectory. The user's static or dynamic identity information is mapped onto the three-dimensional trajectory line, and the human body coordinate system is bound to the world coordinates of the three-dimensional trajectory line, and the front direction of the human body is ensured to be consistent with the direction of the three-dimensional trajectory line; in addition, the human body's limb movement parameters are determined to ensure that the human body's hands and legs swing in a coordinated and rhythmic manner during the movement along the three-dimensional trajectory line.

[0181] Furthermore, the three-dimensional trajectory map generation module 803 is used to obtain a three-dimensional trajectory map after synthesizing the three-dimensional trajectory line with the predetermined map; map the three-dimensional trajectory line to the world coordinate system of the predetermined map for binding, and draw lines for sampling points of the three-dimensional trajectory line from the starting point to the end point in time sequence.

[0182] The user motion trajectory video generation module 804 is used to fuse the dynamic 3D identity information with the 3D trajectory route to generate a motion trajectory frame sequence, and fuse the motion trajectory frame sequence with the 3D trajectory map to generate a user motion trajectory video.

[0183] Specifically, the user motion trajectory video generation module 804 is used to map the dynamic three-dimensional identity information to the sampling points to create the user's real character image information; merge the real character image information with the three-dimensional trajectory route to generate the motion trajectory frame sequence; synthesize the motion trajectory frame sequence with the three-dimensional trajectory map, and the synthesis processing includes binding the coordinate data, motion data, and physical data between the user's real character image information and the three-dimensional trajectory map; scale the real character image information and the three-dimensional trajectory map according to a preset ratio to ensure that the frame sequence of the real character image information is matched and adjusted with the timing of the three-dimensional trajectory map.

[0184] Furthermore, the user motion trajectory video generation module 804 is used to obtain the user trajectory video after fusing the user's augmented reality character image with the three-dimensional trajectory map. The coordinate parameters, motion parameters, physical parameters, etc. of the user's augmented reality character image are bound to the three-dimensional trajectory map so that the human body does not interfere with the map, and the video data is generated according to a certain time and space relationship.

[0185] In one embodiment of the present invention, the device for generating a user motion trajectory of the present invention further includes a user trajectory video display module.

[0186] The user trajectory video display module is used to obtain the user motion trajectory video; render the user motion trajectory video, and the rendering process includes adding one or more combinations of materials, effects, subtitles, icons, and music; and display the rendered user motion trajectory video on a preset application.

[0187] Specifically, the user trajectory video display module is used to render the user's trajectory video, and add more content including but not limited to text, pictures, expressions, music, effects, etc. according to actual conditions, so that the user's trajectory video is vividly and vividly displayed.

[0188] It can be seen from this that the present invention generates a motion trajectory frame sequence by integrating the augmented reality character image presented by the dynamic three-dimensional identity information with the three-dimensional motion trajectory route, and then integrates the motion trajectory frame sequence with the three-dimensional trajectory map to generate a user motion trajectory video, which can effectively achieve seamless connection between the user's augmented reality character image, three-dimensional motion trajectory and three-dimensional trajectory map, with stronger sense of reality, more personalized effect and better personalized experience.

[0189] Fig. 9 An example of a structural diagram of an electronic device is shown in FIG. Fig. 9As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940, and the memory 930 is used to store the user's identity information, trajectory information, map information and other related motion video data. The processor 910 can call the logic instructions in the memory 930, and the processor 910 is used to process the user's identity information, trajectory information, map information and other related motion video data, including but not limited to executing a method for generating a user's motion trajectory, the method comprising:

[0190] generating static three-dimensional identity information based on the acquired user image;

[0191] Generating dynamic three-dimensional identity information according to the static three-dimensional identity information, wherein the dynamic three-dimensional identity information is information describing a deformable three-dimensional human body model of the user;

[0192] Determine a three-dimensional trajectory route of a user's sampling point, and merge the three-dimensional trajectory route with a preset map to generate a three-dimensional trajectory map;

[0193] The dynamic three-dimensional identity information is fused with the three-dimensional trajectory route to generate a motion trajectory frame sequence, and the motion trajectory frame sequence is fused with the three-dimensional trajectory map to generate a user motion trajectory video.

[0194] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0195] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, the computer can execute the method for generating a user motion trajectory provided by the above methods, the method comprising:

[0196] generating static three-dimensional identity information based on the acquired user image;

[0197] Generating dynamic three-dimensional identity information according to the static three-dimensional identity information, wherein the dynamic three-dimensional identity information is information describing a deformable three-dimensional human body model of the user;

[0198] Determine a three-dimensional trajectory route of a user's sampling point, and merge the three-dimensional trajectory route with a preset map to generate a three-dimensional trajectory map;

[0199] The dynamic three-dimensional identity information is fused with the three-dimensional trajectory route to generate a motion trajectory frame sequence, and the motion trajectory frame sequence is fused with the three-dimensional trajectory map to generate a user motion trajectory video.

[0200] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the above-mentioned method for generating a user motion trajectory, the method comprising:

[0201] generating static three-dimensional identity information based on the acquired user image;

[0202] Generating dynamic three-dimensional identity information according to the static three-dimensional identity information, wherein the dynamic three-dimensional identity information is information describing a deformable three-dimensional human body model of the user;

[0203] Determine a three-dimensional trajectory route of a user's sampling point, and merge the three-dimensional trajectory route with a preset map to generate a three-dimensional trajectory map;

[0204] The dynamic three-dimensional identity information is fused with the three-dimensional trajectory route to generate a motion trajectory frame sequence, and the motion trajectory frame sequence is fused with the three-dimensional trajectory map to generate a user motion trajectory video.

[0205] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0206] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0207] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating a user motion trajectory, characterized in that: The method comprises: generating static three-dimensional identity information based on the acquired user image; Generating dynamic three-dimensional identity information according to the static three-dimensional identity information, wherein the dynamic three-dimensional identity information is information describing a deformable three-dimensional human body model of the user; Determine a three-dimensional trajectory route of a user's sampling point, and merge the three-dimensional trajectory route with a preset map to generate a three-dimensional trajectory map; The dynamic three-dimensional identity information is fused with the three-dimensional trajectory route to generate a motion trajectory frame sequence, and the motion trajectory frame sequence is fused with the three-dimensional trajectory map to generate a user motion trajectory video.

2. The method for generating a user motion trajectory according to claim 1, characterized in that: After generating the user motion trajectory video, the method further includes: Obtaining the user's motion trajectory video; Rendering the user motion trajectory video, wherein the rendering includes adding one or more combinations of materials, effects, subtitles, icons, and music; Display the rendered user motion trajectory video on the preset application.

3. The method for generating a user motion trajectory according to claim 2, characterized in that: The rendering process of the user motion trajectory video includes: Recalling one or more pre-stored static three-dimensional identity information; Based on the pre-stored one or more static three-dimensional identity information, and according to calling the static three-dimensional identity information generated by the user image, a corresponding user motion trajectory video is generated.

4. The method for generating a user motion trajectory according to claim 2, characterized in that: The rendering process of the user motion trajectory video includes: Performing personalized modification on the static three-dimensional identity information; Generate the corresponding user motion trajectory video based on the personalized modified static three-dimensional identity information.

5. The method for generating a user motion trajectory according to claim 1, characterized in that: After generating the user motion trajectory video, the method further includes: Live broadcast of the user's motion trajectory video in real time; or The user's motion trajectory video is recorded and played.

6. The method for generating a user motion trajectory according to claim 5, characterized in that: The recording and playing of the user's motion trajectory video includes: One or more combined operations of editing, forwarding, storing, and replaying are performed on the user motion trajectory video.

7. The method for generating a user motion trajectory according to claim 1, characterized in that: The user image includes a first image and a second image, and the step of generating static three-dimensional identity information based on the acquired user image includes: Acquire a first image and a second image of a user, wherein the first image includes user feature information, and the second image includes three-dimensional information corresponding to the first image; Extracting user features from the first image and the second image respectively to generate corresponding user texture information and user three-dimensional information; The user texture information is mapped to the user three-dimensional information to generate the static three-dimensional identity information.

8. The method for generating a user motion trajectory according to claim 7, characterized in that: The user texture information is one or more combinations of facial features, head features, and body features.

9. The method for generating a user motion trajectory according to claim 7, characterized in that: The user three-dimensional information is one or more combinations of facial features, head features, and body features.

10. The method for generating a user motion trajectory according to claim 7, characterized in that: The second image is acquired by any one of a 3D TOF camera, a 3D structured light camera, a binocular stereo vision camera, a coded structured light camera, and a three-dimensional scanner.

11. The method for generating a user motion trajectory according to claim 1, characterized in that: The user image is one or more combinations of a real face model, a real head model, a real body model, and an edited real model of the user pre-stored in the system.

12. The method for generating a user motion trajectory according to claim 1, characterized in that: The step of generating dynamic three-dimensional identity information according to the static three-dimensional identity information comprises: The static three-dimensional identity information is subjected to displacement parameter adjustment to generate a three-dimensional human body model, wherein the displacement parameter is adjusted to a parameter value moving from a first coordinate system to a second coordinate system; The three-dimensional human body model is subjected to motion parameter adjustment to generate the dynamic three-dimensional identity information, wherein the motion parameters include one or more combinations of parameters of center of mass motion, parameters of upper body motion, and parameters of lower body motion.

13. The method for generating a user motion trajectory according to claim 1, characterized in that: The step of determining the three-dimensional trajectory route of the user's sampling point includes: The three-dimensional trajectory route is generated by system preset settings according to user needs, and the three-dimensional trajectory route describes the trajectory route that the user wants to move; or the three-dimensional trajectory route is generated according to the user's real-time movement, and the three-dimensional trajectory route describes the trajectory route generated by the user's real-time movement.

14. The method for generating a user motion trajectory according to claim 1, characterized in that: The step of determining the three-dimensional trajectory route of the user's sampling point and fusing the three-dimensional trajectory route with a preset map to generate a three-dimensional trajectory map includes: Obtaining geographic information of user sampling points, sequentially connecting the geographic information of the sampling points to generate a three-dimensional trajectory route, and marking the starting point and the end point of the three-dimensional trajectory route; Mapping the data of all sampling points onto the preset map, and drawing a line from the starting point to the next adjacent point according to a preset time rhythm until the line reaches the end point; The coordinate system of the sampling point and the coordinate system of the preset map are combined into the same world coordinate system to achieve the fusion of the three-dimensional trajectory route and the preset map.

15. The method for generating a user motion trajectory according to claim 1, characterized in that: The step of fusing the dynamic three-dimensional identity information with the three-dimensional trajectory route to generate a motion trajectory frame sequence includes: Mapping the dynamic three-dimensional identity information to the sampling points to create real-life character image information of the user; The real person image information is integrated with the three-dimensional trajectory route to generate the motion trajectory frame sequence.

16. The method for generating a user motion trajectory according to claim 15, characterized in that: The step of fusing the motion trajectory frame sequence with the three-dimensional trajectory map to generate a user motion trajectory video comprises: The motion trajectory frame sequence is synthesized with the three-dimensional trajectory map, wherein the synthesis process includes binding the coordinate data, motion data, and physical data between the user's real character image information and the three-dimensional trajectory map; The real person image information and the three-dimensional trajectory map are scaled according to a preset ratio to ensure that the frame sequence of the real person image information is matched and adjusted with the time sequence of the three-dimensional trajectory map.

17. A device for generating a user motion trajectory, characterized in that: The device comprises: A user static identity information generation module, used to generate static three-dimensional identity information based on the acquired user image; A user dynamic identity information generation module, used to generate dynamic three-dimensional identity information according to the static three-dimensional identity information, wherein the dynamic three-dimensional identity information is information describing a three-dimensional human body model of the user; A three-dimensional trajectory map generation module is used to determine the three-dimensional trajectory route of the user's sampling point, and merge the three-dimensional trajectory route with a preset map to generate a three-dimensional trajectory map; The user motion trajectory video generation module is used to fuse the dynamic three-dimensional identity information with the three-dimensional trajectory route to generate a motion trajectory frame sequence, and fuse the motion trajectory frame sequence with the three-dimensional trajectory map to generate a user motion trajectory video.

18. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method for generating a user motion trajectory according to any one of claims 1 to 16 are implemented.

19. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for generating a user motion trajectory according to any one of claims 1 to 16 are implemented.

Images