Motion simulation method and device, electronic equipment and storage medium

Abstract

The invention provides a motion simulation method and device, electronic equipment and a storage medium, and the method comprises the steps: selecting a target sequence required by simulation from various gait sequences based on virtual motion parameters, the various gait sequences being gait recognition results based on each frame of image in a real motion video, and the target sequence being selected from the various gait sequences; the real motion video is obtained by dividing the real motion video; and performing motion simulation based on the target sequence. The method, the device, the electronic equipment and the storage medium provided by the invention can be realized by a basic camera without a complete set of shooting equipment, one real motion video can be split into various gait sequences to support motion simulation, and the video data volume required by the motion simulation is greatly reduced, so that the cost required by the motion simulation is greatly reduced, and the user experience is improved. And convenience is provided for the realization of motion simulation. Besides, the gait sequence obtained by splitting the gait recognition result realizes fine control for motion simulation, and is beneficial to optimization of the effect of motion simulation.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a motion simulation method, device, electronic equipment and storage medium. Background Art

[0002] As an important research direction in the field of artificial intelligence, virtual human motion simulation has broad application prospects in scenarios such as game development and video production.

[0003] At present, the motion simulation of virtual humans is usually achieved by kinematic control with 3D models. However, the production cost of ultra-high-detail 3D models is high and the production cycle is long, which greatly limits the application of motion simulation. In addition, there is also a solution that uses motion lenses to shoot 2D real-person videos and combines video editing technology to simulate the motion of virtual humans. This solution can reduce production costs and shorten production cycles, but requires the capture of a large amount of motion videos in the early stage. It has high requirements for the supporting shooting equipment and the video data obtained from the shooting, which brings difficulties to the realization of motion simulation. Summary of the invention

[0004] The present invention provides a motion simulation method, device, electronic device and storage medium, which are used to solve the defect that motion simulation in the related art has high requirements for shooting equipment and video data.

[0005] The present invention provides a motion simulation method, comprising: Based on the virtual motion parameters, a target sequence required for simulation is selected from various gait sequences, wherein the various gait sequences are obtained by dividing the real motion video based on the gait recognition results of each frame image in the real motion video; Based on the target sequence, motion simulation is performed.

[0006] According to a motion simulation method provided by the present invention, the determination of the gait recognition result of each frame image includes: Performing joint point recognition on each frame image in the real motion video to obtain position information of the joint points in each frame image; Based on the joint point position information in each frame image, gait recognition is performed on each frame image to obtain a gait recognition result of each frame image.

[0007] According to a motion simulation method provided by the present invention, the joint point position information includes the center point position information of the character and the center point position information of the two feet of the character.

[0008] According to a motion simulation method provided by the present invention, the gait recognition is performed on each frame image based on the joint point position information in each frame image to obtain the gait recognition result of each frame image, including: Based on the joint point position information in each frame image, capturing the relationship between the gait cycles in the real motion video; Based on the relationship between the gait cycles, gait recognition is performed on the frame images to obtain gait recognition results of the frame images.

[0009] According to a motion simulation method provided by the present invention, the motion simulation is performed based on the target sequence, comprising: Determining playback parameters of the target sequence based on the virtual motion parameters and the motion parameters of the target sequence; Motion simulation is performed based on the target sequence and playback parameters of the target sequence.

[0010] According to a motion simulation method provided by the present invention, the motion simulation is performed based on the target sequence and the playback parameters of the target sequence, including: deriving a simulated motion trajectory based on the target sequence and playback parameters of the target sequence; Based on the simulated motion trajectory, motion simulation is performed.

[0011] According to a motion simulation method provided by the present invention, the virtual motion parameters include at least one of a character frame size, a motion start and end point, and a motion duration.

[0012] The present invention also provides a motion simulation device, comprising: A sequence selection unit, configured to select a target sequence required for simulation from various gait sequences based on virtual motion parameters, wherein the various gait sequences are obtained by dividing the real motion video based on gait recognition results of each frame image in the real motion video; The motion simulation unit is used to perform motion simulation based on the target sequence.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the motion simulation methods described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the motion simulation method described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the motion simulation method described above is implemented.

[0016] The motion simulation method, device, electronic device and storage medium provided by the present invention can pre-divide the video based on the gait recognition results of each frame in the real motion video to obtain various gait sequences, thereby selecting a target sequence from various gait sequences to realize motion simulation. The shooting of the real motion video only requires a basic camera, without a complete set of shooting equipment, and a real motion video can be split into various gait sequences to support motion simulation. The amount of video data required for motion simulation is greatly reduced, which greatly reduces the cost required for motion simulation and provides convenience for the realization of motion simulation. In addition, the gait sequence obtained by splitting the gait recognition result makes the action simulation granularity refined to gait during motion simulation, realizing refined control for motion simulation and helping to optimize the effect of motion simulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technical personnel in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the motion simulation method provided by the present invention.

[0019] Figure 2 It is a schematic diagram of the simulated motion trajectory provided by the present invention.

[0020] Figure 3 It is a structural schematic diagram of the motion simulation device provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technical personnel in the field without creative work are within the scope of protection of the present invention.

[0023] At present, motion simulation of virtual humans is usually achieved through kinematic control with three-dimensional models. However, the production cost of ultra-high-detail three-dimensional models is high, the production cycle is long, and they rely on three-dimensional modeling software from other manufacturers, which greatly limits the application of motion simulation.

[0024] In addition, there is also a solution that uses motion shots (such as push, pull, shake, move, follow, rise, fall, and swing) to shoot two-dimensional real-person videos and combine video editing technology to simulate the movement of virtual people. This solution can avoid the problems of solutions based on three-dimensional model motion simulation, reduce production costs, and shorten production cycles.

[0025] However, the solution of simulating the movement of virtual humans by shooting two-dimensional real-life videos with motion lenses requires the capture of a large amount of motion videos in the early stage, which places high demands on the supporting shooting equipment and the video data obtained from the shooting, bringing difficulties to the realization of motion simulation.

[0026] Moreover, the current solution for simulating virtual human movement based on two-dimensional real-person video can only support the simulation of a few specific actions, for example, it can support the simulation of idle, walking, running, jumping, squatting and other actions. This makes the solution lack of refined control over human movement, resulting in more restrictions when simulating virtual human movement.

[0027] In view of the above problems, an embodiment of the present invention provides a motion simulation method. Figure 1 It is a flow chart of the motion simulation method provided by the present invention, such as Figure 1 As shown, the method includes: Step 110, based on the virtual motion parameters, a target sequence required for simulation is selected from various gait sequences, wherein the various gait sequences are obtained by dividing the real motion video based on the gait recognition results of each frame image in the real motion video.

[0028] Specifically, real motion videos can be collected in advance, and the real motion videos can be divided to form various gait sequences for motion simulation. Here, the real motion video can be a video of a real person's motion that is actually captured, for example, it can be a video of a real person walking. It is understandable that during the walking process, the real person may make various movements and present various gaits. For example, a video of a real person walking during the entire speech process can be captured as a real motion video.

[0029] For the collected real motion video, gait recognition can be performed for each frame of the real motion video, thereby obtaining a gait recognition result for each frame. Gait recognition here can specifically be the recognition of the walking posture of the person in the image. The walking posture here is the gait, which can specifically refer to the recognition of the stride, step frequency, foot swing cycle, etc. The gait recognition result for each frame of the image obtained can be the information reflecting the walking posture of the person in each frame of the image, such as the stride, step frequency, foot swing cycle, etc.

[0030] After obtaining the gait recognition result of each frame image in the real motion video, the real motion video can be divided based on the gait recognition result of each frame image, thereby dividing the real motion video into multiple image sequences according to different gait stages. Here, since the gait recognition results of images in different image sequences are different, the image sequence can be recorded as a gait sequence, that is, a gait sequence is a sequence composed of multiple continuous frames of images in a gait stage, and a gait sequence can reflect the movement performance of a person in a gait stage.

[0031] On this basis, virtual motion parameters can be determined according to the needs of motion simulation. The virtual motion parameters here are the motion parameters of the virtual character required for motion simulation, and the virtual motion parameters can reflect the motion effect presented by the desired motion simulation. The virtual motion parameters can be expressed as the size of the picture frame where the virtual character is located, the starting and ending points of the movement of the picture frame where the virtual character is located in the picture, the movement duration of the virtual character, etc., which are not specifically limited in the embodiments of the present invention.

[0032] After determining the virtual motion parameters, a gait sequence that can match the virtual motion parameters can be selected from various gait sequences obtained in advance as a target sequence required for simulation. The target sequence selected here can be one gait sequence or multiple gait sequences, and when multiple gait sequences are selected as target sequences, the multiple target sequences can be arranged in an order that matches the virtual motion parameters.

[0033] In this process, a gait sequence that can match the virtual motion parameters is selected from various gait sequences acquired in advance. The gait sequence whose gait phase and duration can match the virtual motion parameters can be selected from various gait sequences. For example, the mapping relationship between various gait sequences and sample motion parameters can be pre-set, and the above mapping relationship can be learned through model training. The model thus trained can output a gait sequence that matches the input virtual motion parameters.

[0034] Step 120: Perform motion simulation based on the target sequence.

[0035] Specifically, after obtaining the target sequence, the target sequence can be played to achieve motion simulation that meets the requirements of the virtual motion parameters. It is understandable that, in the case of multiple target sequences, each target sequence can be played in sequence in an order that matches the virtual motion parameters to achieve motion simulation that meets the requirements of the virtual motion parameters.

[0036] In the method provided by the embodiment of the present invention, video segmentation can be performed in advance based on the gait recognition results of each frame in the real motion video to obtain various gait sequences, thereby selecting a target sequence from various gait sequences to achieve motion simulation. The shooting of real motion video only requires a basic camera, without a complete set of shooting equipment, and a real motion video can be split into various gait sequences to support motion simulation. The amount of video data required for motion simulation is greatly reduced, which greatly reduces the cost required for motion simulation and provides convenience for the realization of motion simulation. In addition, the gait sequence obtained by splitting the gait recognition results makes the action simulation granularity during motion simulation refined to gait, realizing refined control of motion simulation and helping to optimize the effect of motion simulation.

[0037] Based on any of the above embodiments, various gait sequences are obtained by dividing the gait recognition results of each frame image in the real motion video, wherein the determination of the gait recognition results of each frame image includes: Performing joint point recognition on each frame image in the real motion video to obtain position information of the joint points in each frame image; Based on the joint point position information in each frame image, gait recognition is performed on each frame image to obtain a gait recognition result of each frame image.

[0038] Specifically, in order to perform gait recognition on each frame image in a real motion video, joint point recognition can be first performed on each frame image. Specifically, the joint points of the characters contained in each frame image are recognized to obtain the positions of the joint points of the characters contained in each frame image in the image, that is, the joint point position information in each frame image is obtained.

[0039] Here, joint point recognition for each frame image can be achieved through a pre-trained joint point recognition model. Specifically, each frame image can be input into the joint point recognition model, and the joint point recognition model can be used to recognize the joint points of each input frame image, and the joint point position information of each frame image output by the joint point recognition model can be obtained. The joint point recognition model here can be built based on a convolutional neural network (CNN).

[0040] It can be understood that, for any frame image, the joint point position information of the image can reflect the behavior and posture of the person in the image.

[0041] After obtaining the joint point position information of each frame image, gait recognition can be performed on each frame image based on this. In the specific process of gait recognition, gait recognition can be performed on each frame image based on the character behavior posture of each frame image itself reflected by the joint point position information of each frame image itself, and the changes in the character behavior posture within the gait cycle reflected by the joint point position information of the images arranged before and / or after each frame image in the real motion video, thereby obtaining the gait recognition result of each frame image.

[0042] Based on any of the above embodiments, the joint point position information includes the center point position information of the character and the center point position information of the two feet of the character.

[0043] That is, in the process of determining the gait recognition result of each frame image, joint point recognition is performed for each frame image, specifically, the center point of the person and the center points of the two feet of the person included in each frame image are recognized, a total of three joint points. Compared with the solution that requires the recognition of all joint points of the person, in the embodiment of the present invention, only the center point of the person and the center points of the two feet are recognized, which greatly reduces the workload of joint point recognition while meeting the subsequent gait recognition requirements, improves the efficiency of joint point recognition, and further improves the efficiency of gait sequence division.

[0044] Based on any of the above embodiments, in the step of determining the gait recognition results of various images, performing gait recognition on each frame of image based on the joint point position information in each frame of image to obtain the gait recognition results of each frame of image includes: Based on the joint point position information in each frame image, capturing the relationship between the gait cycles in the real motion video; Based on the relationship between the gait cycles, gait recognition is performed on the frame images to obtain gait recognition results of the frame images.

[0045] Specifically, after obtaining the joint point position information in each frame image, the relationship between the gait cycles in the real motion video can be captured based on the changes in the character's behavior posture in the gait cycle reflected by the joint point position information of each frame image and the previous and subsequent images in the real motion video. Here, the capture of the relationship between the gait cycles can be achieved through a long short-term memory network (Long Short-Term Memory, LSTM) or other network structures that can learn long-term dependencies, such as Deep Recurrent Neural Networks (Deep RNNs), Bidirectional Recurrent Neural Networks (Bi-RNN), etc.

[0046] After obtaining the relationship between gait cycles in the real motion video, gait recognition can be performed on each frame image based on this. The process of gait recognition here can be to identify the gait stage of each frame image. The gait recognition result of each frame image can be the gait stage of each frame image, or it can be the specific information of the gait of each frame image, such as stride, step frequency, foot swing cycle, etc. The embodiment of the present invention does not make specific limitations on this.

[0047] Based on any of the above embodiments, in step 120, performing motion simulation based on the target sequence includes: Determining playback parameters of the target sequence based on the virtual motion parameters and the motion parameters of the target sequence; Motion simulation is performed based on the target sequence and playback parameters of the target sequence.

[0048] Specifically, after obtaining the target sequence, it is also necessary to combine the virtual motion parameters and the motion parameters of the target sequence itself to determine the playback parameters of the target sequence. Here, the motion parameters of the target sequence itself, that is, the motion parameters of the real person presented by the target sequence itself in the real motion video, can specifically be the duration of the target sequence, the stride and frequency of the gait reflected by the target sequence, etc., which are not specifically limited in the embodiment of the present invention. The playback parameters of the target sequence, that is, the playback speed, start and end time of the playback, image size, image position, etc. of the target sequence during playback.

[0049] In order to determine the playback parameters of the target sequence, the mapping relationship between various sample motion parameters and the motion parameters and playback parameters of various gait sequences can be set in advance, and the above mapping relationship can be learned through model training. The model thus trained can output the playback parameters of the target sequence based on the input virtual motion parameters and the motion parameters of the target sequence.

[0050] Alternatively, the determination of the target sequence and the determination of the playback parameters of the target sequence may also be achieved through the same model. That is, the mapping relationship between various gait sequences and their playback parameters and sample motion parameters may be preset, and the above mapping relationship may be learned through model training, so that the trained model may output a gait sequence matching the input virtual motion parameters and the playback parameters required when the gait sequence is played as the target sequence, which is not specifically limited in the embodiment of the present invention.

[0051] After determining the target sequence and the playback parameters of the target sequence, the target sequence can be played based on the playback parameters of the target sequence to achieve motion simulation that meets the requirements of the virtual motion parameters. It is understandable that, in the case of multiple target sequences, each target sequence can be played in sequence according to the playback parameters of each target sequence in the order matching the virtual motion parameters to achieve motion simulation that meets the requirements of the virtual motion parameters.

[0052] In the method provided by the embodiment of the present invention, different target sequences may correspond to different playback parameters, so that during the motion simulation process, different target sequences can be adaptively played at a faster or slower speed to match the requirements of the virtual motion parameters.

[0053] Based on any of the above embodiments, in step 120, the performing motion simulation based on the target sequence and the playback parameters of the target sequence includes: deriving a simulated motion trajectory based on the target sequence and playback parameters of the target sequence; Based on the simulated motion trajectory, motion simulation is performed.

[0054] Specifically, after determining the target sequence and its playback parameters, the simulated motion trajectory can be derived based on this. The simulated motion trajectory here is the motion trajectory of the picture frame where the character in each frame image in the target sequence is located during the playback process. It can be understood that the simulated motion trajectory includes the position of the picture frame where the character is located corresponding to each time point during the motion simulation process.

[0055] For example, Figure 2 It is a schematic diagram of the simulation motion trajectory provided by the present invention, such as Figure 2 As shown, Figure 2 The three blocks from top to bottom in the figure represent the positions of the frames of the characters at three different time points in the simulated motion trajectory. Assuming that the virtual motion parameters reflect the action of the character walking to the right, in the top block diagram, the frame of the character is located on the left side of the screen and is stationary. As time goes by, the character moves. In the middle block diagram, the frame of the character is located in the middle of the screen and is walking to the right. Then in the bottom block diagram, the frame of the character is located on the right side of the screen and is stationary. At this time, the character has completed the movement.

[0056] After obtaining the simulated motion trajectory, motion simulation can be achieved by playing the simulated motion trajectory. Since the moving rate of the picture frame in the simulated motion trajectory is determined by the respective playback parameters of each target sequence, the moving rate of the picture frame in the simulated motion trajectory can be nonlinearly controlled. Therefore, the method provided by the embodiment of the present invention can solve the problem of artifact sliding existing in the current solution of simulating virtual human motion based on two-dimensional real-person video, thereby greatly improving the realism of motion simulation.

[0057] Based on any of the above embodiments, the virtual motion parameters include at least one of the character frame size, the motion start and end points, and the motion duration.

[0058] Specifically, the virtual motion parameters may include the size of the frame where the virtual character is located, that is, the size of the character frame. The size of the character frame can reflect the specific size ratio of the character frame in the picture. It can be understood that for the same picture size, in order to avoid the character being out of the picture, the larger the character frame, the smaller the stride and step frequency required for motion simulation; in addition, in order to avoid the character's movement being unclear, the smaller the character frame, the larger the stride and step frequency required for motion simulation. Therefore, the gait that matches the character frame size can be determined based on the character frame size, thereby realizing the selection of the gait sequence.

[0059] In addition, the virtual motion parameters may also include the start and end points of the motion, which are the starting and ending positions of the character frame in the picture. It is understandable that the start and end points of the motion can reflect the motion direction of the character, based on which the gait that matches it can be determined, thereby realizing the selection of the gait sequence; in addition, the start and end points of the motion can also reflect the distance of the character's motion. The longer the distance of the character's motion, the more it matches the gait sequence with a longer duration, and the shorter the distance of the character's motion, the more it matches the gait sequence with a shorter duration, thereby realizing the selection of the gait sequence based on the distance of the character's motion.

[0060] Furthermore, the virtual motion parameters may also include motion duration, which is the length of time that the task frame moves in the picture. Gait sequences with equivalent sequence durations may be matched based on the motion duration, thereby achieving the selection of gait sequences.

[0061] Based on any of the above embodiments, the present invention provides a motion simulation method, which may include the following steps: First, collect real motion videos. Based on CNN or other neural network structures, the three joints of the characters in each frame of the real motion video, including the center point of the characters and the center points of the two feet, are identified, thereby obtaining the joint position information of each frame of the real motion video. Subsequently, the joint position information of each frame of the real motion video can be embedded into a neural network structure such as LSTM that has the ability to extract long-term dependencies to capture the relationship between each gait cycle in the real motion video, and based on this, the gait of the characters at different stages is identified, that is, the gait recognition result of each frame is obtained, thereby dividing the real motion video into multiple gait sequences, each gait sequence corresponds to a time period in the real motion video, thereby obtaining gait sequences reflecting different gaits, and based on the time period of different gait sequences in the real motion video, the sequence duration is associated with different gait sequences as a factor in the motion parameters of the gait sequence.

[0062] Subsequently, the virtual motion parameters required for motion simulation can be obtained. The virtual motion parameters here may include at least one of the character frame size, the start and end points of the motion, and the motion duration, and the virtual motion parameters can be obtained by user input. After the virtual motion parameters are obtained, a gait sequence matching the virtual motion parameters can be extracted from each gait sequence based on the virtual motion parameters as a target sequence, and corresponding playback parameters are configured for the target sequence so that the target sequence can be adaptively played at a fast or slow speed.

[0063] After obtaining the target sequence and its playback parameters, the simulated motion trajectory can be output based on it. Since the movement rate of the picture frame in the simulated motion trajectory can be nonlinearly controlled, the problem of artifact sliding can be solved, the robustness of motion simulation is enhanced, and the realism of simulated motion is improved.

[0064] In the method provided by the embodiment of the present invention, the shooting of real motion video can be realized with only a basic camera, without a complete set of shooting equipment, which is shooting-friendly and can support motion actions in any scene. Only one real motion video is needed for one type of action, and all types of real motion videos do not need to be associated, which greatly reduces the requirements of motion simulation for supporting equipment and the requirements for the amount of video data, greatly reduces the cost required for motion simulation, and provides convenience for the implementation of motion simulation.

[0065] In addition, the gait sequence obtained by splitting the gait recognition results makes the action simulation granularity refined to gait during motion simulation, realizing refined control of motion simulation and helping to optimize the effect of motion simulation.

[0066] Moreover, the CNN model required for joint point recognition and the LSTM model required for gait recognition used in the segmentation of real motion videos can support generalized videos and have strong scalability. When expanding motion actions, you only need to input the real motion video of the new action into the CNN and LSTM models without retraining the model.

[0067] Furthermore, gait sequences can be stored offline, that is, the selection of target sequences and motion simulation based on them can be realized in an offline state, which greatly improves the convenience of motion simulation.

[0068] In addition, the motion simulation process does not involve 3D modeling and rendering, and the output video is low-cost, lightweight, fast, efficient, and can reduce machine resource usage.

[0069] The motion simulation device provided by the present invention is described below. The motion simulation device described below and the motion simulation method described above can be referenced to each other.

[0070] Figure 3 Schematic diagram of the structure of the motion simulation device provided by the present invention. Figure 3 As shown, the device comprises: A sequence selection unit 310 is used to select a target sequence required for simulation from various gait sequences based on virtual motion parameters, wherein the various gait sequences are obtained by dividing the real motion video based on the gait recognition results of each frame image in the real motion video; The motion simulation unit 320 is used to perform motion simulation based on the target sequence.

[0071] In the device provided by the embodiment of the present invention, video segmentation can be performed in advance based on the gait recognition results of each frame in the real motion video to obtain various gait sequences, thereby selecting a target sequence from various gait sequences to achieve motion simulation. The shooting of real motion video only requires a basic camera, without a complete set of shooting equipment, and a real motion video can be split into various gait sequences to support motion simulation. The amount of video data required for motion simulation is greatly reduced, which greatly reduces the cost required for motion simulation and provides convenience for the realization of motion simulation. In addition, the gait sequence obtained by splitting the gait recognition results makes the action simulation granularity refined to gait during motion simulation, realizing refined control of motion simulation and helping to optimize the effect of motion simulation.

[0072] Based on any of the above embodiments, the device further includes a gait recognition unit, which is used to: Performing joint point recognition on each frame image in the real motion video to obtain position information of the joint points in each frame image; Based on the joint point position information in each frame image, gait recognition is performed on each frame image to obtain a gait recognition result of each frame image.

[0073] Based on any of the above embodiments, the joint point position information includes the center point position information of the character and the center point position information of the two feet of the character.

[0074] Based on any of the above embodiments, the gait recognition unit is specifically used for: Based on the joint point position information in each frame image, capturing the relationship between the gait cycles in the real motion video; Based on the relationship between the gait cycles, gait recognition is performed on the frame images to obtain gait recognition results of the frame images.

[0075] Based on any of the above embodiments, the motion simulation unit is specifically used for: Determining playback parameters of the target sequence based on the virtual motion parameters and the motion parameters of the target sequence; Motion simulation is performed based on the target sequence and playback parameters of the target sequence.

[0076] Based on any of the above embodiments, the motion simulation unit is specifically used for: deriving a simulated motion trajectory based on the target sequence and playback parameters of the target sequence; Based on the simulated motion trajectory, motion simulation is performed.

[0077] Based on any of the above embodiments, the virtual motion parameters include at least one of the character frame size, the motion start and end points, and the motion duration.

[0078] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the motion simulation method, which includes: Based on the virtual motion parameters, a target sequence required for simulation is selected from various gait sequences, wherein the various gait sequences are obtained by dividing the real motion video based on the gait recognition results of each frame image in the real motion video; Based on the target sequence, motion simulation is performed.

[0079] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0080] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the motion simulation method provided by the above methods, the method includes: Based on the virtual motion parameters, a target sequence required for simulation is selected from various gait sequences, wherein the various gait sequences are obtained by dividing the real motion video based on the gait recognition results of each frame image in the real motion video; Based on the target sequence, motion simulation is performed.

[0081] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the motion simulation method provided by the above methods is implemented, and the method includes: Based on the virtual motion parameters, a target sequence required for simulation is selected from various gait sequences, wherein the various gait sequences are obtained by dividing the real motion video based on the gait recognition results of each frame image in the real motion video; Based on the target sequence, motion simulation is performed.

[0082] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art may understand and implement it without creative effort.

[0083] Through the description of the above implementation methods, the technical personnel in this field can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A motion simulation method, characterized in that: include: Based on the virtual motion parameters, a target sequence required for simulation is selected from various gait sequences, wherein the various gait sequences are obtained by dividing the real motion video based on the gait recognition results of each frame image in the real motion video; Based on the target sequence, motion simulation is performed.

2. The motion simulation method according to claim 1, characterized in that: The determination of the gait recognition results of each frame image includes: Performing joint point recognition on each frame image in the real motion video to obtain position information of the joint points in each frame image; Based on the joint point position information in each frame image, gait recognition is performed on each frame image to obtain a gait recognition result of each frame image.

3. The motion simulation method according to claim 2, characterized in that: The joint point position information includes the center point position information of the character and the center point position information of the two feet of the character.

4. The motion simulation method according to claim 2, characterized in that: The step of performing gait recognition on each frame of image based on the joint point position information in each frame of image to obtain the gait recognition result of each frame of image includes: Based on the joint point position information in each frame image, capturing the relationship between the gait cycles in the real motion video; Based on the relationship between the gait cycles, gait recognition is performed on the frame images to obtain gait recognition results of the frame images.

5. The motion simulation method according to claim 1, characterized in that: The performing motion simulation based on the target sequence includes: Determining playback parameters of the target sequence based on the virtual motion parameters and the motion parameters of the target sequence; Motion simulation is performed based on the target sequence and playback parameters of the target sequence.

6. The motion simulation method according to claim 5, characterized in that: The performing of motion simulation based on the target sequence and the playback parameters of the target sequence includes: deriving a simulated motion trajectory based on the target sequence and playback parameters of the target sequence; Based on the simulated motion trajectory, motion simulation is performed.

7. The motion simulation method according to any one of claims 1 to 6, characterized in that: The virtual motion parameters include at least one of a character frame size, a motion start and end point, and a motion duration.

8. A motion simulation device, characterized in that: include: A sequence selection unit, configured to select a target sequence required for simulation from various gait sequences based on virtual motion parameters, wherein the various gait sequences are obtained by dividing the real motion video based on gait recognition results of each frame image in the real motion video; The motion simulation unit is used to perform motion simulation based on the target sequence.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the motion simulation method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the motion simulation method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the motion simulation method according to any one of claims 1 to 7 is implemented.

Images