Data processing method and device for camera control, equipment and storage medium

Abstract

The invention provides a data processing method and device for camera control, equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring a skeletal animation sequence of a virtual object; segmenting the skeletal animation sequence into a plurality of animation segments based on action segmentation points; obtaining a target presentation part corresponding to each animation segment; and obtaining camera motion data according to the target representation part corresponding to each animation segment and the time point corresponding to each animation segment. According to the scheme, the camera position of the virtual camera at each time point is indicated according to the acquired camera motion data, manual determination is not needed, and the video generation efficiency of the virtual object can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a data processing method, device, equipment and storage medium for camera control. Background Art

[0002] With the continuous development of virtual scene technology, the application of collecting images of virtual objects in virtual space to obtain videos of virtual objects is becoming more and more widespread.

[0003] In related technologies, developer users can manually determine the camera position at each time point when a virtual object performs an action based on the action sequence of the virtual object, and set the camera position at the above-mentioned time points as the motion parameters of the virtual camera. Subsequently, the virtual camera moves according to the above-mentioned motion parameters and shoots the picture of the virtual object to obtain a video of the virtual object performing the specified action.

[0004] In the above solution, the camera position at each time point needs to be manually determined, resulting in low efficiency in setting the motion parameters of the virtual camera, thereby affecting the efficiency of generating the video of the virtual object. Summary of the invention

[0005] The embodiments of the present application provide a data processing method, device, equipment and storage medium for camera control, which can improve the efficiency of generating videos of virtual objects. The technical solution is as follows.

[0006] In one aspect, a data processing method for camera control is provided, the method comprising:

[0007] Acquire a skeleton animation sequence of a virtual object; the skeleton animation sequence includes animations of skeleton movements of the virtual object at various time points;

[0008] The skeleton animation sequence is divided into a plurality of animation segments based on an action segmentation point; the action segmentation point is a time point at which a sudden change occurs in the skeleton action of the virtual object;

[0009] Acquire the target performance part corresponding to each of the animation clips; the target performance part is the body part corresponding to the skeletal action that undergoes key changes in the animation clips;

[0010] Camera motion data is obtained according to the target expression part corresponding to each of the animation clips and the time point corresponding to each of the animation clips; the camera motion data is used to indicate the camera position of the virtual camera at each time point; the camera motion data is used to control the virtual camera for shooting the virtual object executing the skeletal animation sequence.

[0011] In one aspect, a data processing device for camera control is provided, the device comprising:

[0012] A sequence acquisition module is used to acquire a skeleton animation sequence of a virtual object; the skeleton animation sequence includes animations of skeleton movements of the virtual object at various time points;

[0013] A segmentation module, used for segmenting the skeleton animation sequence into a plurality of animation segments based on an action segmentation point; the action segmentation point is a time point at which a sudden change occurs in the skeleton action of the virtual object;

[0014] A part acquisition module, used to acquire a target performance part corresponding to each of the animation clips; the target performance part is a body part corresponding to a skeletal action that undergoes a key change in the animation clip;

[0015] A data acquisition module is used to acquire camera motion data according to the target expression parts corresponding to each of the animation clips and the time points corresponding to each of the animation clips; the camera motion data is used to indicate the camera position of the virtual camera at each time point; the camera motion data is used to control the virtual camera for shooting the virtual object executing the skeletal animation sequence.

[0016] On the other hand, a computer device is provided, which includes a processor and a memory, wherein the memory stores at least one computer instruction, and the at least one computer instruction is loaded and executed by the processor to implement the above-mentioned data processing method for camera control.

[0017] On the other hand, a computer-readable storage medium is provided, wherein at least one computer instruction is stored in the storage medium, and the at least one computer instruction is loaded and executed by a processor to implement the above-mentioned data processing method for camera control.

[0018] In another aspect, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned data processing method for camera control.

[0019] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0020] The embodiment of the present application divides the skeleton animation sequence of the virtual object into multiple animation clips based on the action segmentation points; analyzes the key actions and posture information of each animation clip, and obtains the target performance part corresponding to each animation clip; thereby obtaining the camera motion data. The camera motion data can indicate the camera position of the virtual camera at each time point, and control the virtual camera that shoots the virtual object executing the skeleton animation sequence. Compared with the prior art, the embodiment of the present application indicates the camera position of the virtual camera at each time point according to the obtained camera motion data, without the need for manual determination, and can improve the generation efficiency of the video of the virtual object.

[0021] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of an implementation environment of a solution provided by an embodiment of the present application;

[0023] Figure 2 is a flow chart of a data processing method for camera control provided by an embodiment of the present application;

[0024] Figure 3 is a flow chart of a data processing method for camera control provided by another embodiment of the present application;

[0025] Figure 4 is a flow chart of a data processing method for camera control provided by another embodiment of the present application;

[0026] Figure 5 is a flowchart of a data processing method for camera control provided by yet another embodiment of the present application;

[0027] Figure 6 is a flowchart of a data processing method for camera control provided by another embodiment of the present application;

[0028] Figure 7 is a flowchart of a method for creating a shot animation based on virtual human skeleton dance animation data provided by an embodiment of the present application;

[0029] Figure 8 is a schematic diagram of an implementation of a data processing method for camera control provided by an embodiment of the present application;

[0030] Fig. 9 is a flowchart of a method for generating a shot sequence based on virtual human skeleton dance animation data provided by an embodiment of the present application;

[0031] Fig.10 is a schematic diagram of a thigh bone involved in an embodiment of the present application;

[0032] Fig.11 is a schematic diagram of original data and smoothed data involved in an embodiment of the present application;

[0033] Fig.12 It is a schematic diagram of the difference between manual recognition involved in one embodiment of the present application and the action segmentation point of the embodiment of the present application;

[0034] Fig.13 is a schematic diagram of a dance segment performance part involved in an embodiment of the present application;

[0035] Fig.14 is a schematic diagram of camera positions for a dance segment according to an embodiment of the present application;

[0036] Fig.15 This is an interface diagram of a shot sequence based on virtual human skeleton dance animation data provided by an embodiment of the present application;

[0037] Fig.16 is a block diagram of a data processing device for camera control according to an embodiment of the present application;

[0038] Fig.17 It is a structural block diagram of a computer device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0040] For ease of understanding, several terms involved in this application are explained below.

[0041] Real-time rendering: refers to the computer calculating the picture while outputting it for display, with a frame rate of at least 25 frames or above to make the human eye feel coherent and real-time. Its advantage is that it can be controlled and interacted in real time. The disadvantage is that it is limited by the load capacity of the system. If necessary, the picture effect (the fineness of the model, the application of light and shadow, the fineness of the texture) must be sacrificed to meet the requirements of the real-time system.

[0042] Virtual human: A virtual digital human refers to a three-dimensional model synthesized by using digital technology to simulate the real human skeletal structure and appearance.

[0043] Skeletal animation: Skeletal animation is a commonly used animation technology that achieves the dynamic performance of the character by controlling the character's skeletal system and defining the movements.

[0044] Keyframe analysis: Skeletal animation keyframe analysis refers to analyzing the skeleton animation sequence, identifying and extracting the keyframes in it. By analyzing the skeleton animation sequence, you can find the key moments such as the starting point, turning point and end point of the character's action, so as to better understand and control the animation.

[0045] Virtual camera: A virtual camera is a concept used in computer graphics to simulate and control the behavior and perspective of a camera in a virtual environment. It can simulate a camera in the real world and create a variety of visual effects by adjusting parameters such as perspective, focal length, aperture, etc.

[0046] Dance camera movement: refers to the movement of the camera when shooting a dance video. The camera movement of a dance video should be able to fully show the dancer's movements and skills. The selection and switching of shots should be able to accurately capture the dancer's key movements to show their grace and strength. Appropriate use of close-ups, long shots, and medium shots can highlight the details and overall image of the dancer.

[0047] Please refer to Figure 1 , which shows a schematic diagram of a solution implementation environment provided by an embodiment of the present application. The solution implementation environment can be implemented as the architecture of a configuration system for a virtual camera. The solution implementation environment may include: a terminal device 110 and a server 120.

[0048] The terminal device 110 can be an electronic device such as a personal computer (PC), a tablet computer, a mobile phone, a wearable device, a vehicle-mounted terminal, etc. A client that runs a target application can be installed in the terminal device 110, and the target application can be an application with an animation function, such as a game engine. Exemplarily, the target application is a game engine, and the target application can run a camera tool, and configure the virtual camera according to the camera tool to generate a virtual camera sequence. The camera tool can be a plug-in based on the target application, or it can be an application independent of the target application. In addition, the present application does not limit the form of the target application, including but not limited to an application (Application, App) installed in the terminal device 110, a small program, etc., and can also be in the form of a web page.

[0049] The server 120 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The server 120 may be a background server of the target application, used to provide background services for the client of the target application.

[0050] The terminal device 110 and the server 120 may communicate via a network, such as a wired or wireless network.

[0051] Please refer to Figure 2 , which shows a flow chart of a data processing method for camera control provided by an embodiment of the present application, the method can be Figure 1 The terminal device 110 shown in FIG. Figure 2 As shown, the method may include step 210 , step 220 , step 230 and step 240 .

[0052] Step 210: Obtain a skeleton animation sequence of the virtual object; the skeleton animation sequence includes animations of skeleton movements of the virtual object at various time points.

[0053] Obtain a skeleton animation sequence of a virtual object, wherein the virtual object may be a virtual person, a virtual animal, or any virtual object in virtual reality that simulates the real world.

[0054] The skeleton animation sequence of the virtual object can be input by the developer user, and the skeleton animation sequence includes the animation of the skeleton movement of the virtual object at each time point.

[0055] For example, the virtual object is a virtual person, and the skeletal animation sequence includes the motion data of the virtual person's bones such as the head, upper arm, forearm, thigh, calf and spine at various time points.

[0056] Step 220: Divide the skeleton animation sequence into multiple animation clips based on the action segmentation points; the action segmentation points are the time points at which the skeleton actions of the virtual object suddenly change.

[0057] By analyzing the skeleton animation sequence of the virtual object obtained in step 210, the computer device can obtain the time point at which the skeleton action of the virtual object suddenly changes, and determine the action segmentation point based on the time point.

[0058] After obtaining the above-mentioned action segmentation points, the computer device can divide the skeletal animation sequence into a number of animation segments according to the action segmentation points.

[0059] For example, the computer device may extract the animation of the skeletal action between two adjacent action segmentation points in time sequence into an animation segment.

[0060] At the same time, the computer device can also obtain the time points corresponding to each animation clip. For example, the time points corresponding to each animation clip may include: the starting time point of each animation clip, or the ending time point of each animation clip, or the starting time point and the ending time point of each animation clip.

[0061] Step 230: Obtain the target performance part corresponding to each animation clip; the target performance part is the body part corresponding to the skeletal action that undergoes key changes in the animation clip.

[0062] The body part corresponding to the above-mentioned skeletal action that undergoes key changes can be the body part to which the skeleton corresponds to the key action in the animation clip belongs. That is to say, the computer device can extract the body part to which the skeleton corresponds to the key action of each animation clip by analyzing the multiple animation clips obtained in step 220 one by one, thereby determining the target expression part corresponding to each animation clip.

[0063] In a possible implementation, taking the virtual object as a virtual person as an example, the body parts of a virtual person can be divided into three parts: upper limbs, lower limbs and the whole body. For an animation clip, the target performance part corresponding to the animation clip can be one or more of the upper limbs, lower limbs and the whole body.

[0064] Optionally, the above-mentioned target performance parts may also be divided in other ways. For example, taking the virtual object as a virtual person as an example, the body parts of a virtual person can be divided into four parts: upper limbs, lower limbs, head and whole body. The embodiment of the present application does not limit the way of dividing the body parts of the virtual object.

[0065] Step 240: Obtain camera motion data according to the target expression part corresponding to each animation clip and the time point corresponding to each animation clip; the camera motion data is used to indicate the camera position of the virtual camera at each time point; the camera motion data is used to control the virtual camera for shooting the virtual object executing the skeletal animation sequence.

[0066] The computer device may obtain the camera motion data according to the target expression parts corresponding to each animation clip obtained in step 230 and the time points corresponding to each animation clip obtained in step 220. For example, the computer device may determine the camera position of the virtual camera when shooting the virtual object at each time point according to the target expression parts corresponding to each animation clip and the time points corresponding to each animation clip obtained in step 220, and the camera position may include the position information of the virtual camera in the virtual space.

[0067] The position information of the virtual camera in the virtual space may include the coordinate position information of the virtual camera in the spatial coordinate system of the virtual space; or, the position information of the virtual camera in the virtual space may also include the position information of the virtual camera in the object coordinate system of the virtual object, wherein the object coordinate system of the virtual object may be a coordinate system with the center of the virtual object as the origin.

[0068] In some embodiments, the method also includes: during the process of the virtual object executing a skeletal animation sequence, creating a virtual camera based on camera motion data, and setting the camera position of the virtual camera at each time point; capturing the image of the first object model corresponding to the virtual object at each time point through the virtual camera to obtain the target video.

[0069] In an embodiment of the present application, after the computer device determines the camera motion data, it can also create a virtual camera according to the above-mentioned camera motion parameters and capture images of the virtual object when controlling the virtual object to execute the above-mentioned skeletal animation sequence, and combine the captured image sequence into a target video.

[0070] Take creating a dance video of a virtual person as an example:

[0071] During the process of the virtual human executing the skeleton animation sequence, the computer device creates a virtual camera based on the camera motion data, and sets the camera position of the virtual camera at each time point, as well as the camera movement method between adjacent camera positions;

[0072] The 3D game engine renders the images in the 3D scene in real time according to the parameters of the virtual camera. The rendering machine outputs the audio and video stream of the virtual person dancing in the 3D scene in real time. The dance video finally generated can highlight the details and overall image of the virtual person's dance.

[0073] The embodiment of the present application provides a technical solution for obtaining a target video based on camera motion data, including creating a virtual camera, combining a real-time rendered image, and obtaining a target video of a virtual object. The embodiment of the present application can be applied to create video animations in purely virtual environments such as virtual human performances, live broadcasts, or CG animations. It has a wide range of applications and strong practical applicability, ensuring the generation effect and creation efficiency of video animations.

[0074] In summary, the embodiment of the present application divides the skeletal animation sequence of the virtual object into multiple animation clips based on the action segmentation points; analyzes each animation clip to obtain the target expression part corresponding to each animation clip; thereby obtaining camera motion data based on the target expression part corresponding to each animation clip, thereby realizing a solution for determining the camera position at the corresponding time point by using the body part corresponding to the skeletal action that undergoes a key change when the skeletal action in the skeletal animation sequence undergoes a sudden change. The solution can automatically determine the camera position at each time point within the time period when the virtual object executes the skeletal animation sequence, and at the same time ensure that the camera position at each time point can correspond to the body part corresponding to the skeletal action that undergoes a key change, thereby improving the generation efficiency of the camera motion data while ensuring the accuracy of the camera motion data, thereby improving the generation efficiency of the video of the virtual object.

[0075] Based on the above Figure 2 For the examples shown, please refer to Figure 3 , which shows a flow chart of a data processing method for camera control provided by another embodiment of the present application. Figure 3 As shown, before the above step 220, step 212 and step 214 may also be included.

[0076] Step 212: extracting target bone data from the bone animation sequence; the target bone data includes the rotation angle of the target bone of the virtual object on the specified axis at each time point.

[0077] Step 214: Obtain action segmentation points based on the target skeleton data.

[0078] The computer device may extract target bone data from the skeleton animation sequence of the virtual object obtained in step 210 ; and obtain action segmentation points where the skeleton action of the virtual object suddenly changes on the target bone data.

[0079] Among them, the object model of the virtual object may include multiple bones, such as head, upper arm, lower arm, hand, thigh, calf, foot, spine, etc.

[0080] Different bones have different amplitudes during movement. In an embodiment of the present application, bones with larger amplitudes during movement can be selected as target bones; wherein, the above-mentioned target bones can be multiple specified bones among all bones in the object model of the virtual object. For example, the target bones of the virtual object can include the head, upper arms, forearms, thighs, calves, spine, etc.

[0081] At each time point, each bone in the object model of the virtual object corresponds to a respective action state, and the action state may correspond to the rotation angle of the bone on each axis. The above-mentioned multiple designated bones correspond to respective designated axes.

[0082] Different designated bones have different characteristics during movement. Specifically, for a designated bone, during the movement, it usually has a larger amplitude of movement in one or part of the axes. For example, for the thigh bone, during the movement, it usually has a larger amplitude of movement in the axis of the front-to-back movement of the thigh (that is, it can have a larger rotation angle in the axis of the front-to-back movement), and a smaller amplitude of movement in the axis of the left-to-right movement or the rotational movement of the thigh (that is, the rotation angle in the axis of the left-to-right or rotational movement is smaller). This is determined by the skeletal structure of the human body. In this regard, for the thigh bone, the axis of the front-to-back movement can be pre-set as the designated axis of the thigh bone; for another example, for the calf bone, during the movement, it usually has a larger amplitude of movement in the axis of the front-to-back movement and the left-to-right movement of the calf, and a smaller amplitude of movement in the axis of the rotational movement of the calf. In this regard, for the calf bone, one or two of the axes of the front-to-back movement and the left-to-right movement can be pre-set as the designated axis of the calf bone.

[0083] In an embodiment of the present application, the computer device can first extract the bone data of the target bone in all axes from the bone animation sequence, and then extract the target bone data of the target bone in the specified axis from the bone data of the target bone in all axes; or, the computer device can also directly extract the data of the target bone of the virtual object in the specified axis from the bone animation sequence.

[0084] Taking the thigh bone as an example, the common leg movement is mainly kicking in the front-back direction, followed by lifting the leg left and right, and there is almost no axial rotation around the middle axis of the leg bone. Therefore, the axial data of the thigh bone in the front-back direction in the skeletal animation sequence can be directly extracted from the skeletal animation sequence; or the data (rotation angle) of the thigh bone in all axial directions in the skeletal animation sequence can be first extracted, and then the axial data of the thigh bone in the front-back direction can be extracted.

[0085] The embodiment of the present application provides a basic data for obtaining the action segmentation point in step 220, specifically, the computer device obtains the action segmentation point based on the target skeleton data, and the target skeleton data includes the rotation angle of the target skeleton of the virtual object on the specified axis at each time point. The embodiment of the present application extracts the action data of the skeleton that can reflect the action characteristics to determine the action segmentation point. On the one hand, the interference of the action data of unimportant skeletons is eliminated to ensure the accuracy of the action segmentation point determination; on the other hand, only the action data of a part of the skeleton needs to be used to determine the action segmentation point, which reduces the amount of data to be processed and improves the processing efficiency.

[0086] Based on the above Figure 2 or Figure 3 For the examples shown, please refer to Figure 4, which shows a flow chart of a data processing method for camera control provided by another embodiment of the present application. Figure 4 As shown, in some embodiments, the above step 220 can be implemented as step 220a, step 220b, step 220c and step 220d.

[0087] Step 220a: Smooth the target bone data to obtain smoothed bone data.

[0088] In the embodiment of the present application, since the target skeleton data is composed of the position and posture data of each target skeleton at each time point, it is necessary to perform data smoothing, filtering and other processing to extract data features. If the computer device directly processes the target skeleton data, it is necessary to spend a lot of time and computing power, which is not conducive to subsequent efficient processing. Therefore, the computer device can smooth the target skeleton data to simplify the processing flow of the target skeleton data.

[0089] Smoothing can eliminate noise and outliers in the data, making subsequent analysis more accurate and reliable. At the same time, the smoothed data can more accurately extract the characteristics and performance of the movement.

[0090] Step 220b: Detect the first time point at which the rotation angle suddenly changes from the smoothed skeleton data.

[0091] In an embodiment of the present application, the first time point at which the rotation angle suddenly changes from the smoothed skeletal data can be detected by detecting the time point at which the peak or trough of the curve of the smoothed skeletal data is located. For example, a curve showing the rotation angle of a specified bone in a specified axis over time is drawn based on the smoothed skeletal data, and the time points corresponding to the peaks and troughs on the curve are determined as the first time point.

[0092] In some embodiments, when the computer device determines the peaks and troughs on the curve, it can determine the tangents of each point on the curve, and determine the time points corresponding to the tangents parallel to the time axis as the time points corresponding to the peaks and troughs.

[0093] Step 220c: Obtain a second time point from the target bone data; the second time point is a time point in the target bone data that is before the first time point and is closest to the first time point and at which a sudden change in the rotation angle occurs.

[0094] Since the above-mentioned smoothed bone data is the result of smoothing the target bone data, the peaks and troughs of the smoothed bone data may not accurately correspond to the time points when the movement of the target bone suddenly changes. Therefore, after determining the above-mentioned first time point, the time point of the previous peak / trough of the first time point is also determined from the corresponding curve of the target bone data as the second time point.

[0095] Among them, the method of determining the time point of the peak / trough in the curve corresponding to the target bone data is similar to the method of detecting the first time point from the smoothed bone data, which will not be repeated here.

[0096] Step 220d: Obtain action segmentation points based on the second time point.

[0097] In a possible implementation, the computer device may use each second time point as an action segmentation point.

[0098] In another possible implementation, the computer device may also filter each second time point and determine a portion of the second time points as the action segmentation points. For example, the computer device may filter each second time point according to the distance between adjacent second time points and determine a portion of the second time points as the action segmentation points.

[0099] For example, for any two adjacent second time points among the second time points, the computer device can determine the time interval between the two adjacent second time points, and if the time interval is less than the time interval threshold, one of the two adjacent second time points is eliminated, for example, the previous second time point of the two adjacent second time points is eliminated, or the latter second time point of the two adjacent second time points is eliminated; correspondingly, if the time interval is not less than the time interval threshold, the two second time points are not eliminated in this elimination operation.

[0100] The embodiment of the present application provides a technical solution for obtaining the action segmentation point in step 220, specifically, the computer device performs smoothing on the target skeleton data, obtains the first time point at which the rotation angle changes suddenly according to the skeleton data after smoothing, then obtains the second time point at which the rotation angle changes suddenly and is located before the first time point and closest to the first time point from the target skeleton data, and obtains the action segmentation point based on the second time point. Compared with determining the action segmentation point directly from the target skeleton data, the embodiment of the present application determines the action segmentation point based on the target skeleton data and the skeleton data after smoothing. On the one hand, it can reduce the complexity and difficulty of obtaining the action segmentation point, and can efficiently obtain the action segmentation point; on the other hand, it can improve the accuracy of the action segmentation point and avoid the interference of unimportant noise or abnormal values ​​on the skeleton data.

[0101] Based on the above Figures 2 to 4 For any of the embodiments shown, please refer to Figure 5 , which shows a flow chart of a data processing method for camera control provided by another embodiment of the present application. Figure 5 As shown, when the target bone includes multiple specified bones, and the multiple specified bones correspond to respective specified axes; the above step 230 can be implemented as step 230a and step 230b.

[0102] Step 230a: Obtain the change speed of the rotation angles of the plurality of designated bones within a time period corresponding to a first animation clip; the first animation clip is any one of the animation clips.

[0103] In an embodiment of the present application, in an animation clip, multiple designated bones may generate movement, and different designated bones correspond to different body parts. In this regard, in an embodiment of the present application, for each designated bone among the multiple designated bones, the computer device can count the change in the rotation angle of the designated bone in the first animation clip, and then divide the counted change by the length of the first animation clip to obtain the change speed of the designated bone in the first animation clip.

[0104] Step 230b: Determine the body part where the designated bone with the fastest change speed among the multiple designated bones is located as the target expression part corresponding to the first animation clip.

[0105] In an embodiment of the present application, for each specified bone among multiple specified bones, the greater the change speed of the specified bone in the first animation clip, the more likely the body part where the specified bone is located is to be the most expressive part in the first animation clip. Therefore, the computer device can determine the body part where the specified bone with the fastest change speed among the multiple specified bones is located as the target expressive part corresponding to the first animation clip.

[0106] The embodiment of the present application provides a technical solution for obtaining the target performance part corresponding to each animation clip in step 230, including obtaining the change speed of the rotation angle of each of the multiple specified bones in each animation clip, and then determining the body part where the specified bone with the largest change speed among the multiple specified bones is located as the target performance part. The embodiment of the present application compares the changes of all bones in each animation clip, and determines the body part where the bone with the most obvious change is located as the target performance part, which can reflect the core action part in each animation clip, can quickly determine the target performance part of each animation clip, ensure the accuracy of the target performance part, and facilitate the generation of subsequent camera motion data.

[0107] Based on the above Figures 2 to 5For any of the embodiments shown, please refer to Figure 6 , which shows a flow chart of a data processing method for camera control provided by another embodiment of the present application. Figure 6 As shown, the above step 240 can be implemented as step 240a, step 230b and step 230c.

[0108] Step 240a: According to the target expression part corresponding to each animation clip, the lens angle corresponding to each animation clip is obtained; the lens angle is used to indicate the range ratio occupied by one or more body parts of the virtual object in the picture captured by the virtual camera.

[0109] The computer device can obtain the lens angle corresponding to each animation clip based on the target expression part corresponding to each animation clip obtained in the above step 230, so as to indicate the distance between the virtual camera and the virtual object and the body part of the virtual object photographed by the virtual camera.

[0110] The lens field of view refers to the difference in the proportion of the range occupied by one or more body parts of the virtual object in the picture of the virtual camera due to the different distances between the virtual camera and the virtual object when the focal length is constant.

[0111] Taking the virtual object being a virtual person as an example, there are five types of camera angles, from near to far, namely close-up (above the shoulders of the virtual person's body), close shot (above the chest of the virtual person's body), medium shot (above the knees of the virtual person's body), panoramic shot (the entire body of the virtual person and part of the surrounding environment), and long shot (the environment in which the virtual person is located).

[0112] In some embodiments, the target performance part includes at least one of the following: upper limbs, lower limbs and whole body; the lens scene includes at least one of the following: close-up, close shot, medium shot, panoramic shot and long shot;

[0113] In the above step 240a, according to the target expression part corresponding to each animation clip, the shot scene corresponding to each animation clip is obtained, including:

[0114] In response to the target expression part corresponding to the second animation clip including the upper limbs, determining that the lens scene corresponding to the second animation clip includes a close-up, a near shot or a medium shot;

[0115] In response to the target expression part corresponding to the second animation clip including lower limbs or the whole body, determining that the lens scene corresponding to the second animation clip includes a panoramic view or a long shot;

[0116] The second animation clip is any one of the animation clips.

[0117] In an embodiment of the present application, for each animation clip, the computer device can determine the shot angle of the current animation clip based on the target performance part, that is, determine whether the current animation clip corresponds to one of close-up, near shot, medium shot, panoramic shot and long shot.

[0118] Among them, when the target performance part includes the upper limbs, the upper limbs of the virtual object need to be photographed from a closer range. At this time, the computer device can determine that the lens scene corresponding to the current animation clip includes close-up, close-up or medium shot.

[0119] For example, taking the target skeleton of a virtual object including the head, upper arms, forearms, thighs, calves, spine, etc., when the target performance part includes the upper limbs, the computer device obtains and determines the corresponding bone range of the target performance part as the upper limbs, thereby determining whether the corresponding shot size of the current animation clip is a close-up, close-up or medium shot.

[0120] Optionally, for example, if it is determined that the corresponding bone range of the target expression part as the upper limbs only includes the head, then the lens angle corresponding to the current animation clip can be a close-up; for another example, if it is determined that the corresponding bone range of the target expression part as the upper limbs includes the upper arm and forearm, then the lens angle corresponding to the current animation clip can be a medium shot.

[0121] When the target performance part includes the lower limbs or the whole body, it is necessary to shoot the entire virtual object from a farther range. At this time, the computer device can determine that the lens scene corresponding to the current animation clip includes a panoramic view or a long shot.

[0122] For example, taking the target skeleton of a virtual object including the head, upper arms, forearms, thighs, calves, spine, etc., when the target expression part includes the lower limbs or the whole body, the computer device obtains and determines the corresponding bone range of the target expression part as the lower limbs or the whole body, so as to determine whether the corresponding shot size of the current animation clip is a panoramic view or a long shot.

[0123] Optionally, for example, if it is determined that the target expression part is the lower limb and the corresponding bone range only includes the calf, then the lens angle corresponding to the current animation clip can be a panoramic view; for another example, if it is determined that the target expression part is the whole body and the corresponding bone range includes the spine and thigh, then the lens angle corresponding to the current animation clip can be a long shot.

[0124] Among them, the above-mentioned strategy of obtaining the lens angle corresponding to each animation clip according to the target expression part corresponding to each animation clip can also be set to other strategies. For example, in response to the target expression part corresponding to the second animation clip including the upper limbs, it is determined that the lens angle corresponding to the second animation clip includes a close-up or a near shot; in response to the target expression part corresponding to the second animation clip including the lower limbs or the whole body, it is determined that the lens angle corresponding to the second animation clip includes a medium shot, a panoramic shot or a long shot.

[0125] The embodiments of the present application provide a classification scheme for target expression parts, a classification scheme for lens views, and a scheme for determining lens views according to target expression parts. The target expression parts include upper limbs, lower limbs, and the whole body, etc. The lens views include close-up, close-up, medium shot, panoramic view, and long shot, etc. The lens views corresponding to the upper limbs can be close-up, close-up, or medium shot, and the lens views corresponding to the lower limbs or the whole body can be panoramic view or long shot. The embodiments of the present application determine the type of lens views based on the target expression parts in each animation clip. The computer device can adjust the different distances between the camera position and the virtual object in each animation clip, as well as the key display parts of the virtual object in each animation clip according to the correct lens views, so as to facilitate the correct setting of subsequent camera parameters and accurately capture the core actions of the virtual object in each animation clip.

[0126] Optionally, there may be other solutions for the above-mentioned classification of the target expression part and the lens angle, and determination of the lens angle according to the target expression part.

[0127] Step 240b: According to the lens angles corresponding to the animation clips, obtain the switching method of the camera position between every two adjacent animation clips in each animation clip.

[0128] The computer device may determine the switching method of the camera position between every two adjacent animation clips in each animation clip based on the lens angles corresponding to each animation clip obtained in the above step 240a.

[0129] Among them, the camera position switching method refers to the motion shot connecting two camera positions. It can be achieved by moving the camera position or changing the focal length of the lens, such as pushing, pulling, turning, shifting, penetrating, following, shaking, and comprehensive motion shots, etc.

[0130] In some embodiments, in the above step 240b, according to the lens angles corresponding to the animation clips, obtaining the switching method of the camera position between every two adjacent animation clips in each animation clip includes:

[0131] In response to the different lens scenes corresponding to the two adjacent animation clips, determining that the camera position switching mode between the two adjacent animation clips is pull-back or push-forward;

[0132] In response to the fact that the lens scenes corresponding to the two adjacent animation clips are the same, it is determined that the switching mode of the camera position between the two adjacent animation clips is panning, translation or circular orbiting.

[0133] In an embodiment of the present application, if the lens angles corresponding to two adjacent animation clips are different, it means that it is necessary to adjust the distance between the camera position and the object model of the virtual object. At this time, it can be determined that the switching method of the camera position between the two adjacent animation clips is to pull back or push forward; if the lens angles corresponding to two adjacent animation clips are the same, it means that it is not necessary to adjust the distance between the camera position and the object model of the virtual object, but it may be necessary to adjust the direction of the camera position relative to the object model of the virtual object. At this time, it can be determined that the switching method of the camera position between the two adjacent animation clips is to pan, translate or circle.

[0134] Among them, panning means that the position of the camera remains stationary, and the camera is rotated in the horizontal direction with the position of the camera as the origin, so that the camera lens moves in an arc.

[0135] Among them, translation refers to moving the camera in the horizontal direction according to a set trajectory so that the camera lens moves in a straight line.

[0136] The circular surround refers to taking the object model of the virtual object as the center surround point, and the camera performs circular surround shooting around the object model of the virtual object, so that the camera lens moves in a circle.

[0137] The embodiment of the present application provides a technical solution for switching camera positions according to the lens angle, which is divided into two situations: the lens angles of two adjacent animation clips are the same or different. The embodiment of the present application ensures the rationality of the camera position switching method, can avoid the screen freeze when switching adjacent animation clips, and thus make the connection between different animation clips smooth.

[0138] Step 240c: Generate camera motion data based on the lens angle, the camera position switching method, and the time points corresponding to each animation clip.

[0139] The computer device can generate camera motion data based on the lens angle corresponding to each animation clip obtained in the above step 240a, the switching method of the camera position between each two adjacent animation clips obtained in the above step 240b, and the time point corresponding to each animation clip obtained in step 220.

[0140] In some embodiments, in the above step 240c, the camera motion data is generated based on the lens view, the camera position switching method, and the time point corresponding to each animation clip, including:

[0141] Based on the lens view, the switching method of the camera position, and the time point corresponding to each animation clip, the spatial position of the virtual camera in the virtual space at each time point is determined; the virtual space contains an object model of the virtual object;

[0142] Based on the spatial position of the virtual camera at each time point and the spatial position of each object model in the virtual space, spatial position relationship information between the virtual camera and each object model at each time point is generated; the spatial position relationship information is used to correct the spatial position of the virtual camera in the virtual space at each time point;

[0143] Camera motion data is generated based on the corrected spatial position of the virtual camera in the virtual space at each time point.

[0144] The computer device can obtain the spatial position of the virtual camera in the virtual space at each time point based on the lens angle corresponding to each animation clip obtained in the above step 240a, the switching method of the camera position between each adjacent animation clip obtained in the above step 240b, and the time point corresponding to each animation clip obtained in step 220, and generate the spatial position relationship information between the virtual camera and each object model at each time point.

[0145] The spatial position of the virtual camera in the virtual space at each time point is corrected according to the spatial position relationship information; and the camera motion data is generated based on the corrected spatial position of the virtual camera in the virtual space at each time point.

[0146] An embodiment of the present application provides a technical solution for generating camera motion data, including detecting the spatial position of a virtual camera in a virtual space and generating spatial position relationship information, and correcting the spatial position of the virtual camera in the virtual space at each time point according to the spatial position relationship information. The camera data generated in this way is corrected, avoiding the overlap of the virtual camera with various object modules or the penetration of the model caused by the close distance, ensuring the rationality of the virtual camera position, improving the aesthetics of the subsequent virtual camera acquisition pictures, and thereby improving the efficiency of generating camera motion data.

[0147] In some embodiments, before generating the camera motion data based on the corrected spatial position of the virtual camera in the virtual space at each time point, the method further includes:

[0148] The spatial position relationship information is output to the spatial position correction interface so that the user can correct the spatial position of the virtual camera in the virtual space at each time point in the spatial position correction interface based on the spatial position relationship information.

[0149] Before the computer device generates the camera motion data, the spatial position relationship information can be output to the spatial position correction interface, so that the user can correct the spatial position of the virtual camera in the virtual space at each time point in the spatial position correction interface.

[0150] An embodiment of the present application provides a correction scheme for the spatial position of a virtual camera in a virtual space. The computer device outputs the spatial position relationship information generated above to a spatial position correction interface. The user can manually adjust the spatial position of the virtual camera in the virtual space in the spatial position correction interface to achieve the user's expected visual effect. The manual correction method is retained to provide personalized services for the user.

[0151] In some embodiments, before generating the camera motion data based on the corrected spatial position of the virtual camera in the virtual space at each time point, the method further includes:

[0152] In response to the spatial position relationship information indicating that the distance between the virtual camera and the first object model at the third time point is less than a distance threshold, adjusting the distance between the virtual camera and the first object model at the third time point so that the distance between the virtual camera and the first object model at the third time point is not less than the distance threshold; the first object model is an object model of the virtual object; the third time point is any time point among the time points;

[0153] In response to the spatial position relationship information indicating that the virtual camera collides with the second object model at a fourth time point, the spatial position of the virtual camera at the fourth time point is adjusted so that no collision occurs between the virtual camera and the second object model at the fourth time point; the second object model is an object model of an object other than the virtual object; the fourth time point is any time point among all time points.

[0154] The computer device may adjust the spatial position of the virtual camera before generating the camera motion data based on the corrected spatial position of the virtual camera in the virtual space at each time point.

[0155] On the one hand, the computer device can set a minimum distance between the virtual camera and the object model of the virtual object at each time point to avoid penetration of the virtual camera and the object model of the virtual object. When the distance between the virtual camera and the object model of the virtual object is less than the minimum distance at a certain time point, the distance between the virtual camera and the object model of the virtual object at the time point is adjusted so that the distance between the virtual camera and the object model of the virtual object at the time point is greater than or equal to the minimum distance.

[0156] On the other hand, the computer device can perform collision detection on the virtual camera and the object models of other objects other than the virtual object at each time point. When the virtual camera collides with the object models of other objects other than the virtual object at a certain time point, the spatial position of the virtual camera at that time point is adjusted so that the virtual camera does not collide with the object models of other objects other than the virtual object at that time point.

[0157] In an embodiment of the present application, the maximum distance that a virtual camera can move can be determined by detecting the distance between an object model of a virtual object and a boundary (such as a wall) of a virtual space by ray detection. Specifically, for example, ray detection is performed based on the object model of a virtual object, and the distances between the object model of the virtual object and the boundary (such as a wall) of a virtual space in six directions, namely, front, back, left, right, top, and bottom, are obtained respectively, and the distances in the six directions are respectively set as the maximum distances that a virtual camera can move in the corresponding six directions.

[0158] During the movement of the virtual camera, if a collision occurs with the object model A in front of the virtual object, the farthest distance in front of the virtual object needs to be reduced to the distance between the object model of the virtual object and the object model A.

[0159] The embodiment of the present application provides another correction scheme for the spatial position of a virtual camera in a virtual space. The computer device automatically responds and corrects the spatial position of the virtual camera in the virtual space to achieve high efficiency in correcting the spatial position and improve the effect of subsequent virtual camera capture images.

[0160] In summary, the embodiment of the present application obtains the lens view corresponding to each animation clip according to the target expression part corresponding to each animation clip obtained in step 230, thereby obtaining the switching method of the camera position between each adjacent animation clip in each animation clip, and generating camera motion data. The lens view of a single animation clip is determined by the target expression part of a single animation clip, and then the connecting lens of two adjacent animation clips is determined by the switching method, and finally the camera motion data with coherence, aesthetics and accuracy is generated, which can improve the generation efficiency of camera motion data, and then improve the generation efficiency of the video of the virtual object.

[0161] Based on the scheme shown in the above embodiment of the present application, dance information can be used to automatically control the movement of the virtual camera, and this scheme has a wide range of application scenarios. When rendering virtual human assets in a purely virtual environment such as virtual human performances, live broadcasts, or CG animations, there is often a demand for a large number of dance action videos. It is necessary to create beautiful, reasonable, and diverse virtual camera movements based on elements such as dance and action to show the beautiful dance movements of the virtual human.

[0162] The work of motion capture or manual setting for the movement of each lens is often repetitive and monotonous, which consumes a lot of manpower. In this case, the solution shown in the above application can greatly improve the efficiency of setting the camera motion data.

[0163] Please refer to Figure 7 , which shows a flow chart of a method for creating a shot animation based on virtual human skeleton dance animation data provided by an embodiment of the present application. Figure 7As shown, the method adopts the following steps:

[0164] S71 Dance data analysis: Analyze the virtual human skeleton dance animation data to extract key movements and posture information. Determine the appropriate camera switching time and angle by analyzing the rhythm and movement characteristics of the dance animation;

[0165] S72 Shot Planning: Based on the results of data analysis, the algorithm and rules are used to automatically plan the shot animation. Considering the importance and fluidity of the sense of rhythm and dance movements, a shot animation plan with reasonable beauty and rich camera movements is generated;

[0166] S73 lens switching: Automatically switch and convert lenses according to the lens planning scheme. By adjusting the position, angle, focal length and other parameters of the camera, a smooth transition between lenses is achieved to ensure the continuity of visual effects;

[0167] Through the above steps, a rich and available camera animation sequence is generated, and the camera movement sequence creation of the virtual human dance video is completed quickly and automatically.

[0168] Please refer to Figure 8 , which shows a schematic diagram of an implementation of a data processing method for camera control provided by an embodiment of the present application. Taking the creation of a dance video based on the skeleton dance animation data of a virtual person as an example, Figure 8 As shown, the implementation steps include the following.

[0169] S810, obtaining skeleton dance animation data of a virtual human.

[0170] The computer device obtains the skeleton dance animation data of the virtual person uploaded by the user, and obtains the skeleton animation sequence of the virtual person at the complete time point.

[0171] S820, preprocessing.

[0172] The computer device performs coordinate conversion on the skeleton animation sequence acquired in step 810, and converts the coordinates of the skeleton animation sequence into a unified coordinate system to facilitate subsequent data analysis and processing.

[0173] S830, extract target bone data.

[0174] The computer device extracts the target bone data according to the bone animation sequence after the coordinate conversion in step 820; wherein the target bone data includes the rotation angle of the target bone of the virtual person on the specified axis at each time point.

[0175] In the embodiment of the present application, in the skeleton animation sequence of the virtual person, many skeletons are involved in the dance movements, but not every skeleton data is of great significance to the analysis of the movements. Therefore, some important skeleton data can be selected for analysis, i.e., target skeletons, such as skeleton animation sequences corresponding to the thigh, calf, forearm, head, and spine.

[0176] In addition, according to the characteristics of the selected bones and the characteristics of the human body when performing actions, the value of a certain axis where the action is obvious can be selected for extraction. By selecting data on a specific axis, the characteristics and changes of the action can be analyzed more accurately.

[0177] For example, the common leg movements of the thigh are kicking in the front-to-back direction, followed by lifting the leg in the left-to-right direction, and there is almost no rotational movement around the middle axis of the leg bone. Therefore, for the thigh bone, the bone animation sequence on the axis where the thigh bone is located in the front-to-back direction can be selected for analysis.

[0178] For example, the common spinal movements are bending or stretching in the front-to-back direction, followed by lateral pulling in the left-right direction. Therefore, for the spinal bones, you can select the skeletal animation sequence on the axis where the spine is located in the front-to-back direction for analysis.

[0179] S840, smoothing the target bone data.

[0180] The computer device may perform smoothing processing on the target bone data acquired in step 830 .

[0181] Smoothing can eliminate noise and outliers in the target bone data, making subsequent analysis more accurate and reliable; at the same time, the smoothed bone data can more accurately extract the characteristics and performance of virtual human dance.

[0182] S850, segment the target skeleton data.

[0183] The computer device obtains action segmentation points based on the target skeleton data extracted in step 830 and the skeleton data after smoothing in step 840; based on the action segmentation points, the target skeleton data is segmented into multiple animation segments.

[0184] Optionally, the computer device can perform mutation detection on the bone data after smoothing in step 840 to detect the first time point at which the rotation angle suddenly changes; obtain the second time point from the target bone data extracted in step 830; and divide the target bone data into multiple animation segments based on each second time point.

[0185] The second time point is a time point in the target skeleton data that is before the first time point and is closest to the first time point and at which a sudden change in the rotation angle occurs.

[0186] S860, determining the target presentation part of each animation clip.

[0187] The computer device obtains the change speed of the rotation angle of each of the multiple specified bones within the time period corresponding to each animation segment based on the animation segment segmented in step 850;

[0188] The body part where the designated bone with the fastest rotation angle change rate is located is determined as the target expression part corresponding to each animation clip.

[0189] For ease of handling, the core performance areas can be divided into three parts: upper limbs, lower limbs and whole body.

[0190] S870, obtaining the camera angle corresponding to the animation clip, and the switching method of the camera angles corresponding to two adjacent animation clips.

[0191] The computer device obtains the lens scene corresponding to each animation clip according to the target expression part corresponding to each animation clip obtained in step 860;

[0192] Then, according to the lens angles corresponding to each animation clip, the switching method of the camera position between every two adjacent animation clips in each animation clip is obtained.

[0193] The lens field of view is used to indicate the proportion of the range occupied by one or more body parts of the virtual object in the picture captured by the virtual camera.

[0194] For example, if the target expression part corresponding to the current animation clip includes upper limbs, then it can be determined that the lens angle corresponding to the current animation clip is medium shot.

[0195] For another example, if the target expression part corresponding to the next adjacent animation clip of the current animation clip includes the whole body, then it can be determined that the shot type corresponding to the next adjacent animation clip is a panoramic view.

[0196] At this time, if the current animation clip is switched to the next adjacent animation clip, it can be determined that the camera position is switched in a pull-back manner.

[0197] S880, spatial position detection.

[0198] The computer device may preliminarily create a virtual camera, and based on the lens view of step 870, the switching method of step 870, and the time points corresponding to the animation clips, detect the spatial position of the virtual camera in the virtual space at each time point; wherein the virtual space includes an object model of a virtual person;

[0199] Optionally, the computer device can detect the position of the virtual person in the virtual space. By detecting the position of the object model of the virtual person, we can determine the specific position coordinates of the virtual person in the virtual space.

[0200] Next, the computer device can detect the distance between the virtual camera and the virtual person. This is to ensure that the virtual camera is not too close to the virtual person, which may cause the phenomenon of penetration. By setting a suitable distance threshold, it can be determined whether the distance between the camera and the virtual person is appropriate. If the distance is detected to be inappropriate, the movement trajectory of the virtual camera should be adjusted in time to avoid producing an undesirable effect.

[0201] In addition, during the switching process of the virtual camera, it is also necessary to detect whether the virtual camera collides with other models. This is to avoid the phenomenon of objects penetrating the model when the virtual camera moves. By detecting the collision between the virtual camera and other models, we can adjust the movement trajectory of the virtual camera in time to avoid undesirable effects.

[0202] S890, generating camera motion data, capturing images of virtual objects, and obtaining dance videos.

[0203] Based on the above steps, the computer device can generate reasonable camera motion data, and set the camera position of the virtual camera at each time point based on the camera motion data; the virtual camera can capture the corresponding dance pictures of the virtual person at each time point to obtain the dance video performed by the virtual person.

[0204] Please refer to Fig. 9 , which shows a flowchart of a method for generating a shot sequence based on virtual human skeleton dance animation data according to an embodiment of the present application, such as Fig. 9 As shown, it mainly includes a user input module 901, an action segmentation and analysis module 902 and a shot sequence generation module 903.

[0205] After segmenting and analyzing the movements of the virtual human, it is necessary to process the spatial relationship between the human and the character. First, it is necessary to detect the position of the human in space and the distance between the camera and the character to ensure that the camera is not too close to the virtual human and causes penetration. Then, it is necessary to detect whether the camera collides with other models during movement to avoid the phenomenon of objects penetrating the model when the camera is moving.

[0206] After obtaining the results of the previous action analysis, the camera is automatically created based on the results of the spatial character relationship processing. According to the parts and speed of the action clips, the camera's motion parameters are automatically set to produce movements that match the characteristics of the virtual human's dance movements.

[0207] 1. Action segmentation and analysis module 902

[0208] (1) Skeletal animation processing

[0209] ① Action data preprocessing: This step mainly involves preprocessing the skeletal animation data, including the conversion of skeletal animation coordinates. By converting the skeletal animation coordinates into a unified coordinate system, subsequent data analysis and processing are facilitated.

[0210] ② Select key or important bone data for analysis: In skeletal animation, many bones are involved in the action, but not every bone's data is important for the analysis of the action. Therefore, in this step, you can select some important bone data for analysis, including the data of the thigh, calf, forearm, head and spine.

[0211] ③ Axis selection: According to the characteristics of the selected bones and the characteristics of the human body when performing actions, you can select the value of a certain axis where the action is obvious for extraction. By selecting data on a specific axis, you can more accurately analyze the characteristics and changes of the action. For example, for the thigh bone, please refer to Fig.10 , which shows a schematic diagram of a thigh bone involved in an embodiment of the present application. Fig.10 As shown in the figure, the common leg movements are mainly kicking in the front-back direction, followed by lifting the legs left and right, and there is almost no rotational movement around the middle axis of the leg bone. Therefore, the data on the axis where the front-back direction of the leg bone is located in the engine is selected for analysis.

[0212] ④ Smoothing: Smoothing the selected axial data. Please refer to Fig.11 , which shows a schematic diagram of the original data and smoothed data involved in one embodiment of the present application. Fig.11 As shown, the horizontal axis represents the time point, and the vertical axis represents the rotation angle of the specified bone on the specified axis. Smoothing can eliminate noise and outliers in the data, making subsequent analysis more accurate and reliable. At the same time, the smoothed data can more accurately extract the characteristics and performance of the movement.

[0213] (2) Motion data analysis

[0214] ① Mutation detection: Take one of the selected bones as an example, please refer to Fig.12 , which shows a schematic diagram of the difference between the manual recognition involved in one embodiment of the present application and the action segmentation point of the embodiment of the present application. Fig.12As shown, the abscissa represents the time point, and the ordinate represents the rotation angle on the specified axis of the specified bone. First, the motion data that has been smoothed is subjected to mutation detection, and the time point at which the tangent suddenly increases or decreases can be found, and the mutation time point can be found on the original data. Through the increase or decrease of the mutation, the specific time of the previous peak or trough of the mutation time point can be determined, thereby obtaining the time point (i.e., turning point) at which the skeletal motion suddenly mutates. The skeletal motion is divided into several fragments according to the time point at which the skeletal motion suddenly mutates. Other selected bones are similar and will not be repeated here.

[0215] ② Mutation speed comparison: Next, you can compare the mutation speed of all selected bones between two time points. Please refer to Fig.13 , which shows a schematic diagram of the performance part of a dance segment involved in an embodiment of the present application. Fig.13 As shown in the figure, 6 key frame time points are selected as examples, and the weights in a single segment are compared. By comparing the mutation speed, the core performance parts of the dance movements in the time period between two time points can be obtained. The core performance parts are summarized into three parts: upper limbs, lower limbs and whole body.

[0216] ③Continuity analysis: After completing the analysis of the performance parts of a single dance segment, it is necessary to conduct continuity analysis or connection analysis on the performance parts before and after the segment. By analyzing the continuity of the performance parts, the performance content and performance parts of each dance movement can be obtained, as well as all the performance parts on the timeline from the starting point to the end point.

[0217] 2. Generate shot sequence module 903

[0218] (1) Processing of spatial character relationships

[0219] ① Position detection: First, it is necessary to detect the position of the person in space. By detecting the position of the virtual human model, its specific position coordinates in the scene can be determined.

[0220] ② Camera distance detection: Next, the distance between the camera and the virtual human needs to be detected. This is to ensure that the camera is not too close to the virtual human, which may cause the phenomenon of penetration. By setting a suitable distance threshold, it can be determined whether the distance between the camera and the virtual human is appropriate.

[0221] ③ Collision detection: During the movement of the camera, it is also necessary to detect whether the camera collides with other models. This is to avoid the phenomenon of objects penetrating the model when the camera moves. If the possibility of collision between the camera and other models is detected, the movement trajectory of the camera can be adjusted in time to avoid undesirable effects.

[0222] (2) Camera creation and parameter setting

[0223] After obtaining the results of the previous action analysis and integrating the results of the spatial character relationship processing, the computer device can automatically create a camera and make camera lens decisions and corrections, including single lens decisions, motion lens planning corrections, and motion continuity corrections, thereby generating a camera motion sequence. Please refer to Fig.14 , which shows a schematic diagram of the camera position of a dance segment involved in an embodiment of the present application. Fig.14 As shown, if the core performance part of a dance clip is the upper limbs, then the fixed camera position of the dance clip can be a close-up; if the core performance part of the next dance clip of the dance clip is the whole body, then the fixed camera position of the next dance clip can be a panoramic view. The movement camera position switching from the dance clip to the next dance clip can be a pull-back. According to the performance part of the action clip and the speed of the action, the movement parameters of the camera are automatically set to produce a movement that conforms to the characteristics of the virtual human dance action.

[0224] Please refer to Fig.15 , which shows an interface diagram of a shot sequence based on virtual human skeleton dance animation data provided by an embodiment of the present application.

[0225] This method aims to use virtual human skeleton dance animation data to automatically create reasonable and beautiful lens animation with rich camera movements. It mainly focuses on pure virtual or virtual scene-based environments such as virtual human performances, live broadcasts or CG animations.

[0226] The embodiment of the present application provides a method for automatically creating shot animation based on virtual human skeleton dance animation data, so as to quickly and automatically complete the creation of a virtual human dance video that is reasonably beautiful and has rich camera movements.

[0227] The solution shown in the embodiment of the present application can be implemented to quickly set the camera position according to the virtual human model in virtual human performances, live broadcasts or CG animations to complete the design of different scene shots. This solution takes into account the structural composition mode and skeleton hierarchy structure of various virtual human objects under the three-dimensional real-time engine, and can automatically calculate the scene for various virtual human asset contents such as those that comply with the skeleton resource rules or those that do not comply with the skeleton resource rules, and quickly perform virtual camera lens framing operations.

[0228] The technical solution of the embodiment of the present application can produce the following beneficial effects:

[0229] 1. Improve dance performance

[0230] By analyzing the key parts, speed and rhythm of dance movements, the virtual camera can be automatically controlled to better highlight the wonderful moments and performance points of dance movements. The movement trajectory and angle of the camera can be coordinated with the dance movements, thereby enhancing the dance performance and making the audience more immersed in the art of dance.

[0231] 2. Enhance viewing experience

[0232] Using dance information to automatically control the movement of the virtual camera can provide the audience with a richer and more exciting viewing experience. Through reasonable camera movement, the audience can appreciate the dance movements from different angles and distances, and feel the dancer's power, elegance and expression. This immersive viewing experience can enhance the audience's sense of participation and emotional resonance, making the audience more engaged and enjoying the artistic charm of dance.

[0233] 3. Simplify the production process

[0234] Traditionally, making dance videos requires a lot of manpower and time to plan and adjust the camera movement. The method of automatically controlling the virtual camera movement using dance information can automatically generate camera movement trajectories and parameters suitable for dance movements, thereby simplifying the production process. This can save the production team's time and energy, improve production efficiency, and reduce production costs.

[0235] In summary, the method of automatically controlling the movement of a virtual camera using dance information can improve dance performance, enhance viewing experience, simplify the production process, and increase creative freedom.

[0236] Fig.16 A block diagram of a data processing device for camera control according to an exemplary embodiment of the present application is shown. The device can be used to perform the following steps: Figure 2 , Figure 3 , Figure 4 , Figure 5 or Figure 6 In the method shown, the device includes:

[0237] The sequence acquisition module 1601 is used to acquire the skeleton animation sequence of the virtual object; the skeleton animation sequence includes the animation of the skeleton action of the virtual object at each time point;

[0238] The segmentation module 1602 is used to segment the skeleton animation sequence into multiple animation segments based on the action segmentation point; the action segmentation point is the time point when the skeleton action of the virtual object suddenly changes;

[0239] The part acquisition module 1603 is used to acquire the target performance part corresponding to each animation clip; the target performance part is the body part corresponding to the skeleton action that undergoes key changes in the animation clip;

[0240] The data acquisition module 1604 is used to obtain camera motion data according to the target expression part corresponding to each animation clip and the time point corresponding to each animation clip; the camera motion data is used to indicate the camera position of the virtual camera at each time point; the camera motion data is used to control the virtual camera for shooting the virtual object executing the skeletal animation sequence.

[0241] In some embodiments, the apparatus further comprises:

[0242] The extraction module is used to extract target bone data from the bone animation sequence; the target bone data includes the rotation angle of the target bone of the virtual object on the specified axis at each time point;

[0243] Based on the target skeleton data, obtain the action segmentation points.

[0244] In some embodiments, the extraction module is used to:

[0245] Smoothing the target bone data to obtain smoothed bone data;

[0246] Detecting the first time point at which the rotation angle suddenly changes from the smoothed skeleton data;

[0247] Acquire a second time point from the target bone data; a time point in the target bone data at the second time point, which is before the first time point and closest to the first time point, at which a sudden change in rotation angle occurs;

[0248] An action segmentation point is obtained based on the second time point.

[0249] In some embodiments, the target skeleton includes a plurality of specified skeletons, and the plurality of specified skeletons correspond to respective specified axes; the part acquisition module 1603 is used to:

[0250] Obtaining the change speed of the rotation angles of the plurality of specified bones in the time period corresponding to the first animation clip; the first animation clip is any one of the animation clips;

[0251] The body part where the designated bone with the fastest change speed among the multiple designated bones is located is determined as the target expression part corresponding to the first animation clip.

[0252] In some embodiments, the data acquisition module 1604 is used to:

[0253] According to the target expression parts corresponding to each animation clip, the lens view corresponding to each animation clip is obtained; the lens view is used to indicate the range ratio occupied by one or more body parts of the virtual object in the picture captured by the virtual camera;

[0254] According to the lens scenes corresponding to each animation clip, a switching method of the camera position between each adjacent animation clip in each animation clip is obtained;

[0255] Camera motion data is generated based on the lens angle, the camera position switching method, and the time points corresponding to each animation clip.

[0256] In some embodiments, the target performance part includes at least one of the following: upper limbs, lower limbs and whole body; the lens scene includes at least one of the following: close-up, close shot, medium shot, panoramic shot and long shot;

[0257] According to the target performance parts corresponding to each animation clip, the corresponding lens scene of each animation clip is obtained, including:

[0258] In response to the target expression part corresponding to the second animation clip including the upper limbs, determining that the lens scene corresponding to the second animation clip includes a close-up, a near shot or a medium shot;

[0259] In response to the target expression part corresponding to the second animation clip including lower limbs or the whole body, determining that the lens scene corresponding to the second animation clip includes a panoramic view or a long shot;

[0260] The second animation clip is any one of the animation clips.

[0261] In some embodiments, according to the lens scenes corresponding to the animation clips, obtaining the switching method of the camera position between every two adjacent animation clips in the animation clips includes:

[0262] In response to the different lens scenes corresponding to the two adjacent animation clips, determining that the camera position switching mode between the two adjacent animation clips is pull-back or push-forward;

[0263] In response to the fact that the lens scenes corresponding to the two adjacent animation clips are the same, it is determined that the switching mode of the camera position between the two adjacent animation clips is panning, translation or circular orbiting.

[0264] In some embodiments, based on the lens view, the camera position switching method, and the time point corresponding to each animation clip, the camera motion data is generated, including:

[0265] Based on the lens view, the switching method of the camera position, and the time point corresponding to each animation clip, the spatial position of the virtual camera in the virtual space at each time point is determined; the virtual space contains an object model of the virtual object;

[0266] Based on the spatial position of the virtual camera at each time point and the spatial position of each object model in the virtual space, spatial position relationship information between the virtual camera and each object model at each time point is generated; the spatial position relationship information is used to correct the spatial position of the virtual camera in the virtual space at each time point;

[0267] Camera motion data is generated based on the corrected spatial position of the virtual camera in the virtual space at each time point.

[0268] In some embodiments, before generating the camera motion data based on the corrected spatial position of the virtual camera in the virtual space at each time point, the method further includes:

[0269] The spatial position relationship information is output to the spatial position correction interface so that the user can correct the spatial position of the virtual camera in the virtual space at each time point in the spatial position correction interface based on the spatial position relationship information.

[0270] In some embodiments, before generating the camera motion data based on the corrected spatial position of the virtual camera in the virtual space at each time point, the method further includes:

[0271] In response to the spatial position relationship information indicating that the distance between the virtual camera and the first object model at the third time point is less than a distance threshold, adjusting the distance between the virtual camera and the first object model at the third time point so that the distance between the virtual camera and the first object model at the third time point is not less than the distance threshold; the first object model is an object model of the virtual object; the third time point is any time point among the time points;

[0272] In response to the spatial position relationship information indicating that the virtual camera collides with the second object model at a fourth time point, the spatial position of the virtual camera at the fourth time point is adjusted so that no collision occurs between the virtual camera and the second object model at the fourth time point; the second object model is an object model of an object other than the virtual object; the fourth time point is any time point among all time points.

[0273] In some embodiments, the apparatus further comprises:

[0274] A virtual camera creation module is used to create a virtual camera based on camera motion data and set the camera position of the virtual camera at each time point during the process of the virtual object executing the skeleton animation sequence;

[0275] The target video acquisition module is used to acquire the target video by capturing the images of the first object model corresponding to the virtual object at each time point through a virtual camera.

[0276] Fig.17The structural block diagram of a computer device 1700 shown in an exemplary embodiment of the present application is shown. The computer device can be implemented as a server in the above-mentioned solution of the present application. The computer device 1700 includes a central processing unit (CPU) 1701, a system memory 1704 including a random access memory (RAM) 1702 and a read-only memory (ROM) 1703, and a system bus 1705 connecting the system memory 1704 and the central processing unit 1701. The computer device 1700 also includes a mass storage device 1706 for storing an operating system 1709, an application program 1710 and other program modules 1711.

[0277] The mass storage device 1706 is connected to the central processing unit 1701 through a mass storage controller (not shown) connected to the system bus 1705. The mass storage device 1706 and its associated computer readable medium provide non-volatile storage for the computer device 1700. That is, the mass storage device 1706 may include a computer readable medium (not shown) such as a hard disk or a Compact Disc Read-Only Memory (CD-ROM) drive.

[0278] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, Erasable Programmable Read Only Memory (EPROM), Electronically-Erasable Programmable Read-Only Memory (EEPROM) flash memory or other solid-state storage technology, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, cassettes, tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will know that the computer storage medium is not limited to the above. The above-mentioned system memory 1704 and mass storage device 1706 can be collectively referred to as memory.

[0279] According to various embodiments of the present disclosure, the computer device 1700 can also be connected to a remote computer on the network through a network such as the Internet. That is, the computer device 1700 can be connected to the network 1708 through the network interface unit 1707 connected to the system bus 1705, or the network interface unit 1707 can be used to connect to other types of networks or remote computer systems (not shown).

[0280] The memory also includes at least one computer program, which is stored in the memory. The central processing unit 1701 implements all or part of the steps in the methods shown in the above embodiments by executing the at least one computer program.

[0281] In an exemplary embodiment, a computer-readable storage medium is also provided, which is used to store at least one computer program, and the at least one computer program is loaded and executed by a processor to implement all or part of the steps in the methods shown in the above embodiments. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0282] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes all or part of the steps in the methods shown in the above embodiments.

[0283] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0284] It should be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A data processing method for camera control, characterized in that: The method comprises: Acquire a skeleton animation sequence of a virtual object; the skeleton animation sequence includes animations of skeleton movements of the virtual object at various time points; The skeleton animation sequence is divided into a plurality of animation segments based on an action segmentation point; the action segmentation point is a time point at which a sudden change occurs in the skeleton action of the virtual object; Acquire the target performance part corresponding to each of the animation clips; the target performance part is the body part corresponding to the skeletal action that undergoes key changes in the animation clips; Camera motion data is obtained according to the target expression part corresponding to each of the animation clips and the time point corresponding to each of the animation clips; the camera motion data is used to indicate the camera position of the virtual camera at each time point; the camera motion data is used to control the virtual camera for shooting the virtual object executing the skeletal animation sequence.

2. The method according to claim 1, characterized in that Before dividing the skeleton animation sequence into a plurality of animation clips based on the action segmentation points, the method further includes: Extracting target bone data from the bone animation sequence; the target bone data includes the rotation angle of the target bone of the virtual object on a specified axis at each time point; Based on the target skeleton data, the action segmentation points are obtained.

3. The method according to claim 2, characterized in that The step of obtaining the action segmentation point based on the target skeleton data includes: Smoothing the target bone data to obtain smoothed bone data; Detecting the first time point at which a sudden change in the rotation angle occurs from the smoothed skeleton data; Acquire a second time point from the target skeleton data; at the second time point, a time point where a rotation angle in the target skeleton data that is before the first time point and closest to the first time point undergoes a sudden change; The action segmentation point is acquired based on the second time point.

4. The method according to claim 2 or 3, characterized in that: The target skeleton includes a plurality of designated skeletons, and the plurality of designated skeletons correspond to the respective designated axes; The step of obtaining the target expression part corresponding to each of the animation clips includes: Obtaining the change speed of the rotation angles of the plurality of designated bones within a time period corresponding to a first animation clip; the first animation clip is any one of the animation clips; The body part where the designated bone with the fastest changing speed among the multiple designated bones is located is determined as the target expression part corresponding to the first animation clip.

5. The method according to any one of claims 1 to 4, characterized in that: The acquiring of camera motion data according to the target expression part corresponding to each of the animation clips and the time point corresponding to each of the animation clips includes: According to the target expression parts corresponding to the animation clips, the lens angle corresponding to the animation clips is obtained; the lens angle is used to indicate the range ratio occupied by one or more body parts of the virtual object in the picture captured by the virtual camera; According to the lens scenes corresponding to the animation clips, a switching method of the camera position between every two adjacent animation clips in the animation clips is obtained; The camera motion data is generated based on the lens view, the switching method of the camera position, and the time points corresponding to each of the animation clips.

6. The method according to claim 5, characterized in that The target expression part includes at least one of the following: upper limbs, lower limbs and whole body; the lens scene includes at least one of the following: close-up, close-up, medium shot, panoramic shot and long shot; The step of obtaining the lens scene corresponding to each of the animation clips according to the target expression part corresponding to each of the animation clips comprises: In response to the target expression part corresponding to the second animation clip including an upper limb, determining that the shot type corresponding to the second animation clip includes a close-up, a near shot, or a medium shot; In response to the target expression part corresponding to the second animation clip including lower limbs or the whole body, determining that the lens scene corresponding to the second animation clip includes a panoramic view or a long shot; The second animation clip is any one of the animation clips.

7. The method according to claim 6, characterized in that The step of obtaining the switching method of the camera position between each adjacent two animation clips in each animation clip according to the lens scene corresponding to each animation clip comprises: In response to the lens scenes corresponding to two adjacent animation clips being different, determining that the camera position switching mode between the two adjacent animation clips is pull-back or push-forward; In response to the lens scenes corresponding to two adjacent animation clips being the same, it is determined that the switching mode of the camera position between the two adjacent animation clips is panning, translation or circular orbiting.

8. The method according to claim 5, characterized in that The generating the camera motion data based on the lens scene, the switching mode of the camera position, and the time point corresponding to each of the animation clips includes: Based on the lens view, the switching mode of the camera position, and the time points corresponding to the animation clips, determining the spatial position of the virtual camera in the virtual space at each time point; the virtual space contains an object model of the virtual object; Based on the spatial position of the virtual camera at each time point and the spatial position of each object model in the virtual space, generating spatial position relationship information between the virtual camera and each object model at each time point; the spatial position relationship information is used to correct the spatial position of the virtual camera in the virtual space at each time point; The camera motion data is generated based on the corrected spatial position of the virtual camera in the virtual space at each time point.

9. The method according to claim 8, characterized in that Before generating the camera motion data based on the corrected spatial position of the virtual camera in the virtual space at each time point, the method further includes: The spatial position relationship information is output to a spatial position correction interface so that the user can correct the spatial position of the virtual camera in the virtual space at each time point in the spatial position correction interface based on the spatial position relationship information.

10. The method according to claim 8, characterized in that Before generating the camera motion data based on the corrected spatial position of the virtual camera in the virtual space at each time point, the method further includes: In response to the spatial position relationship information indicating that the distance between the virtual camera and the first object model at a third time point is less than a distance threshold, adjusting the distance between the virtual camera and the first object model at the third time point so that the distance between the virtual camera and the first object model at the third time point is not less than the distance threshold; the first object model is an object model of the virtual object; the third time point is any time point among the various time points; In response to the spatial position relationship information indicating that the virtual camera collides with the second object model at a fourth time point, the spatial position of the virtual camera at the fourth time point is adjusted so that no collision occurs between the virtual camera and the second object model at the fourth time point; the second object model is an object model of an object other than the virtual object; and the fourth time point is any time point among the various time points.

11. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: During the process of the virtual object executing the skeletal animation sequence, creating the virtual camera based on the camera motion data, and setting the camera position of the virtual camera at each time point; The virtual camera captures images of the first object model corresponding to the virtual object at each time point to obtain a target video.

12. A data processing device for camera control, characterized in that: The device comprises: A sequence acquisition module is used to acquire a skeleton animation sequence of a virtual object; the skeleton animation sequence includes animations of skeleton movements of the virtual object at various time points; A segmentation module, used for segmenting the skeleton animation sequence into a plurality of animation segments based on an action segmentation point; the action segmentation point is a time point at which a sudden change occurs in the skeleton action of the virtual object; A part acquisition module, used to acquire a target performance part corresponding to each of the animation clips; the target performance part is a body part corresponding to a skeletal action that undergoes a key change in the animation clip; A data acquisition module is used to acquire camera motion data according to the target expression parts corresponding to each of the animation clips and the time points corresponding to each of the animation clips; the camera motion data is used to indicate the camera position of the virtual camera at each time point; the camera motion data is used to control the virtual camera for shooting the virtual object executing the skeletal animation sequence.

13. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the data processing method for camera control according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: At least one computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by the processor to implement the data processing method for camera control according to any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; the computer instructions are read and executed by a processor of a computer device to implement the data processing method for camera control as described in any one of claims 1 to 11.

Images