Data processing apparatus and method

Abstract

A data processing apparatus comprising circuitry configured to: obtain movement data defining movement of a generic virtual character having one or more generic characteristics; obtain one or more specific characteristics of a specific virtual character, the one or more specific characteristics being different to corresponding ones of the one or more generic characteristics; adjust the movement data according to the one or more specific characteristics; and generate video data representing a video of the specific virtual character performing movement defined by the adjusted movement data.

Description

[0001] DATA PROCESSING APPARATUS AND METHOD

[0002] BACKGROUND

[0003] Field of the Disclosure

[0004] The present disclosure relates to a data processing apparatus and method.

[0005] Description of the Related Art

[0006] The “background” description provided is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in the background section, as well as aspects of the description which may not otherwise qualify as prior art at the time of filing, are neither expressly or impliedly admitted as prior art against the present disclosure.

[0007] Video tutorials have become increasingly popular due to them allowing users to learn or improve a skill from a remote location without having to meet an in-person trainer. This provides an easier, more flexible way for users to learn at low cost.

[0008] A problem, however, is that, unlike with an in-person trainer, a video tutorial is not able to provide personalised feedback and / or adapt the training based on a person’s individual characteristics.

[0009] For example, for learning a sport, different people have different characteristics (e.g. height, weight, historical injuries and the like) which may change the optimal technique for each person in performing certain actions associated with the sport (e.g. the most appropriate running form, how to safely and effectively throw a ball, etc.). A generic video tutorial, however, will only teach a way of performing the action which is appropriate to a person with one set of characteristics (i.e. those of the trainer in the video). People with a different set of characteristics thus may learn a non-optimal technique or, worse, may injure themselves by trying to perform the action in a way not appropriate for them.

[0010] There is thus a desire for video tutorials to be improved.

[0011] SUMMARY

[0012] The present disclosure is defined by the claims.

[0013] BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Non-limiting embodiments and advantages of the present disclosure are explained with reference to the following detailed description taken in conjunction with the accompanying drawings, wherein: Fig. 1 schematically shows an example data processing apparatus;

[0015] Fig. 2 schematically shows a process for creating an output video;

[0016] Fig. 3 schematically shows an example of controlling a 3D avatar based on movements of a general agent;

[0017] Fig. 4 schematically shows an example of non-adjusted movement of a general agent;

[0018] Fig. 5 schematically shows an example of an interactive screen;

[0019] Fig. 6 schematically shows an example of adjusted movement of applicable to a 3D avatar; Figs. 7A-C schematically show an example divergence between an intended pose and an actual pose;

[0020] Figs. 8A-C schematically show an example comparison between groups of joints;

[0021] Fig. 9 schematically shows an example feedback screen; and

[0022] Fig. 10 shows an example method.

[0023] Like reference numerals designate identical or corresponding parts throughout the drawings.

[0024] DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] Fig. 1 shows a data processing apparatus 100 according to an example of the present technology.

[0026] The data processing apparatus 100 comprises a processor 101 for executing electronic instructions, a memory 102 (e.g. volatile memory) for storing the electronic instructions to be executed and electronic input and output information associated with the electronic instructions, a storage medium 103 (e.g. non-volatile memory) for long term (persistent) storage of information, a communication interface 104 for sending information to and / or receiving information from one or more other apparatuses and a user interface 105 (e.g. a touch screen, a non-touch screen, buttons, a keyboard, a mouse, an audio output device and / or a speech recognition device) for receiving commands from and / or outputting information to a user. Each of the processor 101 , memory 102, storage medium 103, communication interface 104 and user interface 105 are implemented using appropriate circuitry, for example. The processor 101 controls the operation of each of the memory 102, storage medium 103, communication interface 104 and user interface 105.

[0027] In this example, an image of a user 107 is captured using camera 106. Data representing the captured image is communicated to the data processing apparatus 100 via communication interface 105.

[0028] Fig. 2 shows a set of processes performed on the captured image of the user 107 to generate a three-dimensional (3D) avatar providing a controllable 3D virtual representation of the user 107. The 3D avatar has virtual physical characteristics corresponding to the real-life physical characteristics of the user 107. The processes of Fig. 2 are executed by the processor 101 and / or by one or more external servers (not shown) under control of the processor 101. Input data to be processed by the external server(s) is transmitted to the external server(s) and output data resulting from the processing by the external server(s) is received from the external server(s) via the communication interface 104.

[0029] As shown in Fig. 2, an image 201 of the user (e.g. captured by camera 106) is first provided to a step 203 in which a base 3D avatar of the user is generated. The base 3D avatar comprises a 3D pose and 3D mesh of the user. The 3D pose indicates the relative positions of a plurality of joints of the user and the 3D mesh corresponds to a 3D shape of the user. The base 3D avatar combines the 3D pose and 3D mesh so that, as the positions of one or more of the joints of the 3D pose are adjusted, the shape of the 3D mesh changes accordingly. This allows the base 3D avatar to mimic the 3D appearance of the user given any set of 3D joint positions. Various techniques for automatically generating and combining a 3D mesh and pose of a person from an image of that person to generate an avatar are known and are therefore not discussed in detail here. For example, the methods disclosed in [1] and / or [2] may be used.

[0030] The base 3D avatar is then provided to step 204, where a texture is applied (in particular, to the 3D mesh of the base 3D avatar). Various techniques for automatically generating a texture for a mesh are known and are therefore not discussed in detail here. For example, the method disclosed in [3] may be used. The generation of the 3D avatar is thus completed at step 204 by the application of the texture (although, in a simpler example, the application of the texture at step 204 may be omitted).

[0031] The completed 3D avatar is then provided to step 205, where movements of a general agent are applied to the 3D avatar. The movements of the general agent correspond to those of an activity of a tutorial video which is to be personalised for the user. Thus, for example, the movements of the general agent may represent running, a throwing action (e.g. to throw a ball), a lifting action (e.g. to lift a weight) or the like. The general agent is defined as a plurality of joints (corresponding to the joints of the 3D avatar) and a movement profile for each of those joints (each movement profile indicating a changeable position of its joint over a predetermined time period, e.g. the length of the tutorial video). Movements of the general agent are thus applied to the 3D avatar by causing each joint of the 3D avatar to move according to the movement profile of the corresponding joint of the general agent.

[0032] This is exemplified in Fig. 3, which shows the joints 302 defining a general agent 301 and the corresponding joints 302’ of a 3D avatar 303 created for a user. In this example, there are 17 joints defined for each of the general agent 301 and 3D avatar 303 (although the present technology is not limited to this). They comprise joints representing the head, neck, left and right shoulders, left and right wrists, pelvis, left and right hips, left and right knees, left and right ankles and left and right big toes. The correspondence between joints 302 and 302’ is a one-to-one correspondence of joints of the same type. Thus, the head joint of the general agent 301 corresponds to the head joint of the 3D avatar 303, the left wrist joint of the general agent 301 corresponds to the left wrist joint of the 3D avatar 303, and so on.

[0033] Each joint 302’ of the 3D avatar 303 follows the movement profile of the corresponding joint 302 of the general agent 301 . This causes the 3D avatar 303 perform the movements of the general agent 301 associated with the activity. Thus, for example, if the movements of the general agent represent running, the 3D avatar will be shown to run, if the movements of the general agent represent throwing a ball, the 3D avatar will be shown to throw a ball, and so on. This is exemplified in Fig. 3, where the joints 302A’ (big toe), 302B’ (ankle), 302C’ (knee) and 302D’ (hip) of the right leg of the 3D avatar 303 are moved according to the movements (defined by the respective movement profiles) of the corresponding respective joints 302A, 302B, 302C and 302D of the general agent 301 to cause the 3D avatar 303 to raise their right leg. The output of step 205 is a video 206 of the 3D avatar 303 performing the movements of the general agent 301. In an example, the 3D avatar 303 can be rendered from any angle of interest as the video 206 is played back.

[0034] Since the 3D avatar 303 has been generated from the captured image 201 of the user, the movements of the 3D avatar 303 in the video 206 will more closely represent those of the user in real life compared to simply showing a generic avatar performing the movements. For instance, since the relative positions of the joints of the 3D avatar 303 are determined from the captured image 201 of the real-life user, the variation in limb length between different users (with joints along a limb being further away from each other for longer limbs and closer to each other for shorter limbs) will be taken into account in the generation of the 3D avatar 303 and the subsequent movements of the 3D avatar 303 to generated the video 206. The video 206 is thus a personalised training video with the movements of the 3D avatar 303 more closely matching those which can and / or should be carried out by the user in real life when they perform the activity featured in the training video. This helps the user to learn from the video 206 safely and / or more effectively than from a generic training video.

[0035] Optionally, the user may provide additional information 202 to generate the base 3D avatar. This helps the generation of the 3D avatar 303 (step 203) and / or movements of the 3D avatar (step 205) to take into one or more characteristics of the user which may not be apparent from the captured image 201. The user provides this information via the user interface 105, for example. The information includes, for example, any information defining one or more characteristics of the user which may affect the appearance of the 3D avatar 303 and / or movements of the 3D avatar. Example information includes physical measurements of the user (e.g. height, weight, waist and / or inside leg), performance metrics of the user (e.g. the maximum height they can jump, how fast they can run a predetermined distance, etc.) and any injury information about the user. The output video is thus generated taking this information into account. This is exemplified in Figs. 4 to 6, which show how standard movements predefined for the general agent 301 (and thus applied to the 3D avatar 303 in the output video 206) can be adjusted to take into account specific characteristic(s) of the user indicated in additional information 202. In this example, the activity is performing a squat (an exercise to strengthen the legs in which a user starts in a standing position, bends their knees to lower themselves into a squat position and then returns to the standing position).

[0036] Fig. 4 shows the standard movements for the activity (defined by movement profiles for each joint 302 of the general agent). These are the movements applied to the 3D avatar 303 when no additional information 202 is provided by the user. The 3D pose of the general agent 301 at three key points (denoted as times ti , t2and t3) during the activity is shown. In particular, a standing position is shown at ti , a lowermost squat position is shown at t2and a standing position is once again shown at t3. It can be seen that, in the lowermost squat position at time t2, the pelvis and right and left hip joints 401 move to a position lower than the positions of the left and right knee joints 402A, 402B. However, this standard squat may not be suitable for all users. For example, if a user has a knee injury, it may not be possible for them to attempt this standard squat (or to do so may cause discomfort or injury). The standard squat is therefore adjusted, if necessary, depending on additional information 202 provided by the user.

[0037] Fig. 5 shows an example in which an interactive screen 501 displayed on an electronic display 500 (connected to or comprised as part of the user interface 105, for example) to enable the user to input the additional information 202. The electronic display 500 may be a touch screen display and / or alternative input device(s) (also connected to or comprised as part of the user interface 105, e.g. a keyboard and mouse) may be provided to enable user interaction (e.g. input of information, selection of virtual buttons and the like) with the interactive screen 501.

[0038] The interactive screen 501 shows an image 502 of the 3D avatar created for the user and also provides a number of fields 503 for the user to input additional information 202. In this example, the additional information which can be input are physical measurements of the user’s height, weight, waist and inside leg (to determine the relevant dimensions of the 3D avatar 303) and any injuries the user has. In this example, the user has indicated they have reduced mobility in their right knee. The user may input the physical measurements directly (e.g. using a physical or virtual numeric keypad) and select any injuries they have from an interactive menu (containing categories of injury for different body parts), for example. Once the user has entered all additional information, they select the confirm virtual button 504.

[0039] Based on the additional information 202 provided by the user, the dimensions of the generated 3D avatar created for the user are adjusted (based on the input physical measurements) and the standard movements of the general agent 301 for the activity shown in Fig. 4 are adjusted (based on the input injury information). In this example, due to the indicated reduced mobility of the user’s right knee, the standard movements are adjusted to reduce the depth of the lowermost squat position shown at time t2. Thus, while the standing positions at times ti and t3remain the same, the lowermost squat position at t2is adjusted so the pelvis and right and left hip joints 401 no longer move to a position lower than the positions of the left and right knee joints 402A, 402B. Rather, while movement of the left and right knee joints 402A, 402B still occurs to allow the squat movement to be performed, the positions of the pelvis and right and left hip joints 401 remain above the left and right knee joints 402A, 402B.

[0040] This adjusted movements of Fig. 6 are associated with reduced stress on the knee joints compared to the standard movements of Fig. 4. This helps reduce stress on the user’s knees (in particular, their right knee with reduced mobility) when the 3D avatar 303 applies these adjusted movements in the output video 206.

[0041] It will be appreciated that Figs. 4 to 6 are only an example and that, depending on the type of activity to be demonstrated in the output video 206 and the additional information 202 provided by the user (e.g. injury information or, more generally, information indicating physical capabilities of the user), different adjustments to the standard movements of the general agent 301 (and therefore of the 3D avatar 303) may be made. For example, depending a how high a user can jump, a demonstrated technique for throwing a basketball into a hoop may be adjusted, depending on how fast a user can run (e.g. based on how much time it takes a user to run a predetermined distance), a demonstrated technique for appropriate running form may be adjusted, and so on.

[0042] In an example, standard movements for an activity (defined by a standard movement profile for each joint of the general agent) are defined together with one or more sets of adjusted movements for that activity (defined by a respective adjusted movement profile for each joint of the general agent) in a database (stored in storage medium 103, for example) with each set of adjusted movements being associated with one or more characteristics of the user derivable from additional information 202 input by the user. For example, a first set of adjusted movements may be associated with reduced knee mobility of a user, a second set of adjusted movements may be associated with reduced back mobility of the user, a third set of adjusted movements may be associated with reduced shoulder mobility and so on.

[0043] Alternatively, or in addition, the standard movements themselves may be adjusted depending on quantitative physical capability information provided by the user. For example, if a user indicates the maximum height or length they are able to jump, the standard movements for a jumping activity will be adjusted so this maximum is not exceeded or if the user indicates their minimum reaction time (e.g. in response to the sound indicating the start of a sprint), the standard movements for a sprint start activity will be adjusted to reflect this minimum reaction time. Other types of additional information 202 provided by the user which may cause an adjustment of the standard movements may include the equipment a user has. For example, for a cycling activity, the standard movements may be adjusted depending on the type of bike a user has (e.g. road bike or hybrid bike), whether or not the user is clipped in to the peddles, and so on.

[0044] In an example, instead of, or in addition to, the user manually providing the additional information 202, at least a portion of the additional information may be generated automatically via a calibration session. In such a calibration session, for example, the user is instructed to perform one or more predetermined movements while image(s) of the user performing the movement(s) are captured by the camera 106. The image(s) are then analysed (e.g. by processor 101 and / or one or more external servers) to determine one or more characteristics of the user. For example, based on the extent to which a user is able to complete a predetermined movement (e.g. the angle at which the user is able to bend at the waist to move their hands towards the floor), a measure of the flexibility of the user may be estimated (e.g. with a greater angle of bending at the waist indicating a greater flexibility and a lesser angle of bending at the waist indicating a lesser flexibility). The standard movements defining activities demonstrated in the output video 206 can then be suitable adjusted to reflect the determined flexibility of the user.

[0045] An image captured by the camera 106 may also be used to provide real time feedback to a user as they attempt to perform an activity demonstrated in an output video 206. This is exemplified in Figs. 7A to 7C, 8A to 8C and Fig. 9.

[0046] Fig. 7A shows a pose 701 of the general agent (either as standard or as adjusted taking into account additional information 202 from the user, as previously described). The pose 701 is applied to the user’s 3D avatar 303 which is shown in the output video 206. This is the pose intended for the user to adopt.

[0047] Fig. 7B shows the actual pose 702 of the user. The pose 702 is determined from an image captured by the camera 106 and / or using a suitable motion capture system (e.g. mocopi ® from Sony ®) while the 3D avatar 303 with the pose 701 of the general agent is shown in the output video 206. The pose 702 is determined from a captured image in the same way as described for step 203 of Fig. 2, for example.

[0048] Fig. 7C shows a comparison of the intended pose 701 of the general agent overlaid on the actual pose 702 of the user. It can be seen there is a difference (divergence) between the poses 701 and 702. This indicates the user is not adopting the intended pose 701 shown in the output video 206 correctly. It is therefore desirable for feedback to be provided to the user to assist them in adjusting their actual pose 702 to more closely match that of the intended pose 701 . In an example, in order to provide the user with appropriate feedback, a comparison is performed between predetermined groups of joints of the intended pose 701 and actual pose 702. This is exemplified in Figs. 8A to 8C.

[0049] As shown in Fig. 8A, each of the joints 302 of the intended pose 701 of the general agent is assigned to a particular group of joints. In particular, the right elbow and wrist joints are assigned to group 701 A, the left and right shoulder, head and neck joints are assigned to group 701 B, the left elbow and wrist joints are assigned to group 701 C, the pelvis and left and right hip joints are assigned to group 701 D, the left knee, ankle and big toe joints are assigned to group 701 E and the right knee, ankle and big toe joints are assigned to group 701 F.

[0050] Similarly, as shown in Fig. 8B, each of the joints 703 of the actual pose 702 of the user is correspondingly assigned to a particular group of joints. In particular, the right elbow and wrist joints are assigned to group 702A, the left and right shoulder, head and neck joints are assigned to group 702B, the left elbow and wrist joints are assigned to group 702C, the pelvis and left and right hip joints are assigned to group 702D, the left knee, ankle and big toe joints are assigned to group 702E and the right knee, ankle and big toe joints are assigned to group 702F.

[0051] The positions of the joints in corresponding groups of joints (that is, groups comprising the same joint types) are then compared, as shown in Fig. 8C. Thus, the positions of the joints in group 701 A are compared with those of group 702A, the positions of the joints in group 701 B are compared with those in group 702B, and so on.

[0052] In an example, the comparison involves translationally aligning predetermined anchor points of the poses 701 and 702 and rotationally aligning the poses 701 and 702 so that the angles defined by corresponding features of the poses 701 and 702 are aligned. In this example, the anchor points 704A, 704B for translational alignment are the left ankle joints of the poses 701 and 702 and, for rotational alignment, the angles of the lines 705A, 705B between the neck and head joints of the poses 701 and 702 are made to be equal.

[0053] Once the poses 701 and 702 are translationally and rotationally aligned, the distances between corresponding joints in each of the corresponding groups of joints are determined. This is exemplified in Fig. 8C, which, for corresponding joint groups 701 F and 702F, shows a distance di between the left big toe joints of the poses 701 and 702, a distance d2between the left ankle joints of the poses 701 and 702 and a distance d3between the left knee joints of the poses 701 and 702. Fig. 8C also shows, for corresponding joint groups 701 E and 702E, a distance d4between the right big toe joints of the poses 701 and 702, a distance d5between the left ankle joints of the poses 701 and 702 (where, in this case, d5= 0 since the left ankle joints of poses 701 and 702 are translationally aligned) and a distance d6between the right knee joints of the poses 701 and 702. Although not shown in Fig. 8C, distances between corresponding joints in each of the other corresponding joint groups are also determined. The distances between the corresponding joints in each of the corresponding groups of joints are then compared to determine whether any of the distances exceed another of the distances by more than a predetermined threshold. For example, the distances may be ranked in descending order (with the largest distance first) and it may be determined whether the size of the largest distance exceeds the size of the shortest distance by more than a predetermined threshold. The predetermined threshold may be set to be equivalent to 5, 10 or 15 cm, for example (based on a dimensional calibration between the physical space in which the user is located and the virtual space in which the poses 701 and 702 are defined).

[0054] For each instance of corresponding joint groups (that is, each of corresponding joint groups 701A and 702A, corresponding joint groups 702B and 702B, and so on), if it is determined the threshold is exceeded, it is determined there is a sufficiently large discrepancy between the part(s) of the poses 701 and 702 corresponding to those corresponding joint groups for appropriate corrective feedback to be provided to the user forthose part(s) of the poses 701 and 702. The corrective feedback indicates, for example, a direction in which the user should move the joint associated with the largest distance between corresponding joint groups to reduce this distance. On the other hand, if it is determined the threshold is not exceeded (indicating a mere translation of all joints of the corresponding joint groups by the same amount, as occurs, for example, if the user is adopting the relevant part(s) of the pose correctly but at a different vertical height due bending their knee(s) too much or too little), it is determined that corrective feedback for the corresponding part(s) of the poses 701 and 702 is not required.

[0055] Thus, In the example of Fig. 8C, the threshold is exceeded for both corresponding groups 701 E and 702E (with d6exceeding d5by more than the threshold) and corresponding groups 701 F and 702F (with d2exceeding d3by more than the threshold). It is not, however, exceeded for any of the remaining instances of corresponding groups (e.g. groups 701 A and 702A, 701 B and 702B, and so on). Corrective feedback is thus provided for the left ankle joint of the user (associated with distance d2) and for the right knee joint of the user (associated with distance d6).

[0056] Such corrective feedback is exemplified in Fig. 9, in which a feedback screen 900 is displayed on the electronic display 500 showing a real time-generated version of a 3D avatar 901 of the user (determined from the current captured image of the user in the same way as described for steps 203 and 204 of Fig. 2 and / or using any suitable motion capture system such as mocopi ® from Sony ®, for example) together with graphics 902A and 902B overlaid on the parts of the 3D avatar 901 corresponding, respectively, to the left ankle and right knee joints of the user. Again, the 3D avatar 901 can be rendered from any angle of interest. The graphics 902A and 902B indicate the direction in which the user needs to move these joints to reduce the distances d2and d6and thus bring the user’s actual pose 702 into better conformity with the intended pose 701. In this case, for example, the user can see they need to lift their left leg higher (according to graphic 902A) and bend their right knee more (according to graphic 902B). A textual message 903 is also shown to the user on the feedback screen 900 instructing the user what to do. It will be appreciated that other types of feedback (e.g. audio feedback) could also be provided.

[0057] In an example, the electronic display 500 is configured to output the video 206 (showing the 3D avatar 303 of the user performing the activity as intended) and may display the 3D avatar 901 (showing the user’s current pose in real time, optionally with corrective graphics like those shown in Fig. 9) simultaneously (e.g. side by side) with the output video 206.

[0058] The present technology thus allows tutorial videos for activities to be automatically tailored to a user’s particular characteristics, thereby providing the user with individualised tutorial videos to facilitate improved learning. In particular, for activities such as sports, such videos may allow the user to learn or improve at the sport more effectively and safely (especially if the user is recovering from injury, for example). The present technology also allows feedback to be provided to the user in real time to help them quickly and easily identify problems with their technique and make suitable adjustments to better conform with the demonstrated activity in the tutorial video.

[0059] Although the above examples relate to a single user, the present technology may be extended to provide training videos involving 3D avatars of multiple users performing a team activity. This allows, for example, sports teams to refine tactical plays and practice joint moves. The present technology may also be applied to non-physical sport applications. For example, the movements of a general agent in a video game may be applied to a 3D avatar of a video game character and the movements and / or joint positions of the general agent tailored to the characteristics and / or available in-game equipment of the video game character. This allows, for example, a video game tutorial to be generated once using a general agent and then applied in a bespoke manner to any video game character a user wishes to play as.

[0060] Fig. 10 shows an example method. The method is implemented by the processor 101 , for example.

[0061] The method starts at step 1001.

[0062] At step 1002, movement data defining movement of a generic virtual character having one or more generic characteristics is obtained.

[0063] General agent 301 is an example of the generic virtual character. The movement profile of each of one or more virtual joints of the general agent is an example of the movement data. The movement profile of each virtual joint indicates a position of the virtual joint in a virtual space over a reproduction time period of the video.

[0064] At step 1003, one or more specific characteristics of a specific virtual character are obtained. The specific virtual character is the 3D avatar 302 of the user, for example. The one or more specific characteristics are different to corresponding ones of the one or more generic characteristics of the generic virtual character.

[0065] For example, the one or more specific characteristics may include one or more dimensions (e.g. height measurement, weight measurement, waist measurement and / or inside leg measurement) of the specific virtual character which are different to the corresponding dimensions of the generic virtual character. This may result in the position of one or more of the virtual joints being changed for at least a portion of the respective movement profile(s) of those joint(s). For instance, if the specific virtual character is taller than the generic virtual character, one or more of the joints (e.g. head, neck, shoulder, pelvis, hip and / or knee joints) of the specific virtual character may be at a higher position throughout its respective movement profile than that of the corresponding joint(s) of the generic virtual character.

[0066] The one or more specific characteristics may also include information indicative of a movement limitation of the specific virtual character (e.g. reduced mobility of a particular joint).

[0067] Such examples of the one or more specific characteristics are shown in Fig. 5.

[0068] At step 1004, the movement data is adjusted according to the one or more specific characteristics. For example, the movement profile of each joint is adjusted as exemplified in Figs. 4 and 6.

[0069] At step 1005, video data is generated representing a video of the specific virtual character performing movement defined by the adjusted movement data.

[0070] The method ends at step 1006.

[0071] Example(s) of the present disclosure are defined by the following numbered clauses:

[0072] 1 . A data processing apparatus comprising circuitry configured to: obtain movement data defining movement of a generic virtual character having one or more generic characteristics; obtain one or more specific characteristics of a specific virtual character, the one or more specific characteristics being different to corresponding ones of the one or more generic characteristics; adjust the movement data according to the one or more specific characteristics; and generate video data representing a video of the specific virtual character performing movement defined by the adjusted movement data.

[0073] 2. A data processing apparatus according to clause 1 , wherein the movement data comprises a movement profile of each of one or more virtual joints, the movement profile of each virtual joint indicating a position of the virtual joint in a virtual space over a reproduction time period of the video.

[0074] 3. A data processing apparatus according to clause 1 or 2, wherein the movement data defines movement carried out when performing a sport.

[0075] 4. A data processing apparatus according to any preceding clause, wherein the one or more specific characteristics comprise one or more dimensions of the specific virtual character.

[0076] 5. A data processing apparatus according to any preceding clause, wherein the one or more specific characteristics comprising information indicative of a movement limitation of the specific virtual character.

[0077] 6. A data processing apparatus according to any preceding clause, wherein the specific virtual character is a three-dimensional avatar generated from a captured image of a user.

[0078] 7. A data processing apparatus according to clause 6, wherein the circuitry is configured to: obtain data indicative of movement of the user during reproduction of the video; compare the indicated movement of the user with the movement of the specific virtual character in the video; generate a notification for the user in response to detecting a divergence between the movement of the user and the movement of the specific virtual character in the video.

[0079] 8. A data processing apparatus according to any one of clauses 1 to 5, wherein the specific virtual character is a video game character.

[0080] 9. A computer-implemented data processing method comprising: obtaining movement data defining movement of a generic virtual character having one or more generic characteristics; obtaining one or more specific characteristics of a specific virtual character, the one or more specific characteristics being different to corresponding ones of the one or more generic characteristics; adjusting the movement data according to the one or more specific characteristics; and generating video data representing a video of the specific virtual character performing movement defined by the adjusted movement data.

[0081] 10. A program for controlling a computer to perform a method according to clause 9.

[0082] 11. A computer-readable medium storing a program according to clause 10. Numerous modifications and variations of the present disclosure are possible in light of the above teachings. It is therefore to be understood that, within the scope of the claims, the disclosure may be practiced otherwise than as specifically described herein.

[0083] In so far as embodiments of the disclosure have been described as being implemented, at least in part, by one or more software-controlled information processing apparatuses, it will be appreciated that a machine-readable medium (in particular, a non-transitory machine-readable medium) carrying such software, such as an optical disk, a magnetic disk, semiconductor memory or the like, is also considered to represent an embodiment of the present disclosure. In particular, the present disclosure should be understood to include a non-transitory storage medium comprising code components which cause a computer to perform any of the disclosed method(s).

[0084] It will be appreciated that the above description for clarity has described embodiments with reference to different functional units, circuitry and / or processors. However, it will be apparent that any suitable distribution of functionality between different functional units, circuitry and / or processors may be used without detracting from the embodiments.

[0085] Described embodiments may be implemented in any suitable form including hardware, software, firmware or any combination of these. Described embodiments may optionally be implemented at least partly as computer software running on one or more computer processors (e.g. data processors and / or digital signal processors). The elements and components of any embodiment may be physically, functionally and logically implemented in any suitable way. Indeed, the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the disclosed embodiments may be implemented in a single unit or may be physically and functionally distributed between different units, circuitry and / or processors.

[0086] Although the present disclosure has been described in connection with some embodiments, it is not intended to be limited to these embodiments. Additionally, although a feature may appear to be described in connection with particular embodiments, one skilled in the art would recognize that various features of the described embodiments may be combined in any manner suitable to implement the present disclosure.

[0087] REFERENCES

[0088]

[0001] Feng et al, “Collaborative Regression of Expressive Bodies using Moderation”, International Conference on 3D Vision (3DV), pages 792 — 804, December 2021

[0089] [2] Pavlakos et al, “Expressive Body Capture: 3D Hands, Face, and Body from a Single Image”, Proceedings IEEE Conf, on Computer Vision and Pattern Recognition (CVPR), pages 10975—10985, 2019 [3] Chen et al, “Text2Tex: Text-driven Texture Synthesis via Diffusion Models”, arXiv preprint arXiv:2303.11396, 2023

Claims

CLAIMS1 . A data processing apparatus comprising circuitry configured to: obtain movement data defining movement of a generic virtual character having one or more generic characteristics; obtain one or more specific characteristics of a specific virtual character, the one or more specific characteristics being different to corresponding ones of the one or more generic characteristics; adjust the movement data according to the one or more specific characteristics; and generate video data representing a video of the specific virtual character performing movement defined by the adjusted movement data.

2. A data processing apparatus according to claim 1 , wherein the movement data comprises a movement profile of each of one or more virtual joints, the movement profile of each virtual joint indicating a position of the virtual joint in a virtual space over a reproduction time period of the video.

3. A data processing apparatus according to claim 1 , wherein the movement data defines movement carried out when performing a sport.

4. A data processing apparatus according to claim 1 , wherein the one or more specific characteristics comprise one or more dimensions of the specific virtual character.

5. A data processing apparatus according to claim 1 , wherein the one or more specific characteristics comprising information indicative of a movement limitation of the specific virtual character.

6. A data processing apparatus according to claim 1 , wherein the specific virtual character is a three-dimensional avatar generated from a captured image of a user.

7. A data processing apparatus according to claim 6, wherein the circuitry is configured to: obtain data indicative of movement of the user during reproduction of the video; compare the indicated movement of the user with the movement of the specific virtual character in the video; generate a notification for the user in response to detecting a divergence between the movement of the user and the movement of the specific virtual character in the video.

8. A data processing apparatus according to claim 1 , wherein the specific virtual character is a video game character.

9. A computer-implemented data processing method comprising: obtaining movement data defining movement of a generic virtual character having one or more generic characteristics; obtaining one or more specific characteristics of a specific virtual character, the one or more specific characteristics being different to corresponding ones of the one or more generic characteristics; adjusting the movement data according to the one or more specific characteristics; and generating video data representing a video of the specific virtual character performing movement defined by the adjusted movement data.

10. A program for controlling a computer to perform a method according to claim 9.

11. A computer-readable medium storing a program according to claim 10.

Images