Image processing method, device, electronic device and storage medium

Abstract

The embodiments of the present disclosure provide an image processing method, device, electronic device, and storage medium. The method includes: determining a skeleton chain to be registered and a preset skeleton chain of a target part in a video frame to be processed; determining wrist state information of the wrist skeleton point to be registered based on the first pose information of the wrist skeleton point to be registered and the second pose information of the preset wrist skeleton point; determining finger state information of the finger skeleton point to be registered based on the wrist state information, the third pose information of the finger skeleton point to be registered, and the fourth pose information of the preset finger skeleton point; and displaying the target part in the video frame to be processed based on the wrist state information and the finger state information. The embodiments of the present disclosure improve the authenticity of the display of the target part in the display interface without penetrating the model, thereby improving the user experience.

Description

Technical Field

[0001] The present disclosure relates to image processing technology, and more particularly to an image processing method, apparatus, electronic device, and storage medium. Background Art

[0002] Whether it is animation redirection or artificial intelligence limb recognition, there may be a need to migrate a set of animation data from one set of skeletal models to another set of skeletal models for display.

[0003] If the two skeletal models differ significantly, for example, in size or pose, the migrated image may show noticeable clipping. This is particularly noticeable for models with a large number of bone points, resulting in poor animation quality and a lack of realism, potentially impacting the user experience. Summary of the Invention

[0004] The present disclosure provides an image processing method, device, electronic device and storage medium, so as to improve the authenticity of the display of the target part in the display interface without penetrating the mold, thereby improving the user experience.

[0005] In a first aspect, an embodiment of the present disclosure provides an image processing method, the method comprising:

[0006] Determine a skeleton chain to be registered and a preset skeleton chain of the target part in the video frame to be processed; wherein the skeleton chain to be registered includes a wrist skeleton point to be registered and a finger skeleton point to be registered, and the preset skeleton chain includes a preset wrist skeleton point and a preset finger skeleton point;

[0007] Determining wrist state information of the wrist skeletal point to be registered according to the first pose information of the wrist skeletal point to be registered and the second pose information of the preset wrist skeletal point; wherein the wrist state information includes wrist rotation information and wrist pose information;

[0008] Determining the finger state information of the finger skeletal point to be registered based on the wrist state information, the third posture information of the finger skeletal point to be registered, and the fourth posture information of the preset finger skeletal point; wherein the finger state information includes finger rotation information and finger posture information;

[0009] Based on the wrist state information and the finger state information, the target part is displayed in the video frame to be processed.

[0010] In a second aspect, an embodiment of the present disclosure further provides an image processing device, the device comprising:

[0011] A skeleton chain determination module is used to determine the skeleton chain to be registered and the preset skeleton chain of the target part in the video frame to be processed; wherein the skeleton chain to be registered includes the wrist skeleton points to be registered and the finger skeleton points to be registered, and the preset skeleton chain includes the preset wrist skeleton points and the preset finger skeleton points;

[0012] A wrist determination module, configured to determine wrist state information of the wrist skeletal point to be registered based on the first pose information of the wrist skeletal point to be registered and the second pose information of the preset wrist skeletal point; wherein the wrist state information includes wrist rotation information and wrist pose information;

[0013] a finger determination module, configured to determine finger state information of the finger skeletal point to be registered based on the wrist state information, the third posture information of the finger skeletal point to be registered, and the fourth posture information of the preset finger skeletal point; wherein the finger state information includes finger rotation information and finger posture information;

[0014] A display module is configured to display the target part in the video frame to be processed based on the wrist state information and the finger state information.

[0015] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0016] one or more processors;

[0017] a storage device for storing one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any one of the embodiments of the present disclosure.

[0019] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to execute the image processing method as described in any one of the embodiments of the present disclosure.

[0020] The technical solution of the embodiment of the present disclosure is that when it is determined that bone repositioning is required, the wrist bone point to be registered in the to-be-registered bone chain connected to the animation can be registered with the preset wrist bone point in the preset bone chain to obtain the wrist state information of the wrist bone point to be registered. On the basis of determining the wrist state information, the finger state information of each finger bone point to be registered can be determined in turn according to each preset finger bone point in the preset bone chain and the posture information of the registered finger bone point. Then, based on the wrist state information and the finger state information, the display information of the target part and the video frame to be processed is adjusted, which solves the problem of easy model penetration during the migration of animation data of the target part and the problem of poor display effect and poor authenticity of the target part, and realizes improving the authenticity of the display of the target part in the display interface without model penetration, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0022] Figure 1 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0023] Figure 2 A schematic diagram of a skeleton chain to be registered provided in an embodiment of the present disclosure;

[0024] Figure 3 A schematic diagram of a preset skeleton chain provided in an embodiment of the present disclosure;

[0025] Figure 4 A schematic diagram of an arm skeleton chain adjustment provided by an embodiment of the present disclosure;

[0026] Figure 5 A flowchart of another image processing method provided by an embodiment of the present disclosure;

[0027] Figure 6 A schematic diagram of a wrist skeleton point to be registered and at least one finger skeleton point to be registered directly associated with the wrist skeleton point to be registered provided by an embodiment of the present disclosure;

[0028] Figure 7 A schematic diagram of a preset wrist skeleton point and at least one preset finger skeleton point directly associated with the preset wrist skeleton point provided by an embodiment of the present disclosure;

[0029] Figure 8 A flowchart of another image processing method provided by an embodiment of the present disclosure;

[0030] Figure 9 This is a schematic diagram of a finger skeleton point to be registered before rotation provided by an embodiment of the present disclosure;

[0031] Figure 10 A schematic diagram of a finger skeleton point to be registered after rotation provided by an embodiment of the present disclosure;

[0032] Figure 11 A schematic structural diagram of an image processing device provided by an embodiment of the present disclosure;

[0033] Figure 12 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0035] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0036] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0037] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0038] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0039] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0040] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0041] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0042] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0043] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0044] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0045] Before introducing the present technical solution, an example description of the application scenario can be given first. The present technical solution can be applied to scenarios where there are limb animations in the display interface, such as animation redirection, artificial intelligence limb recognition, etc. The limb animation can be triggered by a button displayed on the display interface, or by a preset continuous motion process of the target part of the target object. For example, in a game, the user controls the target character in the display interface to release skills by pressing buttons or touching, etc., which can cause the target part of the target character to move to show the skill special effects. At this time, when the limb animation is displayed in the display interface, the solution provided by the embodiment of the present disclosure can be adopted to achieve the effect of repositioning the skeletal part. In other words, as long as there is animation data migration of limb animation in the display interface, such as animation data migration of hands or feet in scenes such as image display, video display, and live broadcast display, the technical solution provided by the embodiment of the present disclosure can be used for rendering and display to improve the display effect. If there are target parts in the skeletal model whose skeletal points are dependent on each other, for example, the target part can be a hand or a foot, etc., the technical solution of the embodiment of the present disclosure can be used to migrate the animation data. The apparatus for executing the image processing method provided in the embodiments of the present disclosure may be integrated into application software having image processing capabilities, and the software may be installed in an electronic device, optionally a mobile terminal or a PC. The application software may be software for image / video processing, and the specific application software will not be described in detail here; as long as the application software can implement image / video processing, it will be sufficient.

[0046] Figure 1 This is a flow chart of an image processing method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to situations where a target part in a video frame to be processed is processed and displayed. The method can be executed by an image processing device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC or a server, etc.

[0047] like Figure 1 As shown, the method includes:

[0048] S110: Determine the skeletal chain to be registered and the preset skeletal chain of the target part in the video frame to be processed.

[0049] The video frame to be processed may be a video frame in a video containing the target part. The target part may be a part including the wrist and fingers, such as a hand or a foot. The skeleton chain to be registered may be the initial skeleton model to be registered of the target part, such as Figure 2 The skeleton chain to be registered includes the wrist skeleton point to be registered and the finger skeleton point to be registered. The wrist skeleton point to be registered can be the skeleton point corresponding to the wrist in the skeleton chain to be registered, such as Figure 2Root' in the , the finger bone point to be registered can be the bone point corresponding to the finger in the bone chain to be registered, such as Figure 2 The preset skeleton chain can be the skeleton model of the target part to be registered, such as Figure 3 The preset bone chain includes a preset wrist bone point and a preset finger bone point. The preset wrist bone point can be the bone point corresponding to the wrist in the preset bone chain, such as Figure 3 The Root in the preset finger bone point can be the bone point corresponding to the finger in the preset bone chain, such as Figure 3 The bone points except Root.

[0050] Specifically, a video frame to be processed can be obtained, and a target part in the video frame to be processed can be determined. Accordingly, a skeletal chain to be registered can be extracted according to the target part in the video frame to be processed, and a preset skeletal chain corresponding to the target part can be determined according to the action corresponding to the target part.

[0051] It should be noted that the preset skeleton chain can be a pre-set skeleton chain corresponding to the actions of different target parts. Through the embodiment of the present disclosure, the skeleton chain to be aligned can be aligned to the posture corresponding to the preset skeleton chain, that is, it can be rendered and displayed according to the preset skeleton chain to complete the action of the target part.

[0052] S120: Determine wrist state information of the wrist skeletal point to be registered according to the first pose information of the wrist skeletal point to be registered and the second pose information of the preset wrist skeletal point.

[0053] The first pose information may be the position and posture information of the wrist skeletal points to be registered. The second pose information may be the position and posture information of the preset wrist skeletal points. The wrist state information may be information used to represent the wrist skeletal point registration result. The wrist state information includes wrist rotation information and wrist pose information. The wrist rotation information can be used to represent coordinate rotation transformation and can be represented by a wrist rotation matrix. The wrist pose information may include position information and posture information.

[0054] Specifically, because the first pose information of the wrist skeletal points to be registered differs from the second pose information of the preset wrist skeletal points in the global coordinate system, the two need to be aligned. Alignment processing can be performed based on the first pose information of the wrist skeletal points to be registered and the second pose information of the preset wrist skeletal points to determine the wrist state information required for registration of the wrist skeletal points to be registered, i.e., the information required to convert the pose of the wrist to be registered.

[0055] For example, a local coordinate system to be registered can be established based on the wrist skeletal points to be registered, and a preset local coordinate system can be established based on the preset wrist skeletal points to determine the wrist rotation information corresponding to the coordinate system transformation, and the wrist posture information required for the transformation of the wrist skeletal points to be registered to the preset wrist skeletal points can be determined, and then, the wrist rotation information and wrist posture information can be used as wrist state information.

[0056] S130. Determine the finger state information of the finger skeleton points to be registered based on the wrist state information, the third posture information of the finger skeleton points to be registered, and the fourth posture information of the preset finger skeleton points.

[0057] The third pose information may be the position and posture information of the finger skeletal points to be registered. The fourth pose information may be the position and posture information of the preset finger skeletal points. The finger state information may be information used to represent the finger skeletal point registration results. The finger state information includes finger rotation information and finger pose information. The finger rotation information can be represented by a finger rotation matrix, and the finger pose information may include position information and posture information.

[0058] Specifically, since the finger skeletal points are dependent on the wrist skeletal points and / or the finger skeletal points, and each finger skeletal point is dependent on the wrist skeletal points, either directly or indirectly, an alignment process can be performed based on the wrist state information, the third pose information of the finger skeletal points to be registered, and the fourth pose information of the preset finger skeletal points to determine the finger state information of the finger skeletal points to be registered, i.e., the information required to convert the pose of the finger to be registered.

[0059] S140: Display the target part in the video frame to be processed based on the wrist state information and the finger state information.

[0060] Specifically, the skeletal chain to be registered can be processed according to the wrist state information and the finger state information, so as to process the skeletal chain to be registered as the target part and render it in the video frame to be processed, so as to accurately display the target part in the video frame to be processed according to the posture information of the preset skeletal chain.

[0061] Based on the above examples, in order to improve the authenticity and accuracy of the display effect of the target part, the display position and display posture of each bone point in the target part in the display interface can be adjusted based on the wrist status information and finger status information to obtain the target video frame corresponding to the video frame to be processed.

[0062] The display position can be information describing the position, such as the coordinates of a skeletal point on a display interface. The display posture can be information describing the posture, such as the angle between a skeletal point and each coordinate axis on a display page. The target video frame can be the video frame obtained after processing the target part of the video frame to be processed.

[0063] Specifically, after obtaining the wrist status information and the finger status information, the wrist bone points of the target part in the video frame to be processed can be adjusted according to the wrist status information, and the finger bone points of the target part in the video frame to be processed can be adjusted according to the finger status information to determine the display position and display posture of each bone point in the target part, and render each bone point of the target part to the display interface based on the display position and display posture, and use the video frame as the target video frame.

[0064] It should be noted that when adjusting the display position and display posture of each bone point in the target part in the display interface, a skinning effect can be added to the bone chain of the target part to make the display of the target part closer to the skin and improve the display effect.

[0065] Based on the above examples, in order to improve the display effect of the target part, enhance the visual effect and avoid display distortion, the following methods can be used for display:

[0066] If, based on the wrist status information, it is determined that the position offset between the wrist bone point to be aligned and the preset wrist bone point is greater than the preset offset threshold, then the length information of the arm bone chain and / or leg bone chain to which the wrist bone point to be aligned belongs is determined; based on the length information and the position offset, the display position of at least one bone point in the arm bone chain and / or leg bone chain is adjusted.

[0067] The position offset may be the distance between the position information of the wrist bone point to be registered and the position information of the preset wrist bone point. The preset offset threshold may be the maximum distance that the wrist bone point to be registered can be offset. The arm bone chain may be the bone chain corresponding to the target part being the arm. The leg bone chain may be the bone chain corresponding to the target part being the leg. The length information may be the length value of the arm bone chain and / or the leg bone chain.

[0068] Specifically, if it is determined based on the wrist state information that the position offset between the wrist bone point to be registered and the preset wrist bone point is greater than the preset offset threshold, it indicates that directly adjusting the wrist bone point to be registered cannot be aligned with the preset wrist bone point. In this case, this can be achieved by adjusting at least one bone point in the arm bone chain and / or leg bone chain to which the wrist bone point to be registered belongs. The length information of the arm bone chain and / or leg bone chain to which the wrist bone point to be registered belongs can be determined, and the display position of at least one bone point in the arm bone chain and / or leg bone chain can be adjusted based on the length information and the position offset. For example, the display position of each bone point can be determined by proportional stretching so that the wrist bone point to be registered can be aligned according to the wrist state information.

[0069] For example, the arm skeleton chain adjustment diagram is as follows Figure 4 As shown, W is the wrist bone point to be registered, T is the preset wrist bone point, Figure 4 As shown in the figure above, the position offset between the wrist bone point to be registered and the preset wrist bone point is greater than the preset offset threshold, that is, the wrist bone point to be registered cannot reach the preset wrist bone point. Therefore, the arm skeleton chain ULW is lengthened. According to the length information of the upper arm UL and the length information of the lower arm LW, it can be lengthened proportionally to correct the display position of L and W, as shown in the following example: Figure 4 As shown in the figure below, the wrist bone point W to be registered can be registered with the preset wrist bone point T.

[0070] The technical solution of the embodiment of the present disclosure is that when it is determined that bone repositioning is required, the wrist bone point to be registered in the to-be-registered bone chain connected to the animation can be registered with the preset wrist bone point in the preset bone chain to obtain the wrist state information of the wrist bone point to be registered. On the basis of determining the wrist state information, the finger state information of each finger bone point to be registered can be determined in turn according to each preset finger bone point in the preset bone chain and the posture information of the registered finger bone point. Then, based on the wrist state information and the finger state information, the display information of the target part and the video frame to be processed is adjusted, which solves the problem of easy model penetration during the migration of animation data of the target part and the problem of poor display effect and poor authenticity of the target part, and realizes improving the authenticity of the display of the target part in the display interface without model penetration, thereby improving the user experience.

[0071] Figure 5 This is a flow chart of another image processing method provided by an embodiment of the present disclosure. Based on the aforementioned embodiment, wrist state information of the wrist skeletal points to be registered can be first determined. For specific implementations, please refer to the detailed description of this technical solution. The explanations of terms that are identical or corresponding to those in the aforementioned embodiments are not repeated here.

[0072] like Figure 5 As shown, the method includes:

[0073] S210: Determine the skeletal chain to be registered and the preset skeletal chain of the target part in the video frame to be processed.

[0074] S220: Determine first rotation information according to the first pose information of the wrist skeleton point to be registered and the third pose information of at least one finger skeleton point to be registered that is directly associated with the wrist skeleton point to be registered.

[0075] The third pose information may be the position information and posture information of at least one finger bone point to be registered that is directly associated with the wrist bone point to be registered. The first rotation information may be the rotation information corresponding to the wrist bone point to be registered, which may be in the form of a rotation matrix or a quaternion.

[0076] Specifically, at least one finger bone point to be registered that is directly associated with the wrist bone point to be registered can be determined. This can be understood as extending from the wrist bone point to be registered as the starting point toward each fingertip, with the first finger bone point to be registered in each extension direction being the directly associated finger bone point to be registered, for example Figure 2 Where Root' is the wrist skeletal point to be registered, and U', P', K', F', and A' are all finger skeletal points directly associated with the wrist skeletal point to be registered. The rotation matrix corresponding to the wrist skeletal point to be registered can be determined based on the first pose information of the wrist skeletal point to be registered and the third pose information of at least one finger skeletal point directly associated with the wrist skeletal point to be registered. The first rotation information can then be determined based on the rotation matrix.

[0077] Based on the above example, if the at least one finger skeleton point to be registered includes a first finger skeleton point to be registered and a second finger skeleton point to be registered, the first rotation information can be determined more quickly and accurately by the following method:

[0078] Determine a first vector based on the first pose information and the pose information of the first finger skeleton point to be registered; determine a second vector based on the first pose information and the pose information of the second finger skeleton point to be registered; and determine first rotation information based on the first vector and the second vector.

[0079] The first vector may be a vector pointing from the wrist skeletal point to the first finger skeletal point to be registered. The second vector may be a vector pointing from the wrist skeletal point to the second finger skeletal point to be registered. It should be noted that the first finger skeletal point to be registered and the second finger skeletal point to be registered are any two of the finger skeletal points to be registered that are directly associated with the wrist skeletal point to be registered, and are not specifically limited in this embodiment.

[0080] Specifically, based on the position information in the first pose information and the position information in the pose information of the first finger skeletal point to be registered, it can be determined that the vector pointing from the wrist skeletal point to the first finger skeletal point to be registered is the first vector. Based on the position information in the first pose information and the position information in the pose information of the second finger skeletal point to be registered, it can be determined that the vector pointing from the wrist skeletal point to the second finger skeletal point to be registered is the second vector. By performing a cross product based on the first vector and the second vector, a first vertical vector perpendicular to both the first vector and the second vector can be obtained, and a second vertical vector can be obtained by performing a cross product based on any one of the first vector and the second vector and the first vertical vector. Furthermore, a rotation matrix is ​​constructed based on the first vertical vector, the second vertical vector and the vector corresponding to the second vertical vector (the first vector or the second vector). The rotation matrix can be used as the first rotation information, and the rotation matrix can also be transformed into a quaternion, and the quaternion can be used as the first rotation information.

[0081] For example, Figure 6 The schematic diagram of the wrist skeleton point to be registered and at least one finger skeleton point to be registered directly associated with the wrist skeleton point to be registered is shown, wherein the wrist skeleton point to be registered is O', and the finger skeleton points to be registered directly associated with the wrist skeleton point to be registered are A', B', C', D' and E'. Taking the finger skeleton point C' to be registered as the first finger skeleton point to be registered and the finger skeleton point B' to be registered as the second finger skeleton point to be registered, the first vector can be determined to be O'C' and the second vector to be O'B'. The first vertical vector up'=O'C'×O'B', the first vector O'C' is selected to determine the second vertical vector, denoted forward'=O'C', and the second vertical vector left'=up'×forward'. Furthermore, the rotation matrix can be determined based on left', up' and forward', and the rotation matrix is ​​converted to obtain the quaternion q', which is the first rotation information.

[0082] S230: Determine second rotation information based on the second posture information of the preset wrist skeleton point and the fourth posture information of at least one preset finger skeleton point directly associated with the preset wrist skeleton point.

[0083] The fourth posture information may be the position information and posture information of at least one preset finger bone point directly associated with the preset wrist bone point. The second rotation information may be the rotation information corresponding to the preset wrist bone point, which may be in the form of a rotation matrix or a quaternion.

[0084] Specifically, at least one preset finger bone point directly associated with the preset wrist bone point can be determined. It can be understood that the first preset finger bone point in each extension direction extending from the preset wrist bone point as the starting point to each fingertip is the directly associated preset finger bone point, for example Figure 3Where Root is the preset wrist skeletal point, and V, Q, L, G, and A are all preset finger skeletal points directly associated with the preset wrist skeletal point. Based on the second pose information of the preset wrist skeletal point and the fourth pose information of at least one preset finger skeletal point directly associated with the preset wrist skeletal point, a rotation matrix corresponding to the preset wrist skeletal point can be determined, and second rotation information can be determined based on the rotation matrix.

[0085] Based on the above example, if the at least one preset finger skeleton point includes a first preset finger skeleton point and a second preset finger skeleton point, the second rotation information can be determined more quickly and accurately by the following method:

[0086] Based on the second posture information and the posture information of the first preset finger skeleton point, determine the third vector; based on the second posture information and the posture information of the second preset finger skeleton point, determine the fourth vector; based on the third vector and the fourth vector, determine the second rotation information.

[0087] The third vector may be a vector pointing from the preset wrist skeletal point to the first preset finger skeletal point. The fourth vector may be a vector pointing from the preset wrist skeletal point to the second preset finger skeletal point. It should be noted that the first preset finger skeletal point and the second preset finger skeletal point are any two preset finger skeletal points directly associated with the preset wrist skeletal point, and are not specifically limited in this embodiment.

[0088] Specifically, based on the position information in the second posture information and the position information in the first posture information of the finger skeleton point, it can be determined that the vector pointing from the preset wrist skeleton point to the first preset finger skeleton point is the third vector. Based on the position information in the second posture information and the position information in the posture information of the second preset finger skeleton point, it can be determined that the vector pointing from the preset wrist skeleton point to the second preset finger skeleton point is the fourth vector. By cross-producting the third vector and the fourth vector, a third vertical vector perpendicular to both the third vector and the fourth vector can be obtained, and a fourth vertical vector can be obtained by cross-producting any one of the third vector and the fourth vector with the third vertical vector. Furthermore, a rotation matrix is ​​constructed based on the third vertical vector, the fourth vertical vector and the vector corresponding to the fourth vertical vector (the third vector or the fourth vector). The rotation matrix can be used as the second rotation information, and the rotation matrix can also be transformed into a quaternion, and the quaternion can be used as the second rotation information.

[0089] For example, Figure 7The schematic diagram of the preset wrist skeleton point and at least one preset finger skeleton point directly associated with the preset wrist skeleton point is shown, wherein the preset wrist skeleton point is O, and the preset finger skeleton points directly associated with the preset wrist skeleton point are A, B, C, D and E. Taking the preset finger skeleton point C as the first preset finger skeleton point and the preset finger skeleton point B as the second preset finger skeleton point, the third vector can be determined to be OC and the fourth vector is OB. The third vertical vector up = OC × OB, the third vector OC is selected to determine the fourth vertical vector, denoted forward = OC, and the fourth vertical vector left = up × forward. Furthermore, the rotation matrix can be determined based on left, up and forward, and the rotation matrix can be converted to obtain the quaternion q, which is the second rotation information.

[0090] S240: Determine wrist state information of the wrist skeleton point to be registered based on the first rotation information and the second rotation information.

[0091] Specifically, the wrist skeleton points to be registered may be registered according to the preset wrist skeleton points based on the first rotation information and the second rotation information, and the registered information is used as the wrist state information of the wrist skeleton points to be registered.

[0092] Based on the above example, in order to more accurately and quickly determine the wrist state information, an alignment transformation matrix can be introduced for processing, which can be:

[0093] determining an alignment transformation matrix according to the second rotation information and the inverse of the first rotation information;

[0094] The wrist state information is determined based on the alignment transformation matrix and the first pose information of the wrist skeleton points to be registered.

[0095] The alignment transformation matrix may be a rotation transformation matrix for aligning the wrist bone points to be registered with the preset wrist bone points.

[0096] Specifically, the second rotation information is multiplied by the inverse of the first rotation information to obtain an alignment transformation matrix. The first pose information is processed according to the alignment transformation matrix to obtain position information and pose information in the wrist pose information.

[0097] Exemplarily, if the first rotation information is q' and the second rotation information is q, then the alignment transformation matrix r is r=q*Inverse(q'), where Inverse(·) represents an inverse operation.

[0098] Based on the above example, in order to more accurately align the wrist skeleton point to be registered with the preset wrist skeleton point, the wrist pose information in the wrist state information can be determined based on the alignment transformation matrix and the first pose information of the wrist skeleton point to be registered in the following way:

[0099] Based on the position information in the second pose information, position information in the wrist pose information is determined; based on the alignment transformation matrix and the position information in the first pose information, posture information in the wrist pose information is determined.

[0100] Specifically, the position information in the second pose information is determined as the position information in the wrist pose information. The position information in the first pose information is multiplied by the alignment transformation matrix to obtain the pose information in the wrist pose information.

[0101] For example, if the position information in the second pose information is Op, then the position information in the wrist pose information is Op. If the position information in the first pose information is Or and the alignment transformation matrix is ​​r, then the wrist pose information in the wrist state information is O'r=Or*r.

[0102] Through S220-S240, the first rotation information and the second rotation information can be accurately and quickly determined to unify the wrist skeleton point to be registered and the preset wrist skeleton point, thereby improving the registration efficiency of the wrist skeleton point.

[0103] S250: Determine the finger state information of the finger skeleton points to be registered based on the wrist state information, the third posture information of the finger skeleton points to be registered, and the fourth posture information of the preset finger skeleton points.

[0104] S260: Display the target part in the video frame to be processed based on the wrist state information and the finger state information.

[0105] The technical solution of the embodiment of the present disclosure is to determine the to-be-registered skeleton chain and the preset skeleton chain of the target part in the video frame to be processed, determine the first rotation information according to the first pose information of the wrist skeleton point to be registered and the third pose information of at least one finger skeleton point to be registered directly associated with the wrist skeleton point to be registered, determine the second rotation information according to the second pose information of the preset wrist skeleton point and the fourth pose information of at least one preset finger skeleton point directly associated with the preset wrist skeleton point, and determine the wrist state information of the wrist skeleton point to be registered based on the first rotation information and the second rotation information, so as to convert the wrist skeleton point to be registered into the wrist state information. The wrist skeleton points are aligned according to the preset wrist skeleton points. The finger state information of the finger skeleton points to be aligned is determined according to the wrist state information, the third pose information of the finger skeleton points to be aligned and the fourth pose information of the preset finger skeleton points. Based on the wrist state information and the finger state information, the target part is displayed in the video frame to be processed, which solves the problem of poor display effect and poor authenticity caused by the penetration phenomenon when the wrist animation data in the target part is migrated, and realizes the improvement of the display effect and authenticity of the wrist in the target part without penetration, thereby improving the user experience.

[0106] Figure 8 This is a flow chart of another image processing method provided by an embodiment of the present disclosure. Based on the wrist state information of the wrist skeletal point to be registered, the finger state information of each finger skeletal point to be registered can be determined in sequence. The specific implementation method can be found in the detailed description of this technical solution. The explanations of terms that are the same as or corresponding to the above embodiments are not repeated here.

[0107] like Figure 8 As shown, the method includes:

[0108] S301: Determine the skeleton chain to be registered and the preset skeleton chain of the target part in the video frame to be processed.

[0109] S302: Determine wrist state information of the wrist skeletal point to be registered according to the first pose information of the wrist skeletal point to be registered and the second pose information of the preset wrist skeletal point.

[0110] S303: Retrieve the preset first dependency information of the wrist skeleton points and the finger skeleton points to be registered, and the second dependency information of the preset wrist skeleton points and the preset finger skeleton points.

[0111] The dependency information can be information used to describe the finger or wrist skeletal points that each finger skeletal point directly depends on. The dependency information includes the finger or wrist skeletal points that at least one finger skeletal point located on the same branch in the skeletal chain directly depends on. The first dependency information corresponds to the skeletal chain to be registered, and the second dependency information corresponds to the preset skeletal chain.

[0112] Specifically, first dependency information for the wrist skeletal point to be registered and the finger skeletal points to be registered can be set based on the connection information of the wrist skeletal point to be registered and each finger skeletal point to be registered in the skeletal chain to be registered. Second dependency information for the preset wrist skeletal point and the preset finger skeletal point can also be set based on the connection information of the preset wrist skeletal point and each preset finger skeletal point in the preset skeletal chain. The first dependency information and the second dependency information can be retrieved for subsequent skeletal point registration.

[0113] It should be noted that the dependency information corresponding to the finger skeleton point is extended from the finger skeleton point to the wrist, and the finger skeleton point or wrist skeleton point adjacent to the finger skeleton point in the extension direction is used as the dependency information.

[0114] For example, Figure 2In the skeleton chain to be registered shown, taking the branch Root'-U'-V'-W'-X'-Y' as an example, the finger skeleton point U' to be registered directly depends on the wrist skeleton point Root' to be registered, that is, the first dependency information of U' is Root'; the finger skeleton point V' to be registered directly depends on the finger skeleton point U' to be registered, that is, the first dependency information of V' is U'; in the same way, the first dependency information corresponding to other finger skeleton points to be registered can be determined to obtain the first dependency information of the wrist skeleton point to be registered and the finger skeleton point to be registered. Figure 3 In the preset skeleton chain shown, taking the branch Root-VWXY as an example, the preset finger skeleton point V directly depends on the preset wrist skeleton point Root, that is, the second dependency information of V is Root; the preset finger skeleton point W directly depends on the preset finger skeleton point V, that is, the second dependency information of W is V; in the same way, the second dependency information corresponding to other preset finger skeleton points can be determined to obtain the second dependency information of the preset wrist skeleton point and the preset finger skeleton point.

[0115] S304. Determine the processing priority of the finger skeleton points to be registered and the preset finger skeleton points based on the first dependency information and the second dependency information, so as to determine the finger state information of the finger skeleton points to be registered in sequence based on the processing priority.

[0116] The processing priority may be a priority determined according to the dependency information. Taking any skeleton point as an example, the priority of the skeleton point is lower than the priority of the dependency information corresponding to the skeleton point.

[0117] Specifically, the processing priority of each to-be-registered finger skeleton point may be determined according to the first dependency information, and the processing priority of each preset finger skeleton point may be determined according to the second dependency information.

[0118] For example, Figure 2 As shown, taking the branch Root'-U'-V'-W'-X'-Y' as an example, the processing priority of the finger skeleton points to be registered is U'-V'-W'-X'-Y' from high to low. Figure 3 As shown, taking the branch Root-VWXY as an example, the processing priority of the preset finger skeleton points is VWXY from high to low.

[0119] Through S303-S304, the processing priority of each finger skeleton point to be registered and the processing priority of each preset finger skeleton point can be determined, so that the finger skeleton points to be registered can be registered in order according to the processing priority.

[0120] S305: Determine at least one finger skeleton point group.

[0121] The finger bone point group can be a combination of the to-be-registered bone chain and the corresponding finger bone points in the preset bone chain. The finger bone point group includes the preset finger bone points and the to-be-registered finger bone points that are located in the corresponding bone chain branches and have the same processing priority.

[0122] For example, Figure 2 and Figure 3 As shown, the branch Root'-A'-B'-C'-D'-E' in the to-be-registered skeleton chain corresponds to the branch Root-ABCDE in the preset skeleton chain. The preset finger skeleton point A and the to-be-registered finger skeleton point A' have the same processing priority. These two finger skeleton points can be considered a finger skeleton point group. Other finger skeleton point groups can be determined in the same manner.

[0123] S306. For at least one finger skeleton point group, determine the front-dependent skeleton point group and the back-dependent skeleton point group that the current finger skeleton point group depends on according to the processing priority.

[0124] Among them, the front-dependent bone point group can be the wrist bone point group or the finger bone point group that the current finger bone point group depends on. The back-dependent bone point group can be the finger bone point group that depends on the current finger bone point group. The front-dependent bone point group includes the wrist bone point group or the finger bone point group. The back-dependent bone point group includes the finger bone point group. It can be understood that the back-dependent bone point group directly depends on the current finger bone point group, and the current finger bone point group directly depends on the front-dependent bone point group.

[0125] Specifically, for each finger bone point group, the finger bone point group can be used as the current finger bone point group, and the front-dependent bone point group and the back-dependent bone point group on which the current finger bone point group depends are determined in the same manner. According to the processing priority, the finger bone point group that directly depends on the current finger bone point group is determined as the back-dependent bone point group, and the finger bone point group or wrist bone point group that the current finger bone point group directly depends on is determined as the front-dependent bone point group.

[0126] For example, Figure 2 and Figure 3 As shown, the finger bone point group consisting of the preset finger bone point A and the finger bone point A' to be registered is used as the current finger bone point group. According to the processing priority, the front dependent bone point group on which the current finger bone point group depends can be determined to be the wrist bone point group consisting of the preset wrist bone point Root and the wrist bone point Root' to be registered. The back dependent bone point group on which the current finger bone point group depends can be determined to be the finger bone point group consisting of the preset wrist bone point B and the wrist bone point B' to be registered. The front dependent bone point group and the back dependent bone point group on which other finger bone point groups depend can be determined in the same way as above.

[0127] S307. Determine whether the front dependent skeleton point group is a wrist skeleton point group or a finger skeleton point group. If the front dependent skeleton point group is a wrist skeleton point group, execute S308; if the front dependent skeleton point group is a finger skeleton point group, execute S309.

[0128] S308. Determine the finger state information of the finger skeleton points to be aligned in the current finger skeleton point group based on the wrist rotation information in the wrist state information, the posture information in the current finger skeleton point group, and the posture information in the subsequent dependent skeleton point group, and execute S310.

[0129] Specifically, based on the posture information in the current finger skeleton point group and the posture information in the subsequent dependent skeleton point group, the rotation matrix of the skeleton points to be registered in the current finger skeleton point group can be determined. Based on the rotation matrix and the wrist rotation information, the current finger rotation information of the skeleton points to be registered in the current finger skeleton point group can be determined. Based on the current finger rotation information and the wrist rotation information, the current finger posture information of the skeleton points to be registered in the current finger skeleton point group can be determined. Furthermore, the current finger rotation information and the current finger posture information are determined as the finger state information of the finger skeleton points to be registered in the current finger skeleton point group.

[0130] S309. Determine the finger state information of the finger bone points to be registered in the current finger bone point group based on the target finger rotation information in the finger state information of the bone points to be registered in the front dependent bone point group, the posture information in the current finger bone point group, and the posture information of the rear dependent bone point group, and execute S310.

[0131] Specifically, based on the posture information in the current finger skeleton point group and the posture information in the subsequent dependent skeleton point group, the rotation matrix of the skeleton point to be registered in the current finger skeleton point group can be determined. Based on the rotation matrix and the target finger rotation information, the current finger rotation information of the skeleton point to be registered in the current finger skeleton point group can be determined. Based on the current finger rotation information and the target finger rotation information, the current finger posture information of the skeleton point to be registered in the current finger skeleton point group can be determined. Furthermore, the current finger rotation information and the current finger posture information are determined as the finger state information of the finger skeleton point to be registered in the current finger skeleton point group.

[0132] Through S305-S308 or S305-S309, the finger status information of each finger bone point to be registered in each branch of the skeletal chain to be registered can be determined in turn to ensure that the finger rotation information and finger posture information of each finger bone point to be registered are accurately aligned with each preset finger bone point, thereby improving the accuracy and efficiency of the registration.

[0133] Based on the above example, the current finger rotation information and the current finger pose information of the to-be-registered finger bone points in the current finger bone point group can be determined respectively in the following manner to determine the finger state information of the to-be-registered finger bone points in the current finger bone point group, thereby reducing the complexity of determining the finger state information and improving the accuracy of determining the finger state information:

[0134] According to the target finger rotation information, the third posture information and the fourth posture information in the current finger skeleton point group, and the third posture information and the fourth posture information in the subsequent dependent skeleton point group, the current finger rotation information of the skeleton point to be aligned in the current finger skeleton point group is determined.

[0135] Specifically, the direction of the current finger skeleton to be registered can be determined based on the third pose information in the current finger skeleton point group and the third pose information in the posterior dependent skeleton point group. The direction of the current preset finger skeleton can be determined based on the fourth pose information in the current finger skeleton point group and the fourth pose information in the posterior dependent skeleton point group. According to the direction of the current finger skeleton to be registered and the direction of the current preset finger skeleton, the rotation matrix required to rotate from the direction of the current finger skeleton to be registered to the direction of the current preset finger skeleton can be determined. The current finger rotation information of the skeleton points to be registered in the current finger skeleton point group can be determined based on the target finger rotation information and the determined rotation matrix.

[0136] Based on the target finger rotation information and the current finger rotation information, the current finger pose information of the to-be-registered skeleton points in the current finger skeleton point group is determined.

[0137] Specifically, based on the position information in the fourth pose information in the current finger skeletal point group, the position information of the current finger pose information of the skeletal point to be registered in the current finger skeletal point group can be determined. Based on the target finger rotation information and the current finger rotation information, the posture information of the skeletal point to be registered in the current finger skeletal point group can be determined. The above-determined position information and posture information are used as the current finger pose information of the skeletal point to be registered in the current finger skeletal point group.

[0138] Based on the current finger rotation information and the current finger pose information, the finger state information of the skeleton points to be registered is determined.

[0139] Specifically, the current finger rotation information and the current finger pose information are used as the finger state information of the skeleton points to be registered.

[0140] Based on the above example, the current finger rotation information of the bone points to be aligned in the current finger bone point group can be determined in the following way, so that the current finger rotation information can be smoothly transitioned, so that the subsequent display effect according to the current finger rotation information is more realistic.

[0141] Determine the third rotation vector based on the third posture information in the current finger bone point group and the third posture information in the subsequent dependent bone point group; and determine the fourth rotation vector based on the fourth posture information in the current finger bone point group and the fourth posture information in the subsequent dependent bone point group; determine the rotation matrix of the finger bone points to be aligned in the current finger bone point group to the corresponding preset finger bone points based on the third rotation vector and the fourth rotation vector; determine the current finger rotation information of the bone points to be aligned in the current finger bone point group based on the rotation matrix and the target finger rotation information.

[0142] The third rotation vector may be a vector pointing from a finger bone point to be registered in the current finger bone point group to a finger bone point to be registered in the posterior dependent bone point group. The fourth rotation vector may be a vector pointing from a preset finger bone point in the current finger bone point group to a preset finger bone point in the posterior dependent bone point group.

[0143] Specifically, based on the position information in the third pose information in the current finger skeleton point group and the position information in the third pose information in the posterior dependent skeleton point group, it can be determined that the vector of the finger skeleton point to be aligned in the current finger skeleton point group points to the finger skeleton point to be aligned in the posterior dependent skeleton point group is the third rotation vector. Based on the position information in the fourth pose information in the current finger skeleton point group and the position information in the fourth pose information in the posterior dependent skeleton point group, it can be determined that the vector of the preset finger skeleton point in the current finger skeleton point group points to the preset finger skeleton point in the posterior dependent skeleton point group is the fourth rotation vector. Based on the third rotation vector and the fourth rotation vector, the rotation quaternion rotated from the direction of the third rotation vector to the direction of the fourth rotation vector can be calculated, which is the rotation matrix. The inverse of the target finger rotation information is multiplied by the rotation matrix to obtain the current finger rotation information of the skeleton point to be aligned in the current finger skeleton point group.

[0144] For example, the schematic diagram of the finger skeleton point to be registered before rotation is as follows: Figure 9As shown, RST is the preset skeleton chain, R'-S'-T' is the skeleton chain to be registered, S' and S represent the current finger skeleton point group, T' and T represent the posterior dependent skeleton point group, and R' and R represent the anterior dependent skeleton point group. The position information in the third pose information of the current finger skeleton point group is Sp', the position information in the third pose information of the posterior dependent skeleton point group is Tp', and the third rotation vector O = T'p-S'p. The position information in the fourth pose information of the current finger skeleton point group is Sp, the position information in the fourth pose information of the posterior dependent skeleton point group is Tp, and the fourth rotation vector C = Tp-Sp. The rotation matrix Q = FromToQuaternion(O,C) of the finger skeleton points to be registered in the current finger skeleton point group to the corresponding preset finger skeleton points represents the rotation quaternion obtained from O to C. The target finger rotation information is R'r, and the current finger rotation information of the skeleton point to be registered in the current finger skeleton point group is S'lr = Inverse (R'r) * Q. The schematic diagram of the finger skeleton point to be registered after rotation is as follows: Figure 10 shown.

[0145] It should be noted that if there is an initial rotation S'r for the bone points to be registered in the current finger bone point group, then Q = FromToQuaternion(O,C)*S'r, that is, the rotation matrix is ​​updated according to the result of multiplying the rotation matrix by the initial rotation of the bone points to be registered in the current finger bone point group.

[0146] S310: Display the target part in the video frame to be processed based on the wrist state information and the finger state information.

[0147] The technical solution of the embodiment of the present disclosure is to determine the to-be-registered skeleton chain and the preset skeleton chain of the target part in the video frame to be processed, determine the wrist state information of the to-be-registered wrist skeleton point according to the first pose information of the to-be-registered wrist skeleton point and the second pose information of the preset wrist skeleton point, retrieve the preset first dependency information of the to-be-registered wrist skeleton point and the to-be-registered finger skeleton point, and the second dependency information of the preset wrist skeleton point and the preset finger skeleton point, determine the processing priority of the to-be-registered finger skeleton point and the preset finger skeleton point according to the first dependency information and the second dependency information, so as to determine the finger state information of the to-be-registered finger skeleton point in turn based on the processing priority, determine at least one finger skeleton point group, and for At least one finger skeleton point group determines the front dependent skeleton point group and the back dependent skeleton point group on which the current finger skeleton point group depends according to the processing priority, and then determines whether the front dependent skeleton point group is a wrist skeleton point group or a finger skeleton point group, so as to determine the finger state information of the finger skeleton points to be aligned in the current finger skeleton point group according to different judgment results, and displays the target part in the video frame to be processed based on the wrist state information and the finger state information, which solves the problem of poor display effect and poor authenticity caused by the penetration phenomenon when migrating the finger animation data of the target part, and realizes improving the display effect and authenticity of the finger in the target part without penetration, thereby improving the user experience.

[0148] Figure 11 A schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure is shown in FIG. Figure 11 As shown, the apparatus includes: a skeleton chain determination module 410 , a wrist determination module 420 , a finger determination module 430 and a display module 440 .

[0149] Among them, the skeleton chain determination module 410 is used to determine the skeleton chain to be registered and the preset skeleton chain of the target part in the video frame to be processed; wherein, the skeleton chain to be registered includes the wrist skeleton point to be registered and the finger skeleton point to be registered, and the preset skeleton chain includes the preset wrist skeleton point and the preset finger skeleton point; the wrist determination module 420 is used to determine the wrist state information of the wrist skeleton point to be registered based on the first posture information of the wrist skeleton point to be registered and the second posture information of the preset wrist skeleton point; wherein, the wrist state information includes wrist rotation information and wrist posture information; the finger determination module 430 is used to determine the finger state information of the finger skeleton point to be registered based on the wrist state information, the third posture information of the finger skeleton point to be registered and the fourth posture information of the preset finger skeleton point; wherein, the finger state information includes finger rotation information and finger posture information; the display module 440 is used to display the target part in the video frame to be processed based on the wrist state information and the finger state information.

[0150] Optionally, the wrist determination module 420 is also used to determine first rotation information based on the first posture information of the wrist skeleton point to be registered and the third posture information of at least one finger skeleton point to be registered that is directly associated with the wrist skeleton point to be registered; determine second rotation information based on the second posture information of the preset wrist skeleton point and the fourth posture information of at least one preset finger skeleton point directly associated with the preset wrist skeleton point; and determine the wrist state information of the wrist skeleton point to be registered based on the first rotation information and the second rotation information.

[0151] Optionally, the wrist determination module 420 is further used to determine an alignment transformation matrix based on the second rotation information and the inverse of the first rotation information; and determine the wrist state information based on the alignment transformation matrix and the first position pose information of the wrist bone point to be aligned.

[0152] Optionally, the wrist determination module 420 is further used to determine the position information in the wrist pose information based on the position information in the second pose information; and determine the posture information in the wrist pose information based on the alignment transformation matrix and the position information in the first pose information.

[0153] Optionally, the device also includes: a dependency processing module, used to retrieve the first dependency information of the preset wrist bone points to be aligned and the finger bone points to be aligned, and the second dependency information of the preset wrist bone points and the preset finger bone points; wherein the dependency information includes the finger bone points or wrist bone points directly dependent on at least one finger bone point located on the same branch in the bone chain; based on the first dependency information and the second dependency information, determine the processing priority of the finger bone points to be aligned and the preset finger bone points, so as to determine the finger status information of the finger bone points to be aligned in turn based on the processing priority.

[0154] Optionally, the finger determination module 430 is also used to determine at least one finger skeleton point group; wherein, the finger skeleton point group includes preset finger skeleton points and finger skeleton points to be aligned that are located at corresponding skeleton chain branches and have the same processing priority; for the at least one finger skeleton point group, the front-dependent skeleton point group and the back-dependent skeleton point group on which the current finger skeleton point group depends are determined according to the processing priority; wherein, the front-dependent skeleton point group includes a wrist skeleton point group or a finger skeleton point group; if the front-dependent skeleton point group is a wrist skeleton point group, then the finger state information of the finger skeleton points to be aligned in the current finger skeleton point group is determined according to the wrist rotation information in the wrist state information, the posture information in the current finger skeleton point group, and the posture information in the back-dependent skeleton point group.

[0155] Optionally, the finger determination module 430 is also used to determine the finger state information of the finger bone points to be aligned in the current finger bone point group based on the target finger rotation information in the finger state information of the bone points to be aligned in the front dependent bone point group, the posture information in the current finger bone point group, and the posture information of the rear dependent bone point group if the front dependent bone point group is a finger bone point group.

[0156] Optionally, the finger determination module 430 is also used to determine the current finger rotation information of the bone points to be registered in the current finger bone point group based on the target finger rotation information, the third posture information and the fourth posture information in the current finger bone point group, and the third posture information and the fourth posture information in the subsequent dependent bone point group; determine the current finger posture information of the bone points to be registered in the current finger bone point group based on the target finger rotation information and the current finger rotation information; and determine the finger state information of the bone points to be registered based on the current finger rotation information and the current finger posture information.

[0157] Optionally, the finger determination module 430 is also used to determine a third rotation vector based on the third posture information in the current finger skeleton point group and the third posture information in the subsequent dependent skeleton point group; and to determine a fourth rotation vector based on the fourth posture information in the current finger skeleton point group and the fourth posture information in the subsequent dependent skeleton point group; determine the rotation matrix of the finger skeleton points to be aligned in the current finger skeleton point group to the corresponding preset finger skeleton points based on the third rotation vector and the fourth rotation vector; and determine the current finger rotation information of the skeleton points to be aligned in the current finger skeleton point group based on the rotation matrix and the target finger rotation information.

[0158] Optionally, the display module 440 is further configured to adjust the display position and display posture of each skeletal point in the target part in the display interface based on the wrist state information and the finger state information, so as to obtain a target video frame corresponding to the video frame to be processed.

[0159] Optionally, the device also includes: an adjustment module for determining the length information of the arm skeleton chain and / or leg skeleton chain to which the wrist skeleton point to be aligned belongs if, based on the wrist state information, it is determined that the position offset between the wrist skeleton point to be aligned and the preset wrist skeleton point is greater than a preset offset threshold; and adjusting the display position of at least one skeleton point in the arm skeleton chain and / or leg skeleton chain based on the length information and the position offset.

[0160] The technical solution of the embodiment of the present disclosure is that when it is determined that bone repositioning is required, the wrist bone point to be registered in the to-be-registered bone chain connected to the animation can be registered with the preset wrist bone point in the preset bone chain to obtain the wrist state information of the wrist bone point to be registered. On the basis of determining the wrist state information, the finger state information of each finger bone point to be registered can be determined in turn according to each preset finger bone point in the preset bone chain and the posture information of the registered finger bone point. Then, based on the wrist state information and the finger state information, the display information of the target part and the video frame to be processed is adjusted, which solves the problem of easy model penetration during the migration of animation data of the target part and the problem of poor display effect and poor authenticity of the target part, and realizes improving the authenticity of the display of the target part in the display interface without model penetration, thereby improving the user experience.

[0161] The image processing device provided by the embodiments of the present disclosure can execute the image processing method provided by any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0162] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0163] Figure 12 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 12 , which shows an electronic device (eg Figure 12 The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 12 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0164] like Figure 12As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An edit / output (I / O) interface 505 is also connected to the bus 504.

[0165] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 12 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0166] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0167] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0168] The electronic device provided by the embodiment of the present disclosure and the image processing method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0169] An embodiment of the present disclosure provides a computer storage medium having a computer program stored thereon. When the program is executed by a processor, the image processing method provided by the above embodiment is implemented.

[0170] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0171] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0172] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0173] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0174] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: determines the skeletal chain to be registered and the preset skeletal chain of the target part in the video frame to be processed; wherein, the skeletal chain to be registered includes the wrist skeletal point to be registered and the finger skeletal point to be registered, and the preset skeletal chain includes the preset wrist skeletal point and the preset finger skeletal point; determines the wrist state information of the wrist skeletal point to be registered based on the first posture information of the wrist skeletal point to be registered and the second posture information of the preset wrist skeletal point; wherein, the wrist state information includes wrist rotation information and wrist posture information; determines the finger state information of the finger skeletal point to be registered based on the wrist state information, the third posture information of the finger skeletal point to be registered and the fourth posture information of the preset finger skeletal point; wherein, the finger state information includes finger rotation information and finger posture information; based on the wrist state information and the finger state information, displays the target part in the video frame to be processed.

[0175] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0176] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0177] The units described in the embodiments of the present disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the skeletal chain determination module may also be described as a "module for obtaining a skeletal chain to be registered and a preset skeletal chain."

[0178] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0179] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0180] According to one or more embodiments of the present disclosure, [Example 1] provides an image processing method, the method comprising:

[0181] Determine a skeleton chain to be registered and a preset skeleton chain of the target part in the video frame to be processed; wherein the skeleton chain to be registered includes a wrist skeleton point to be registered and a finger skeleton point to be registered, and the preset skeleton chain includes a preset wrist skeleton point and a preset finger skeleton point;

[0182] Determining wrist state information of the wrist skeletal point to be registered according to the first pose information of the wrist skeletal point to be registered and the second pose information of the preset wrist skeletal point; wherein the wrist state information includes wrist rotation information and wrist pose information;

[0183] Determining the finger state information of the finger skeletal point to be registered based on the wrist state information, the third posture information of the finger skeletal point to be registered, and the fourth posture information of the preset finger skeletal point; wherein the finger state information includes finger rotation information and finger posture information;

[0184] Based on the wrist state information and the finger state information, the target part is displayed in the video frame to be processed.

[0185] According to one or more embodiments of the present disclosure, [Example 2] provides an image processing method, further comprising:

[0186] Optionally, determining the wrist state information of the wrist skeletal point to be registered according to the first pose information of the wrist skeletal point to be registered and the second pose information of the preset wrist skeletal point includes:

[0187] Determine first rotation information according to the first pose information of the wrist skeleton point to be registered and third pose information of at least one finger skeleton point to be registered that is directly associated with the wrist skeleton point to be registered;

[0188] determining second rotation information based on the second posture information of the preset wrist skeletal point and fourth posture information of at least one preset finger skeletal point directly associated with the preset wrist skeletal point;

[0189] Based on the first rotation information and the second rotation information, wrist state information of the wrist skeleton point to be registered is determined.

[0190] According to one or more embodiments of the present disclosure, [Example 3] provides an image processing method, further comprising:

[0191] Optionally, if the at least one finger skeletal point to be registered includes a first finger skeletal point to be registered and a second finger skeletal point to be registered, determining the first rotation information based on the first pose information of the wrist skeletal point to be registered and the third pose information of at least one finger skeletal point to be registered that is directly associated with the wrist skeletal point to be registered includes:

[0192] Determine a first vector based on the first posture information and the posture information of the first finger skeleton point to be registered;

[0193] Determine a second vector based on the first pose information and the pose information of the second finger skeleton point to be registered;

[0194] First rotation information is determined based on the first vector and the second vector.

[0195] According to one or more embodiments of the present disclosure, [Example 4] provides an image processing method, further comprising:

[0196] Optionally, if the at least one preset finger skeletal point includes a first preset finger skeletal point and a second preset finger skeletal point, determining the second rotation information based on the second pose information of the preset wrist skeletal point and fourth pose information of at least one preset finger skeletal point directly associated with the preset wrist skeletal point includes:

[0197] Determining a third vector based on the second posture information and the posture information of the first preset finger skeleton point;

[0198] Determining a fourth vector based on the second posture information and the posture information of the second preset finger skeleton point;

[0199] Second rotation information is determined based on the third vector and the fourth vector.

[0200] According to one or more embodiments of the present disclosure, [Example 5] provides an image processing method, further comprising:

[0201] Optionally, determining the wrist state information of the wrist skeletal point to be registered based on the first rotation information and the second rotation information includes:

[0202] determining an alignment transformation matrix according to the second rotation information and the inverse of the first rotation information;

[0203] The wrist state information is determined based on the alignment transformation matrix and the first position information of the wrist skeleton point to be registered.

[0204] According to one or more embodiments of the present disclosure, [Example 6] provides an image processing method, further comprising:

[0205] Optionally, determining the wrist pose information in the wrist state information based on the alignment transformation matrix and the first pose information of the wrist skeletal point to be registered includes:

[0206] determining, based on the position information in the second posture information, the position information in the wrist posture information;

[0207] Posture information in the wrist pose information is determined based on the alignment transformation matrix and position information in the first pose information.

[0208] According to one or more embodiments of the present disclosure, [Example 7] provides an image processing method, further comprising:

[0209] Optionally, before determining the finger state information of the finger skeletal point to be registered based on the wrist state information, the third posture information of the finger skeletal point to be registered, and the fourth posture information of the preset finger skeletal point, the method further includes:

[0210] Retrieving pre-set first dependency information of the wrist skeletal point to be registered and the finger skeletal point to be registered, as well as second dependency information of the preset wrist skeletal point and the preset finger skeletal point; wherein the dependency information includes the finger skeletal point or wrist skeletal point directly dependent on at least one finger skeletal point located on the same branch in the skeletal chain;

[0211] Based on the first dependency information and the second dependency information, the processing priority of the finger skeleton points to be registered and the preset finger skeleton points is determined, so as to determine the finger status information of the finger skeleton points to be registered in sequence based on the processing priority.

[0212] According to one or more embodiments of the present disclosure, [Example 8] provides an image processing method, further comprising:

[0213] Optionally, determining the finger state information of the finger skeletal point to be registered based on the wrist state information, the third posture information of the finger skeletal point to be registered, and the fourth posture information of the preset finger skeletal point includes:

[0214] Determine at least one finger skeleton point group; wherein the finger skeleton point group includes preset finger skeleton points and finger skeleton points to be registered that are located at corresponding skeleton chain branches and have the same processing priority;

[0215] For the at least one finger skeleton point group, determining, according to the processing priority, a front-dependent skeleton point group and a back-dependent skeleton point group on which the current finger skeleton point group depends; wherein the front-dependent skeleton point group includes a wrist skeleton point group or a finger skeleton point group;

[0216] If the front-dependent skeleton point group is a wrist skeleton point group, the finger state information of the finger skeleton points to be aligned in the current finger skeleton point group is determined based on the wrist rotation information in the wrist state information, the posture information in the current finger skeleton point group, and the posture information in the rear-dependent skeleton point group.

[0217] According to one or more embodiments of the present disclosure, [Example 9] provides an image processing method, further comprising:

[0218] Optionally, also include:

[0219] If the front-dependent skeleton point group is a finger skeleton point group, the finger state information of the finger skeleton points to be registered in the current finger skeleton point group is determined based on the target finger rotation information in the finger state information of the bone points to be registered in the front-dependent skeleton point group, the posture information in the current finger skeleton point group, and the posture information of the rear-dependent skeleton point group.

[0220] According to one or more embodiments of the present disclosure, [Example 10] provides an image processing method, further comprising:

[0221] Optionally, the determining of the finger state information of the finger skeletal points to be registered in the current finger skeletal point group according to the target finger rotation information in the finger state information of the skeletal points to be registered in the front dependent skeletal point group, the posture information in the current finger skeletal point group, and the posture information of the rear dependent skeletal point group includes:

[0222] Determine the current finger rotation information of the to-be-registered skeletal points in the current finger skeletal point group according to the target finger rotation information, the third pose information and the fourth pose information in the current finger skeletal point group, and the third pose information and the fourth pose information in the subsequent dependent skeletal point group;

[0223] Determine the current finger pose information of the to-be-registered skeleton points in the current finger skeleton point group based on the target finger rotation information and the current finger rotation information;

[0224] Based on the current finger rotation information and the current finger pose information, the finger state information of the to-be-registered skeleton point is determined.

[0225] According to one or more embodiments of the present disclosure, [Example 11] provides an image processing method, further comprising:

[0226] Optionally, determining the current finger rotation information of the to-be-registered skeletal points in the current finger skeletal point group based on the target finger rotation information, the third pose information and the fourth pose information in the current finger skeletal point group, and the third pose information and the fourth pose information in the subsequent dependent skeletal point group includes:

[0227] Determining a third rotation vector according to the third posture information in the current finger skeleton point group and the third posture information in the subsequent dependent skeleton point group; and

[0228] Determining a fourth rotation vector according to the fourth pose information in the current finger skeleton point group and the fourth pose information in the subsequent dependent skeleton point group;

[0229] Determine, based on the third rotation vector and the fourth rotation vector, a rotation matrix for rotating the finger skeleton points to be registered in the current finger skeleton point group to the corresponding preset finger skeleton points;

[0230] Based on the rotation matrix and the target finger rotation information, current finger rotation information of the to-be-registered bone points in the current finger bone point group is determined.

[0231] According to one or more embodiments of the present disclosure, [Example 12] provides an image processing method, further comprising:

[0232] Optionally, displaying the target part in the to-be-processed video frame based on the wrist state information and the finger state information includes:

[0233] Based on the wrist state information and the finger state information, the display position and display posture of each skeletal point in the target part in the display interface are adjusted to obtain a target video frame corresponding to the video frame to be processed.

[0234] According to one or more embodiments of the present disclosure, [Example 13] provides an image processing method, further comprising:

[0235] Optionally, also include:

[0236] If it is determined based on the wrist state information that the position offset between the wrist skeletal point to be registered and the preset wrist skeletal point is greater than a preset offset threshold, then determining the length information of the arm skeletal chain and / or leg skeletal chain to which the wrist skeletal point to be registered belongs;

[0237] Based on the length information and the position offset, the display position of at least one bone point in the arm bone chain and / or the leg bone chain is adjusted.

[0238] According to one or more embodiments of the present disclosure, [Example 14] provides an image processing device, the device including:

[0239] A skeleton chain determination module is used to determine the skeleton chain to be registered and the preset skeleton chain of the target part in the video frame to be processed; wherein the skeleton chain to be registered includes the wrist skeleton points to be registered and the finger skeleton points to be registered, and the preset skeleton chain includes the preset wrist skeleton points and the preset finger skeleton points;

[0240] A wrist determination module, configured to determine wrist state information of the wrist skeletal point to be registered based on the first pose information of the wrist skeletal point to be registered and the second pose information of the preset wrist skeletal point; wherein the wrist state information includes wrist rotation information and wrist pose information;

[0241] a finger determination module, configured to determine finger state information of the finger skeletal point to be registered based on the wrist state information, the third posture information of the finger skeletal point to be registered, and the fourth posture information of the preset finger skeletal point; wherein the finger state information includes finger rotation information and finger posture information;

[0242] A display module is configured to display the target part in the video frame to be processed based on the wrist state information and the finger state information.

[0243] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0244] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0245] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. An image processing method, characterized in that: include: Determine a skeleton chain to be registered and a preset skeleton chain of the target part in the video frame to be processed; wherein the skeleton chain to be registered includes wrist skeleton points and finger skeleton points to be registered, and the preset skeleton chain includes preset wrist skeleton points and preset finger skeleton points. The skeleton chain to be registered is the skeleton model of the target part to be registered, and the preset skeleton chain is the skeleton model of the target part to be registered. Determining wrist state information of the wrist skeletal point to be registered according to the first pose information of the wrist skeletal point to be registered and the second pose information of the preset wrist skeletal point; wherein the wrist state information includes wrist rotation information and wrist pose information; Determining the finger state information of the finger skeletal point to be registered based on the wrist state information, the third posture information of the finger skeletal point to be registered, and the fourth posture information of the preset finger skeletal point; wherein the finger state information includes finger rotation information and finger posture information; Based on the wrist state information and the finger state information, the target part is displayed in the video frame to be processed.

2. The method according to claim 1, characterized in that The step of determining the wrist state information of the wrist skeletal point to be registered according to the first pose information of the wrist skeletal point to be registered and the second pose information of the preset wrist skeletal point comprises: Determine first rotation information according to the first pose information of the wrist skeleton point to be registered and third pose information of at least one finger skeleton point to be registered that is directly associated with the wrist skeleton point to be registered; determining second rotation information based on the second posture information of the preset wrist skeletal point and fourth posture information of at least one preset finger skeletal point directly associated with the preset wrist skeletal point; Based on the first rotation information and the second rotation information, wrist state information of the wrist skeleton point to be registered is determined.

3. The method according to claim 2, characterized in that The step of determining the wrist state information of the wrist skeleton point to be registered based on the first rotation information and the second rotation information includes: determining an alignment transformation matrix according to the second rotation information and the inverse of the first rotation information; The wrist state information is determined based on the alignment transformation matrix and the first position information of the wrist skeleton point to be registered.

4. The method according to claim 3, characterized in that Determining wrist pose information in the wrist state information based on the alignment transformation matrix and the first pose information of the wrist skeleton point to be registered, comprising: determining, based on the position information in the second posture information, the position information in the wrist posture information; Posture information in the wrist pose information is determined based on the alignment transformation matrix and position information in the first pose information.

5. The method according to claim 1, wherein Before determining the finger state information of the finger skeletal point to be registered based on the wrist state information, the third posture information of the finger skeletal point to be registered, and the fourth posture information of the preset finger skeletal point, the method further includes: Retrieving pre-set first dependency information of the wrist skeletal point to be registered and the finger skeletal point to be registered, as well as second dependency information of the preset wrist skeletal point and the preset finger skeletal point; wherein the dependency information includes the finger skeletal point or wrist skeletal point directly dependent on at least one finger skeletal point located on the same branch in the skeletal chain; Based on the first dependency information and the second dependency information, the processing priority of the finger skeleton points to be registered and the preset finger skeleton points is determined, so as to determine the finger status information of the finger skeleton points to be registered in sequence based on the processing priority.

6. The method according to claim 5, characterized in that The determining of the finger state information of the finger skeletal point to be registered based on the wrist state information, the third posture information of the finger skeletal point to be registered, and the fourth posture information of the preset finger skeletal point includes: Determine at least one finger skeleton point group; wherein the finger skeleton point group includes preset finger skeleton points and finger skeleton points to be registered that are located at corresponding skeleton chain branches and have the same processing priority; For the at least one finger skeleton point group, determining, according to the processing priority, a front-dependent skeleton point group and a back-dependent skeleton point group on which the current finger skeleton point group depends; wherein the front-dependent skeleton point group includes a wrist skeleton point group or a finger skeleton point group; If the front-dependent skeleton point group is a wrist skeleton point group, the finger state information of the finger skeleton points to be aligned in the current finger skeleton point group is determined based on the wrist rotation information in the wrist state information, the posture information in the current finger skeleton point group, and the posture information in the rear-dependent skeleton point group.

7. The method according to claim 6, characterized in that Also includes: If the front-dependent skeleton point group is a finger skeleton point group, the finger state information of the finger skeleton points to be registered in the current finger skeleton point group is determined based on the target finger rotation information in the finger state information of the bone points to be registered in the front-dependent skeleton point group, the posture information in the current finger skeleton point group, and the posture information of the rear-dependent skeleton point group.

8. The method according to claim 7, characterized in that The method of determining the finger state information of the finger skeletal points to be registered in the current finger skeletal point group according to the target finger rotation information in the finger state information of the skeletal points to be registered in the front dependent skeletal point group, the posture information in the current finger skeletal point group, and the posture information of the rear dependent skeletal point group includes: Determine the current finger rotation information of the to-be-registered skeletal points in the current finger skeletal point group according to the target finger rotation information, the third pose information and the fourth pose information in the current finger skeletal point group, and the third pose information and the fourth pose information in the subsequent dependent skeletal point group; Determine the current finger pose information of the to-be-registered skeleton points in the current finger skeleton point group based on the target finger rotation information and the current finger rotation information; Based on the current finger rotation information and the current finger pose information, the finger state information of the to-be-registered skeleton point is determined.

9. The method according to claim 8, characterized in that The step of determining the current finger rotation information of the to-be-registered skeletal points in the current finger skeletal point group according to the target finger rotation information, the third pose information and the fourth pose information in the current finger skeletal point group, and the third pose information and the fourth pose information in the subsequent dependent skeletal point group comprises: Determining a third rotation vector according to the third posture information in the current finger skeleton point group and the third posture information in the subsequent dependent skeleton point group; and Determining a fourth rotation vector according to the fourth pose information in the current finger skeleton point group and the fourth pose information in the subsequent dependent skeleton point group; Determine, based on the third rotation vector and the fourth rotation vector, a rotation matrix for rotating the finger skeleton points to be registered in the current finger skeleton point group to the corresponding preset finger skeleton points; Based on the rotation matrix and the target finger rotation information, current finger rotation information of the to-be-registered bone points in the current finger bone point group is determined.

10. The method according to claim 1, characterized in that The step of displaying the target part in the to-be-processed video frame based on the wrist state information and the finger state information includes: Based on the wrist state information and the finger state information, the display position and display posture of each skeletal point in the target part in the display interface are adjusted to obtain a target video frame corresponding to the video frame to be processed.

11. The method according to claim 1, wherein Also includes: If it is determined based on the wrist state information that the position offset between the wrist skeletal point to be registered and the preset wrist skeletal point is greater than a preset offset threshold, then determining the length information of the arm skeletal chain and / or leg skeletal chain to which the wrist skeletal point to be registered belongs; Based on the length information and the position offset, the display position of at least one bone point in the arm bone chain and / or the leg bone chain is adjusted.

12. An image processing device, characterized in that: include: A skeleton chain determination module is used to determine the skeleton chain to be registered and the preset skeleton chain of the target part in the video frame to be processed; wherein the skeleton chain to be registered includes the wrist skeleton points to be registered and the finger skeleton points to be registered, and the preset skeleton chain includes the preset wrist skeleton points and the preset finger skeleton points. The skeleton chain to be registered is the skeleton model of the target part to be registered, and the preset skeleton chain is the skeleton model of the target part to be registered; A wrist determination module, configured to determine wrist state information of the wrist skeletal point to be registered based on the first pose information of the wrist skeletal point to be registered and the second pose information of the preset wrist skeletal point; wherein the wrist state information includes wrist rotation information and wrist pose information; a finger determination module, configured to determine finger state information of the finger skeletal point to be registered based on the wrist state information, the third posture information of the finger skeletal point to be registered, and the fourth posture information of the preset finger skeletal point; wherein the finger state information includes finger rotation information and finger posture information; A display module is configured to display the target part in the video frame to be processed based on the wrist state information and the finger state information.

13. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method according to any one of claims 1 to 11.

14. A storage medium containing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to perform the image processing method according to any one of claims 1 to 11.

Images