Image processing method and device and electronic equipment

Abstract

The invention provides an image processing method and apparatus, and an electronic device. The method comprises the steps of obtaining target image data of a target part of a first object; based on the target image data, determining a spatial relationship parameter between a target part of the first object and at least one predetermined joint point of a second object, the second object being a virtual image, and the predetermined joint point being defined according to a topological structure of a skeleton model of the second object and associated with a to-be-adjusted part of the second object; calculating a rotation control parameter corresponding to the to-be-adjusted part based on the spatial relationship parameter; and based on the rotation control parameter, driving a corresponding predetermined joint point in the skeleton model to perform rotation transformation so as to enable the to-be-adjusted part of the second object to face the target part of the first object.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, specifically to an image processing method, apparatus, and electronic device. Background Technology

[0002] In existing virtual avatar interactive products, the virtual avatar's body and eyes are often fixed in a preset direction, lacking natural communication with the user, resulting in a lack of realism and immersion in the interaction. Summary of the Invention

[0003] In view of the above problems, this disclosure provides an image processing method, apparatus and electronic device.

[0004] This disclosure provides an image processing method, comprising: obtaining target image data of a target part of a first object; determining spatial relationship parameters between the target part of the first object and at least one predetermined joint point of a second object based on the target image data, wherein the second object is a virtual image, and the predetermined joint point is defined according to the topological structure of the skeletal model of the second object and associated with the part of the second object to be adjusted; calculating rotation control parameters corresponding to the part to be adjusted based on the spatial relationship parameters; and driving the corresponding predetermined joint point in the skeletal model to undergo rotation transformation based on the rotation control parameters, so that the part of the second object to be adjusted faces the target part of the first object.

[0005] According to embodiments of this disclosure, determining spatial relationship parameters between a target part of a first object and at least one predetermined joint point of a second object based on target image data includes: determining first position information of the target part in a first coordinate system based on the target image data, the first coordinate system being defined based on the imaging plane of the image acquisition device; transforming the first position information from the first coordinate system to a second coordinate system to obtain second position information, the second coordinate system being defined based on the position of the second object on the display screen; obtaining third position information of the predetermined joint point of the second object in the second coordinate system; and calculating a direction vector from the predetermined joint point to the target part based on the second position information and the third position information to obtain spatial relationship parameters.

[0006] According to embodiments of this disclosure, based on spatial relationship parameters, the rotation control parameters corresponding to the part to be adjusted are calculated, including: for the part to be adjusted, obtaining the reference axis of its associated predetermined joint in the local coordinate system under the initial posture, the initial posture being the standard posture defined by the second object in the skeletal model, and the local coordinate system being a three-dimensional coordinate system with the predetermined joint as the origin and its axis defined by the inherent orientation of the predetermined joint in the skeletal model; and calculating the amount of rotation required to rotate the reference axis to be aligned with the direction of the spatial relationship parameters based on the spatial relationship parameters between the predetermined joint and the target part, as the rotation control parameter corresponding to the part to be adjusted.

[0007] According to embodiments of this disclosure, a reference axis in the local coordinate system of a predetermined joint point associated with it in an initial posture is obtained. Based on the spatial relationship parameters between the predetermined joint point and the target part, the amount of rotation required to rotate the reference axis to align with the direction of the spatial relationship parameters is calculated. This includes: for a first part to be adjusted of a second object, obtaining a first reference axis in the local coordinate system of a first predetermined joint point associated with the first part to be adjusted; and based on the first spatial relationship parameters corresponding to the first predetermined joint point, calculating a first rotation angle required to rotate the projection of the first reference axis in a first reference plane of a second coordinate system to align with the projection direction of the first spatial relationship parameters in the first reference plane, which is used as a rotation control parameter corresponding to the first part to be adjusted.

[0008] According to embodiments of this disclosure, the reference axis of the associated predetermined joint point in the local coordinate system under the initial posture is obtained, and the amount of rotation required to align the reference axis with the direction of the spatial relationship parameter is calculated based on the spatial relationship parameter between the predetermined joint point and the target part. The method further includes: for the second part to be adjusted of the second object, obtaining the second reference axis in the local coordinate system of the second predetermined joint point associated with the second part to be adjusted; and calculating the second rotation angle required to align the projection of the second reference axis in the second reference plane of the second coordinate system with the projection direction of the second spatial relationship parameter in the second reference plane based on the second spatial relationship parameter corresponding to the second predetermined joint point, as the rotation control parameter corresponding to the second part to be adjusted; wherein the second reference plane and the first reference plane have a preset spatial angle.

[0009] According to embodiments of this disclosure, obtaining the reference axis in the local coordinate system of the associated predetermined joint point in the initial posture, and calculating the amount of rotation required to rotate the reference axis to align with the direction of the spatial relationship parameter based on the spatial relationship parameter between the predetermined joint point and the target part, further includes: for the third part to be adjusted of the second object, obtaining the third reference axis in the local coordinate system of the third predetermined joint point associated with the third part to be adjusted; and calculating the third rotation transformation matrix required to rotate the third reference axis to align with the direction of the third spatial relationship parameter in the local coordinate system of the second predetermined joint point based on the third spatial relationship parameter corresponding to the third predetermined joint point, as the rotation control parameter corresponding to the third part to be adjusted.

[0010] According to embodiments of this disclosure, when there are multiple predetermined joints, rotational control parameters are used to drive the corresponding predetermined joints in the skeletal model to undergo rotational transformation so that the part to be adjusted of the second object faces the target part of the first object. This includes: applying the rotational control parameters corresponding to each predetermined joint to drive the rotational transformation of the predetermined joint in sequence according to the driving order from parent joint to child joint as defined by the topology of the skeletal model; wherein, for predetermined joints with a parent-child relationship, the rotational control parameters for driving the rotational transformation of the child joint are based on the rotational transformation result of its parent joint.

[0011] According to embodiments of this disclosure, obtaining target image data of a target part of a first object includes: obtaining original image data containing the facial region of the first object; performing facial key point detection on the original image data to locate the position of the eye region of the first object in the original image data; and extracting image data corresponding to the eye region from the original image data based on the position of the eye region as target image data.

[0012] This disclosure also provides an image processing apparatus, comprising: an acquisition module for acquiring target image data of a target part of a first object; a determination module for determining spatial relationship parameters between the target part of the first object and at least one predetermined joint point of a second object based on the target image data, wherein the second object is a virtual image, and the predetermined joint point is defined according to the topological structure of the skeletal model of the second object and associated with the part of the second object to be adjusted; a calculation module for calculating rotation control parameters corresponding to the part to be adjusted based on the spatial relationship parameters; and a driving module for driving the corresponding predetermined joint point in the skeletal model to perform rotation transformation based on the rotation control parameters, so that the part of the second object to be adjusted faces the target part of the first object.

[0013] This disclosure also provides an electronic device, comprising: an image acquisition device for acquiring target image data of a target part of a first object; a display screen for displaying a second object, the second object being a virtual image; one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the following method: determining spatial relationship parameters between the target part of the first object and at least one predetermined joint point of the second object based on the target image data, wherein the predetermined joint point is defined according to the topological structure of the skeletal model of the second object and associated with the part of the second object to be adjusted; calculating rotation control parameters corresponding to the part to be adjusted based on the spatial relationship parameters; and driving the corresponding predetermined joint point in the skeletal model to undergo rotational transformation based on the rotation control parameters, so that the part of the second object to be adjusted faces the target part of the first object.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0016] Figure 1 The illustrations depict application scenarios of the image processing method, apparatus, and electronic device according to embodiments of the present disclosure.

[0017] Figure 2 A flowchart illustrating an image processing method according to an embodiment of the present disclosure is shown schematically.

[0018] Figure 3 A schematic diagram of a first coordinate system according to an embodiment of the present disclosure is shown;

[0019] Figure 4 A schematic diagram of a second coordinate system according to an embodiment of the present disclosure is shown;

[0020] Figure 5 A schematic block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown; and

[0021] Figure 6 A block diagram schematically illustrates an electronic device suitable for implementing an image processing method according to an embodiment of the present disclosure. Detailed Implementation

[0022] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0024] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0025] It should be noted that the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information in this disclosed technical solution comply with relevant laws and regulations, necessary confidentiality measures have been taken, and it does not violate public order and good morals. In this disclosed technical solution, user authorization or consent has been obtained before acquiring or collecting user personal information.

[0026] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0027] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0028] This disclosure provides an image processing method, apparatus, and electronic device, which will be described below with reference to the accompanying drawings.

[0029] Figure 1 Figure 100 schematically illustrates an application scenario of an image processing method, apparatus, and electronic device according to embodiments of the present disclosure.

[0030] It is important to note that Figure 1 The examples shown are merely examples of scenarios in which the embodiments of this disclosure can be applied, to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0031] like Figure 1As shown, the application scenario 100 according to this embodiment may include a first object 101, which may be a real user who is captured by the system through an image acquisition device (such as a camera) to obtain image data of his target part (such as the eye).

[0032] Application scenario 100 also includes a second object 102, which is a virtual image presented on a display device. The posture of the second object 102 can be dynamically adjusted according to its interaction with the first object 101.

[0033] Application scenario 100 may also include terminal device 103, which may include, but is not limited to, personal computers, game consoles, mobile terminals, or embedded systems. Terminal device 103 can receive data from image acquisition devices, perform processing such as image recognition, 3D spatial calculation, and pose generation algorithms to determine the spatial relationship between the first object 101 and the second object 102, and thereby drive the predetermined joints of the second object 102 to rotate and transform, thereby realizing natural interactive behaviors such as eye contact and head following.

[0034] It should be noted that the image processing method provided in this embodiment can generally be executed by the terminal device 103. Correspondingly, the image processing apparatus provided in this embodiment can generally be disposed in the terminal device 103. The image processing method provided in this embodiment can also be executed by a processing device different from the terminal device 103 but capable of communicating with the terminal device 103. Correspondingly, the image processing apparatus provided in this embodiment can also be disposed in a processing device different from the terminal device 103 but capable of communicating with the terminal device 103.

[0035] It should be understood that Figure 1 The number of terminal devices, first objects, and second objects shown is merely illustrative. Depending on implementation needs, there can be any number of terminal devices, first objects, and second objects.

[0036] The following will be based on Figure 1 The described scene, through Figures 2-4 The image processing method according to the embodiments of this disclosure will be described in detail.

[0037] like Figure 2 As shown, the image processing method of this embodiment may include S210~S240.

[0038] In S210, target image data of the target part of the first object is obtained.

[0039] The first target can be a real user, such as a user in a human-computer interaction, virtual meeting, game, or live streaming scenario.

[0040] The first object can also be any physical entity that needs to be noticed by the virtual avatar, such as a projection of another virtual avatar into the physical world, a specific object, or content displayed on a screen.

[0041] The target area can be set as the eye area of ​​the first object (e.g., a real user).

[0042] In other embodiments, the target area may be the center point of the face or the center of the eyebrows, serving as a stable reference point; or the target area may be the entire head region, with the gaze being inferred comprehensively through head posture.

[0043] The target image data can be the raw image and video stream, or it can be preprocessed data, such as the coordinates of two-dimensional or three-dimensional facial key points output by a machine learning model, the head posture angles (pitch, yaw, roll) calculated directly, and the precise gaze direction vector.

[0044] In S220, based on the target image data, spatial relationship parameters between the target part of the first object and at least one predetermined joint point of the second object are determined, wherein the second object is a virtual image, and the predetermined joint point is defined according to the topological structure of the skeletal model of the second object and associated with the part of the second object to be adjusted.

[0045] The second object is the virtual avatar (digital human) to be driven. Its skeletal model predefines joints associated with the body parts that need to be adjusted, including but not limited to root nodes, head nodes, and eye nodes. Among them, the root node can correspond to the pelvis or center of gravity of the virtual avatar, determining the overall orientation; the head node can control the rotation and tilt of the head; and the eye nodes can directly drive the gaze direction of the eyeballs.

[0046] Spatial relationship parameters can characterize the relative geometric relationship between the target part and a specific joint of the virtual image in three-dimensional space, such as the direction vector from a certain joint of the virtual image to the user's target part.

[0047] The calculation methods can be designed differently for different types of joints. For the root node, a horizontal vector pointing to the user's target area can be calculated to control the approximate orientation of the body; for the head node, a precise three-dimensional direction vector can be calculated to control the rotation of the head; for the eye node, vectors pointing to the target area from the left and right eyes can be calculated separately to simulate the focusing of the eyeballs.

[0048] In S230, based on the spatial relationship parameters, the rotation control parameters corresponding to the part to be adjusted are calculated.

[0049] Rotation control parameters are instruction data that drive the rotation transformation of specific parts of a virtual avatar. For each part to be adjusted (such as the body, head, or eyes), the amount of rotation required to adjust the part from its current posture to face the target part can be calculated based on the spatial relationship parameters between its associated predetermined joints and the target part.

[0050] In other embodiments, different response weights and kinematic constraints can be configured for different joints to determine rotation control parameters. For example, for the root node that drives body orientation, its rotation response can be configured with a low weight and a large angular range of motion (e.g., horizontal rotation angle limited to ±60 degrees), and the response speed can be set relatively slowly to simulate body inertia; the rotation response of the head node can be configured with a medium weight and a moderate angular range of motion (e.g., pitch angle limited to ±45 degrees), and the response speed can be set moderately; the eye node can be given the highest weight and the most restrictive angular range of motion (e.g., both horizontal and vertical rotation are limited to ±30 degrees), and the response speed can be set to the fastest to achieve precise and rapid fine-tuning.

[0051] To eliminate jitter and enhance the naturalness of the motion, the calculated raw rotation control parameters can be smoothed, for example by performing spherical linear interpolation between frames, applying a low-pass filter, or using a spring-damped physical model to simulate a smooth following effect.

[0052] In S240, based on rotation control parameters, the corresponding predetermined joints in the skeletal model are driven to rotate so that the part of the second object to be adjusted faces the target part of the first object.

[0053] Based on the calculated rotation control parameters, the corresponding predetermined joints can be driven to perform rotation transformation, so that the space of the part to be adjusted of the second object is oriented towards the target part of the first object.

[0054] For example, for non-end-effector parts such as the eyes and head, where orientation adjustment is the primary concern, rotation control parameters can be expressed in the form of quaternions or Euler angles. In application, algorithms such as spherical linear interpolation can be used to smoothly and continuously transition the joint's current rotation state to the target rotation state. For end-effector parts such as the hands and feet, which require precise arrival at a specific point in space, if the interactive task involves adjustments to these parts, inverse kinematics algorithms can be employed. This algorithm uses the adjusted target spatial position as a constraint to automatically and inversely calculate the required rotational angle combinations for all intermediate joints along the entire skeletal chain, from the root of the drive chain (such as the shoulder and hip joints) to the end (such as the wrist and ankle).

[0055] It should be noted that, in order not to affect other animations being performed by the digital human (such as lip movements and gestures), an animation layer overlay technique can be used, that is, the "gaze-driven" animation can be treated as an independent, blendable animation layer and integrated with the basic animation. In addition, to further enhance the realism, secondary movements (such as delayed swaying of hair and accessories) and accompanying micro-expressions (such as slight eyebrow raising or blinking during gaze) can be added on top of the main gaze-driven animation, and intermittent gaze-breaking logic can be designed to avoid creating an unnatural continuous gaze.

[0056] In other embodiments, the number of predetermined joints can be multiple. During rotation transformation, the corresponding predetermined joints can be driven sequentially to perform rotation transformations. During the driving process, the hierarchical and forward kinematics principles of skeletal animation can be followed. That is, first, the calculated rotation parameters are applied to the root node, causing the entire torso to turn. Then, based on the torso transformation, the head node is rotated. Finally, based on the head transformation, the left and right eye nodes are rotated.

[0057] By controlling the coordinated rotation of multiple joints, the system simulates the complete natural behavior of a real human when looking, from turning the torso to fine-tuning the head and finally locking the gaze. This ensures a high degree of coordination between the various parts of the digital human in terms of movement timing and spatial relationships, and makes its overall posture present a smoothness and realism that conforms to the laws of biomechanics, thereby significantly enhancing the vividness of human-computer interaction and the user's immersive experience.

[0058] For example, firstly, image data of the user's (first object's) eye area (target area) is acquired via a camera, and its three-dimensional spatial coordinates are located. Then, using the cervical spine joints (predetermined joints) in the head skeleton model of the virtual avatar (second object) as driving points, the direction vector between these joints and the midpoint of the line connecting the user's eyes is calculated as a spatial relationship parameter. Next, based on this direction vector, the pitch and yaw angles required to align the virtual avatar's head with the user's eyes are calculated and converted into rotation control parameters in quaternion form. Then, an interpolation algorithm is used to smoothly apply these parameters to the cervical spine joints, driving the head skeleton to rotate and transform, thereby making the virtual avatar's gaze naturally and in real-time aligned with the user's eyes, forming a realistic eye-tracking interaction effect.

[0059] Understandably, compared to existing technologies that rely solely on two-dimensional image information without introducing accurate three-dimensional spatial calculations, this disclosure embodiment establishes and utilizes the spatial relationship between the target part of the first object and the joints of the second object to achieve rotational control of the part of the second object to be adjusted, thereby greatly enhancing the vividness, naturalness, and user immersion of human-computer interaction.

[0060] Based on the above embodiments, in this embodiment, obtaining target image data of the target part of the first object includes: obtaining original image data containing the facial region of the first object; performing facial key point detection on the original image data to locate the position of the eye region of the first object in the original image data; and extracting image data corresponding to the eye region from the original image data based on the position of the eye region as target image data.

[0061] The original image data of the facial region can fully contain the geometric, texture and structural information of the first object's face, such as facial contours and the distribution of facial features.

[0062] By performing facial keypoint detection and feature localization on the raw image data, the coordinate position of the eye region of the first object in the image space can be identified. Specifically, a pre-trained deep learning model (such as a facial feature point detection model based on a convolutional neural network) can be loaded, or traditional computer vision algorithms can be used to analyze the input image. The detection process can output a set of key feature point coordinates closely related to the eye structure, such as: the inner and outer corners of the left and right eyes, the center point of the upper eyelid, the center point of the lower eyelid, and the center of the pupil. Based on the spatial distribution of these feature points, a boundary region that can completely cover both eyes or one eye can be calculated (e.g., by calculating the convex hull of the feature points or fitting the minimum bounding rectangle / ellipse). This region can be defined as the eye region to be extracted.

[0063] Based on the determined boundary coordinates of the eye region, sub-image data corresponding to that region can be extracted from the original image data. This extraction process may include pixel-level cropping, region mask generation and synthesis, or image encoding. Furthermore, image enhancement operations, such as contrast stretching, illumination normalization, or scale normalization, can be performed during extraction to improve the robustness of subsequent processing. Finally, the obtained sub-image data serves as the output target image data, which can highly focus on key visual features such as eye shape, gaze direction, and eye opening / closing status.

[0064] Understandably, by locating and extracting target image data that reflects the state of the eyes, the high relevance and accuracy of the information upon which subsequent calculations depend are ensured, laying a data foundation for precise spatial relationship analysis. At the same time, by focusing on key areas, data processing efficiency is effectively improved and overall computational complexity is reduced.

[0065] Based on the above embodiments, in this embodiment, determining the spatial relationship parameters between a target part of a first object and at least one predetermined joint point of a second object based on target image data includes: determining first position information of the target part in a first coordinate system based on the target image data, the first coordinate system being defined based on the imaging plane of the image acquisition device; transforming the first position information from the first coordinate system to a second coordinate system to obtain second position information, the second coordinate system being defined based on the position of the second object on the display screen; obtaining third position information of the predetermined joint point of the second object in the second coordinate system; and calculating the direction vector from the predetermined joint point to the target part based on the second position information and the third position information to obtain the spatial relationship parameters.

[0066] The first coordinate system (such as the camera coordinate system) is defined based on the imaging plane and optical center of the image acquisition device. Its origin can be located at the optical center of the device, and the Z-axis is along the optical axis.

[0067] Image acquisition devices can be monocular RGB cameras, monocular grayscale cameras, or binocular cameras, structured light cameras, and other cameras that can directly acquire depth information.

[0068] Based on the target image data, the three-dimensional coordinates of the target part (such as the center point of the eye region) in the first coordinate system can be determined by image processing and three-dimensional reconstruction technology (such as monocular depth estimation, binocular vision or structured light depth perception). These coordinates are the first position information.

[0069] For example, the image acquisition device acquires a face image of a user (such as a person whose face occupies the largest proportion in the image), first detects the 2D coordinates of the eyes in the image, if it is a monocular camera, the monocular depth estimation algorithm can be used to obtain the 3D coordinates of the human eye in the camera coordinate system, if it is a depth camera, the 3D coordinates of the human eye in the camera coordinate system can be obtained directly through the 2D coordinates of the eyes, and finally the 3D coordinates of the eye point C in the left and right eyes are calculated.

[0070] The second coordinate system (such as the world coordinate system or screen coordinate system) is defined based on the rendered position of the second object on the display screen. For example, the center of the digital human rendering window can be defined as the origin of this coordinate system, with its axis aligned with the edge of the screen. Since the relative positions of the image acquisition device and the display screen in physical space are fixed and known, the transformation parameters (including the rotation matrix R and the translation vector t) from the first coordinate system to the second coordinate system can be pre-calibrated. Using these transformation parameters, the first position information can be transformed from the first coordinate system to the second coordinate system to obtain the coordinates of the target part in the second coordinate system, i.e., the second position information. Mathematically, this can be expressed as: Second position information = R·First position information + t.

[0071] In the skeletal model of the second object (virtual avatar), each predetermined joint (such as the root node, head node, and eye node) has its own inherent coordinates in its own model coordinate system or virtual camera coordinate system. According to the coordinate system definition rules of the virtual rendering engine and the known transformation relationship between the virtual camera and the world coordinate system, the coordinates of these joints can be uniformly transformed to the second coordinate system, thereby obtaining the coordinates of each predetermined joint in the second coordinate system, i.e., the third position information.

[0072] For each predetermined joint to be processed, based on its third position information in the second coordinate system and the second position information of the target part in the second coordinate system, a three-dimensional direction vector pointing from the joint to the target part can be calculated by vector subtraction. This direction vector serves as a spatial relationship parameter. The direction of this vector can represent the spatial orientation required from the virtual image's position at the joint to the user's target part, and the magnitude of this vector can reflect the spatial distance between the two.

[0073] For example, after the image acquisition device acquires a user's facial image, it can obtain the image coordinates of the eye region through face detection and key point localization algorithms. Then, combined with the in-camera depth information (e.g., through monocular depth estimation or directly from a depth camera), the three-dimensional coordinates E of the eye center point C in the camera coordinate system can be calculated. c Then, the position of the digital human rendering window center on the screen can be defined as the origin of the world coordinate system. Using the pre-calibrated camera-screen transformation matrix [R|t], the human eye coordinates can be transformed to the world coordinate system, i.e., E. wc =R·E c +t. Next, the coordinates D of the root node in the digital human skeleton model in the virtual camera coordinate system can be obtained. r (X) dr ,Y dr Z dr According to the coordinate system rules of the rendering engine (e.g., ... Figure 3 and Figure 4 As shown, the virtual camera overlaps with the origin of the world coordinate system (the X and Z axes are in the same direction, and the Y axis is in the opposite direction). It can be transformed to the world coordinate system to obtain D. wr (X) dr ,-Y dr Z dr Finally, the calculation can be performed from the root node D of the digital human. wr Pointing to the human eye position E wc The direction vector, i.e., Vec=E wc -D wr This vector can be used as a spatial relationship parameter to control the orientation of the digital human's body. Similarly, the vector pointing from the head node and eye node to E can be calculated. wcThe vectors are used to control the rotation of the head and eyes, respectively.

[0074] By using multi-step coordinate transformation and direction vector calculation, the spatial geometric relationship between the user and the virtual avatar can be established more accurately.

[0075] Based on the above embodiments, in this embodiment, the rotation control parameters corresponding to the part to be adjusted are calculated based on the spatial relationship parameters, including: for the part to be adjusted, obtaining the reference axis of the associated predetermined joint point in the local coordinate system under the initial posture, the initial posture being the standard posture defined by the second object in the skeletal model, and the local coordinate system being a three-dimensional coordinate system with the predetermined joint point as the origin and its axis defined by the inherent orientation of the predetermined joint point in the skeletal model; and calculating the amount of rotation required to rotate the reference axis to be aligned with the direction of the spatial relationship parameters according to the spatial relationship parameters between the predetermined joint point and the target part, which is used as the rotation control parameter corresponding to the part to be adjusted.

[0076] For each part to be adjusted, the reference axis of its associated predetermined joint in the local coordinate system under the initial pose can first be obtained. The initial pose is the standard pose (e.g., a T-pose or bound pose) defined by the second object in its skeletal model and not driven by any animation. The local coordinate system is fixed to the joint, with its origin at the joint, and the orientation of its three axes can be defined by the inherent structure of the joint in the skeletal model (e.g., the Z-axis can be defined as the "forward" direction of the joint's extension, and the Y-axis as the "upward" direction). This reference axis can be selected as the axis in the local coordinate system that represents the default orientation of the part; for example, for the head or eyeball joint, its reference axis can be the local Z-axis.

[0077] Based on the determined reference axis and the obtained spatial relationship parameters associated with the predetermined joint point (i.e., the direction vector from the joint point to the target part of the first object), the rotational transformation amount can be calculated. This rotational transformation amount characterizes the rotational operation required to rotate the reference axis from its initial direction in three-dimensional space until it is completely aligned with the direction of the spatial relationship parameters. This rotational amount is the rotational control parameter corresponding to the part to be adjusted.

[0078] In some embodiments, the reference axis (denoted as unit vector A) and the direction vector corresponding to the spatial relationship parameter (denoted as unit vector B) can be transformed to the same reference coordinate system (e.g., the world coordinate system). The rotational transformation between the two can be calculated by solving for the rotation moment R or the quaternion Q, such that R·A=B (or Q rotates A to B). This rotation matrix R or quaternion Q is the desired rotation amount. For certain parts with specific motion constraints (such as the body that is only allowed to rotate horizontally), the vector can be projected onto a specific plane (e.g., a horizontal plane) first, and then the two-dimensional rotation angle within that plane can be calculated.

[0079] Understandably, by introducing a reference axis based on the standard posture and the inherent orientation of the bones, a clear and stable rotational reference is established for each part to be adjusted.

[0080] Based on the above embodiments, in this embodiment, the reference axis of the associated predetermined joint point in the local coordinate system under the initial posture is obtained. According to the spatial relationship parameters between the predetermined joint point and the target part, the amount of rotation required to rotate the reference axis to be aligned with the direction of the spatial relationship parameters is calculated, including: for the first part to be adjusted of the second object, obtaining the first reference axis in the local coordinate system of the first predetermined joint point associated with the first part to be adjusted; based on the first spatial relationship parameters corresponding to the first predetermined joint point, calculating the first rotation angle required to rotate the projection of the first reference axis in the first reference plane of the second coordinate system to be aligned with the projection direction of the first spatial relationship parameters in the first reference plane, as the rotation control parameter corresponding to the first part to be adjusted.

[0081] In the skeletal model of the second object, a first predetermined joint point (root node) associated with the first part to be adjusted (i.e., the body part) can be located. This node can be defined at the character's pelvis or torso center of gravity to control the displacement and basic orientation of the entire model. Under the initial standard pose defined by the skeletal model, the local coordinate system of the root node can be obtained. In this coordinate system, an axis along the natural extension direction of the bones can be selected as the first reference axis. In standard human modeling, this axis can be the positive Z-axis direction of the local coordinate system, which can semantically represent the character's "facing forward" direction when no animation is applied.

[0082] Based on the calculated first spatial relation parameter, that is, defined in the world coordinate system from the root node position D wr Pointing to the user's target area (such as the center of the eyes E) wc The three-dimensional direction vector vec r To decouple body motion, this vector vec can be... r The first reference plane, projected onto the second coordinate system (world coordinate system), can be defined as a global horizontal plane, resulting in the projection vector vec_r_h. This filters out the vertical component, meaning the body's initial adjustments only respond to changes in the target's horizontal orientation. Simultaneously, the first reference axis representing the body's forward direction is also transformed to the world coordinate system and projected onto the same horizontal plane, yielding its projection vector Z_body_h. Next, the angle between vec_r_h and Z_body_h can be calculated. This angle calculation, combined with cross product operations, determines the rotation direction (clockwise or counterclockwise), ultimately yielding a signed first rotation angle (yaw angle), which serves as the control variable driving the body's horizontal rotation around the vertical axis.

[0083] Understandably, by projecting onto the first reference plane, motion decoupling and numerical stability of body orientation adjustment are achieved.

[0084] Based on the above embodiments, in this embodiment, the reference axis of the associated predetermined joint point in the local coordinate system under the initial posture is obtained, and the amount of rotation required to rotate the reference axis to align with the direction of the spatial relationship parameter is calculated according to the spatial relationship parameter between the predetermined joint point and the target part. The method further includes: for the second part to be adjusted of the second object, obtaining the second reference axis in the local coordinate system of the second predetermined joint point associated with the second part to be adjusted; based on the second spatial relationship parameter corresponding to the second predetermined joint point, calculating the second rotation angle required to rotate the projection of the second reference axis in the second reference plane of the second coordinate system to align with the projection direction of the second spatial relationship parameter in the second reference plane, as the rotation control parameter corresponding to the second part to be adjusted; wherein the second reference plane and the first reference plane have a preset spatial angle.

[0085] For the second part to be adjusted (head) of the second object, the local coordinate system of the second predetermined joint point (head node) associated with the part and its second reference axis (which can be the local Z axis, representing the front direction of the face) can be obtained.

[0086] It should be noted that although the initial definition of the local coordinate system of the second predetermined joint is given under the standard posture, the current rotation of its body root node has been applied during actual driving. Therefore, the actual reference axial direction of the head node can be inherited from the new posture after the body rotation to achieve hierarchical motion.

[0087] To achieve the primary pitch movement of the head, the second spatial relationship parameter (the direction vector pointing from the head node to the target) corresponding to the head node can be projected onto a second reference plane. This plane has a preset spatial angle with the first reference plane (horizontal plane), which can be set to a perpendicular relationship (90 degrees). For example, the sagittal plane, or a vertical plane defined by the current body forward axis and the world "up" vector. After projection, the component of this vector in the vertical plane, vec_pitch, is obtained. Simultaneously, the second reference axis of the head (which may include body rotation information) can be projected onto the same vertical plane to obtain Z_head_v. Then, the angle between vec_pitch and Z_head_v in this vertical plane is calculated to obtain the second rotation angle (pitch angle). This angle can represent how many degrees the head still needs to be raised or lowered to make the face more accurately face the target under a determined horizontal orientation.

[0088] Understandably, by calculating the projection in the vertical plane, the head adjustment is constrained to the pitch dimension, thus decoupling it from the horizontal turning of the body. At the same time, since the calculation reference can include the body rotation result, it ensures that the head movement is a relative adjustment based on the current body posture, rather than an absolute world coordinate adjustment, thereby generating physically correct and natural "turning head" and "head up / down" movements.

[0089] Based on the above embodiments, in this embodiment, the reference axis of the associated predetermined joint point in the local coordinate system under the initial posture is obtained, and the amount of rotation required to rotate the reference axis to align with the direction of the spatial relationship parameter is calculated according to the spatial relationship parameter between the predetermined joint point and the target part. The embodiment also includes: for the third part to be adjusted of the second object, obtaining the third reference axis in the local coordinate system of the third predetermined joint point associated with the third part to be adjusted; and calculating the third rotation transformation matrix required to rotate the third reference axis to align with the direction of the third spatial relationship parameter in the local coordinate system of the second predetermined joint point based on the third spatial relationship parameter corresponding to the third predetermined joint point, as the rotation control parameter corresponding to the third part to be adjusted.

[0090] For the third part to be adjusted (eye) of the second object, the local coordinate system of the third predetermined joint point (eye node, which can be the center of the left / right eyeball) and its third reference axis (i.e. the direction of the line of sight) associated with the eye can be obtained.

[0091] It should be noted that all calculations for the eyes can be performed in the current local coordinate system of the head node. That is, we can first obtain the latest transformation matrix of the head node after applying pitch rotation, and then define the position and orientation reference of the eyeball in this transformed head space. This ensures that "eyeball movement" is an action relative to the "current head pose".

[0092] Obtain the third spatial relationship parameter (the direction vector from the eye node to the target) defined in the world coordinate system. Transform this parameter to the local coordinate system of the second predetermined joint point to obtain the target direction vector in the local coordinate system. That is, inversely transform this vector to the head local coordinate system through the current head transformation matrix, thereby obtaining the local expression of the target direction in the head space, vec_eye_local. Then, calculate the third rotation transformation matrix required to rotate the third reference axis (the initial gaze direction in the head local space) to align with the target direction vector in the local coordinate system. In other words, completely align the third reference axis of the eye with the three-dimensional vector vec_eye_local.

[0093] Since the gaze needs to be precisely directed to a point in three-dimensional space, the eyeball needs to possess rotational capabilities in at least two degrees of freedom (e.g., yaw in the horizontal direction and pitch in the vertical direction). Therefore, the transformation used for alignment can utilize mathematical tools capable of representing arbitrary orientations in three-dimensional space. In this embodiment, the third rotation transformation matrix (or an equivalent unit quaternion) can be directly generated through vector operations (e.g., constructing the rotation axis using the cross product and calculating the rotation angle using the dot product). This matrix serves as the final instruction controlling the eyeball's rotation. By independently performing this calculation on the left and right eye nodes, independent rotation matrices can be obtained. These matrices, acting as rotation control parameters for the eyes, drive the eyeball to make correct horizontal and vertical compound rotations based on the current head posture, ultimately achieving precise and natural gazing at the user's target and completing gaze locking in the interaction chain.

[0094] For example, regarding the eyes of a virtual avatar (the third area to be adjusted), firstly, the reference line-of-sight axis (such as the Z-axis) of the eye joint in its local coordinate system can be obtained. Simultaneously, the world space direction vector (the third spatial relationship parameter) pointing from this eye joint to the user's eye target can be obtained. Then, this vector can be transformed into the current local coordinate system after the head joint has undergone its own rotation, thereby obtaining the relative direction of the target within the head space. Furthermore, in this head local coordinate system, the three-dimensional transformation required to rotate the reference line-of-sight axis of the eyeball to be completely aligned with this relative direction can be calculated, generating a third rotation transformation matrix (such as through axis-angle calculation or quaternion interpolation) to achieve binocular convergence and stereoscopic vision simulation.

[0095] Understandably, by calculating the complete rotation transformation matrix in the local space of the second predetermined joint point, high degrees of freedom and high precision of eye control can be achieved, ensuring that no matter how the head moves, the gaze of the eyeball can be precisely adjusted in accordance with anatomical constraints with the head as the reference frame.

[0096] In embodiments of this disclosure, when there are multiple predetermined joints, rotational control parameters are used to drive the corresponding predetermined joints in the skeletal model to undergo rotational transformation so that the part to be adjusted of the second object faces the target part of the first object. This includes: applying the rotational control parameters corresponding to each predetermined joint to drive the rotational transformation of the predetermined joint in sequence according to the driving order from parent joint to child joint as defined by the topology of the skeletal model; wherein, for predetermined joints with a parent-child relationship, the rotational control parameters for driving the rotational transformation of the child joint are based on the rotational transformation result of its parent joint.

[0097] In some embodiments, the driving process can proceed sequentially from the root node (parent joint) to the end node (child joint) according to the parent-child hierarchical dependency defined by the topology of the skeletal model. Specifically, rotation control parameters can first be applied to the root joint to drive the body parts of the second object to perform basic orientation adjustments. After the body rotation is completed, the corresponding rotation control parameters can be applied sequentially to the head joint and eye joint to drive the subsequent turning movements of the head and eyes, respectively.

[0098] It should be noted that the above driving sequence is not only sequential in time, but also forms a transmission chain of spatial transformation. That is, the movement of the child node is always calculated and executed based on the new coordinate system after the spatial transformation of its parent node.

[0099] For example, before calculating or applying the rotation control parameters of a child node, the current rotation transformation result of its parent node (which can be represented as a rotation matrix or quaternion) can be obtained first, and this transformation result can be applied to update the local coordinate system reference frame of the child node. For instance, when calculating the target rotation of the eyeball, instead of directly using the direction vector in the world coordinate system, the vector can be first transformed to the local coordinate system after the head node is transformed, and then the fine-tuning rotation amount required for the eyeball relative to the new coordinate system can be calculated.

[0100] In other embodiments, the driving process can generate detailed motion data logs, recording the rotation timing, angle changes, and coordination indicators of each joint, for offline analysis or real-time evaluation of the degree of coordination of multiple parts of the body (such as the motion delay difference between the body, head, and eyes, trajectory consistency, etc.), and as feedback input for optimizing driving parameters, forming a closed-loop control system.

[0101] Understandably, the driving strategy based on spatial transformation inheritance mathematically guarantees the correct superposition of rotational transformations, avoiding motion distortion caused by asynchronous coordinate systems.

[0102] In another aspect, this disclosure provides an image processing apparatus. Figure 5 A block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown schematically.

[0103] like Figure 5 As shown, the image processing device 500 includes an acquisition module 510, a determination module 520, a calculation module 530, and a driving module 540.

[0104] According to some embodiments of this disclosure, the image processing apparatus 600 can be used to implement the reference. Figures 2-5 The image processing method described according to embodiments of the present disclosure.

[0105] The acquisition module 510 can perform, for example, operation S210, to obtain target image data of the target part of the first object.

[0106] The determining module 520 can perform, for example, operation S220, to determine, based on target image data, spatial relationship parameters between a target part of a first object and at least one predetermined joint point of a second object, wherein the second object is a virtual image, and the predetermined joint point is defined according to the topological structure of the skeletal model of the second object and associated with the part of the second object to be adjusted.

[0107] The calculation module 530 can perform, for example, operation S230, to calculate the rotation control parameters corresponding to the part to be adjusted based on the spatial relationship parameters.

[0108] The drive module 540 can perform, for example, operation S240, to drive the corresponding predetermined joints in the skeletal model to rotate based on rotation control parameters, so that the part of the second object to be adjusted faces the target part of the first object.

[0109] For example, any plurality of the acquisition module 510, determination module 520, calculation module 530, and driving module 540 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the acquisition module 510, determination module 520, calculation module 530, and driving module 540 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the acquisition module 510, determination module 520, calculation module 530, and driving module 540 can be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0110] In another aspect, this disclosure provides an electronic device, including an image acquisition device, a display screen, a storage device, and one or more processors.

[0111] Image acquisition equipment is used to obtain target image data of the target part of the first object.

[0112] The display screen is used to display a second object, which is a virtual avatar.

[0113] A storage device is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors perform the following method: determining spatial relationship parameters between a target part of a first object and at least one predetermined joint point of a second object based on target image data, wherein the predetermined joint point is defined according to the topological structure of the skeletal model of the second object and associated with the part of the second object to be adjusted; calculating rotation control parameters corresponding to the part to be adjusted based on the spatial relationship parameters; and driving the corresponding predetermined joint point in the skeletal model to undergo rotational transformation based on the rotation control parameters so that the part of the second object to be adjusted faces the target part of the first object.

[0114] It should be understood that the electronic device in the embodiments of this disclosure corresponds to the image processing method in the embodiments of this disclosure, and their specific implementation details are the same, which will not be repeated here.

[0115] Figure 6 A block diagram schematically illustrates an electronic device suitable for implementing an image processing method according to an embodiment of the present disclosure.

[0116] like Figure 6 As shown, an electronic device 600 according to an embodiment of this disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this disclosure.

[0117] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 602 and / or RAM 603. It should be noted that programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.

[0118] According to embodiments of this disclosure, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the I / O interface 605: an input section 606 including target hardware, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0119] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0120] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 602 and / or RAM 603 and / or one or more memories other than ROM 602 and RAM 603 described above.

[0121] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the image processing method provided in the embodiments of this disclosure.

[0122] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0123] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0124] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0125] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0126] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0127] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0128] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. An image processing method, comprising: Obtain target image data of the target part of the first object; Based on the target image data, spatial relationship parameters between the target part of the first object and at least one predetermined joint point of the second object are determined, wherein the second object is a virtual image, and the predetermined joint point is defined according to the topological structure of the skeletal model of the second object and associated with the part of the second object to be adjusted. Based on the spatial relationship parameters, calculate the rotation control parameters corresponding to the part to be adjusted; Based on the rotation control parameters, the corresponding predetermined joints in the skeletal model are driven to rotate so that the part of the second object to be adjusted faces the target part of the first object.

2. The method according to claim 1, wherein determining the spatial relationship parameters between the target portion of the first object and at least one predetermined joint point of the second object based on the target image data comprises: Based on the target image data, the first position information of the target part in the first coordinate system is determined, and the first coordinate system is defined based on the imaging plane of the image acquisition device; The first position information is transformed from the first coordinate system to the second coordinate system to obtain the second position information, wherein the second coordinate system is defined based on the position of the second object on the display screen; Obtain the third position information of the predetermined joint points of the second object in the second coordinate system; Based on the second location information and the third location information, a direction vector from the predetermined joint point to the target part is calculated to obtain the spatial relationship parameters.

3. The method according to claim 1, wherein calculating the rotation control parameters corresponding to the part to be adjusted based on the spatial relationship parameters includes: For the part to be adjusted, obtain the reference axis of its associated predetermined joint point in the local coordinate system under the initial posture. The initial posture is the standard posture defined by the second object in the skeleton model. The local coordinate system is a three-dimensional coordinate system with the predetermined joint point as the origin and its axis defined by the inherent orientation of the predetermined joint point in the skeleton model. Based on the spatial relationship parameters between the predetermined joint point and the target part, the amount of rotation required to rotate the reference axis to align with the direction of the spatial relationship parameters is calculated, and this amount is used as the rotation control parameter corresponding to the part to be adjusted.

4. The method according to claim 3, wherein obtaining the reference axis of the associated predetermined joint in the local coordinate system of the part to be adjusted in the initial posture, and calculating the amount of rotation required to rotate the reference axis to align with the direction of the spatial relationship parameter based on the spatial relationship parameter between the predetermined joint and the target part, comprises: For the first part to be adjusted of the second object, obtain the first reference axis in the local coordinate system of the first predetermined joint point associated with the first part to be adjusted; Based on the first spatial relationship parameter corresponding to the first predetermined joint point, calculate the first rotation angle required to rotate the projection of the first reference axis onto the first reference plane in the second coordinate system to align with the projection direction of the first spatial relationship parameter onto the first reference plane, and use this as the rotation control parameter corresponding to the first part to be adjusted.

5. The method according to claim 4, wherein for the part to be adjusted, obtaining the reference axis in the local coordinate system of its associated predetermined joint point in the initial posture, and calculating the amount of rotation required to rotate the reference axis to align with the direction of the spatial relationship parameter based on the spatial relationship parameter between the predetermined joint point and the target part, further comprises: For the second part to be adjusted of the second object, obtain the second reference axis in the local coordinate system of the second predetermined joint point associated with the second part to be adjusted; Based on the second spatial relationship parameter corresponding to the second predetermined joint point, calculate the second rotation angle required to rotate the projection of the second reference axis in the second reference plane of the second coordinate system to align with the projection direction of the second spatial relationship parameter in the second reference plane, and use it as the rotation control parameter corresponding to the second part to be adjusted. The second reference plane and the first reference plane have a preset spatial angle.

6. The method according to claim 5, wherein for the part to be adjusted, obtaining the reference axis in the local coordinate system of its associated predetermined joint point in the initial posture, and calculating the amount of rotation required to rotate the reference axis to align with the direction of the spatial relationship parameter based on the spatial relationship parameter between the predetermined joint point and the target part, further comprises: For the third part to be adjusted of the second object, obtain the third reference axis in the local coordinate system of the third predetermined joint point associated with the third part to be adjusted; Based on the third spatial relationship parameter corresponding to the third predetermined joint point, in the local coordinate system of the second predetermined joint point, the third rotation transformation matrix required to rotate the third reference axis to be aligned with the direction of the third spatial relationship parameter is calculated, and used as the rotation control parameter corresponding to the third part to be adjusted.

7. The method according to claim 1, wherein when there are multiple predetermined joint points, the step of driving the corresponding predetermined joint points in the bone model to undergo rotational transformation based on the rotation control parameters, so that the part to be adjusted of the second object faces the target part of the first object, includes: According to the driving order from parent joint to child joint as defined by the topology of the bone model, the rotation control parameters corresponding to each predetermined joint are sequentially applied to drive the rotation transformation of the predetermined joint. Specifically, for a predetermined joint with a parent-child relationship, the rotation control parameters that drive the rotation transformation of the child joint are based on the rotation transformation result of its parent joint.

8. The method according to claim 1, wherein obtaining target image data of the target part of the first object comprises: Obtain raw image data containing the facial region of the first object; Facial key point detection is performed on the original image data to locate the position of the eye region of the first object in the original image data; Based on the location of the eye region, image data corresponding to the eye region is extracted from the original image data and used as the target image data.

9. An image processing apparatus, comprising: The acquisition module is used to obtain target image data of the target part of the first object; The determining module is used to determine, based on the target image data, the spatial relationship parameters between the target part of the first object and at least one predetermined joint point of the second object, wherein the second object is a virtual image, and the predetermined joint point is defined according to the topological structure of the skeletal model of the second object and associated with the part of the second object to be adjusted; The calculation module is used to calculate the rotation control parameters corresponding to the part to be adjusted based on the spatial relationship parameters; The driving module is used to drive the corresponding predetermined joint points in the bone model to rotate based on the rotation control parameters, so that the part of the second object to be adjusted faces the target part of the first object.

10. An electronic device, comprising: An image acquisition device used to obtain target image data of a target part of a first object; A display screen is used to display a second object, which is a virtual image; One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the following method: Based on the target image data, spatial relationship parameters between the target part of the first object and at least one predetermined joint point of the second object are determined, wherein the predetermined joint point is defined according to the topological structure of the skeletal model of the second object and associated with the part of the second object to be adjusted; based on the spatial relationship parameters, rotation control parameters corresponding to the part to be adjusted are calculated; based on the rotation control parameters, the predetermined joint point in the skeletal model is driven to rotate so that the part of the second object to be adjusted faces the target part of the first object.

Images