Method and apparatus for generating anthropomorphized image, electronic device, and storage medium

Abstract

The present disclosure provides a method and device for generating a personified image, electronic equipment and a storage medium. The method comprises: obtaining a target image containing a target object; determining a personality characteristic and geometric information of the target object based on the target image; determining a facial feature generation area of the target object from the target image based on the geometric information; generating a facial feature line drawing of the target object based on the personality characteristic; and generating an image of the target object personified based on the facial feature line drawing, texture characteristics and material characteristics of the facial feature generation area. Thus, the generation efficiency of the personified image of any object can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to methods, apparatus, electronic devices and storage media for generating anthropomorphic images. Background Technology

[0002] In related technologies, within the fields of computer graphics, augmented reality (AR), and digital entertainment, techniques for anthropomorphizing objects mainly fall into two categories: 1. 3D (Three Dimensional) Modeling and Animation Technology: Typical examples include the production of some animated films. The production of such films requires the complete digital reconstruction of real objects using 3D modeling software, redesigning rigging and skinning, and achieving anthropomorphic animation through manual key-frame or motion capture technology.

[0003] 2. Simple AR sticker technology: Using simple image recognition technology, preset cartoon stickers, such as common eye and mouth materials, are mechanically superimposed on the two-dimensional coordinates of an object's surface.

[0004] However, the above solutions are costly and time-consuming to produce anthropomorphic images: 3D modeling technology requires the intervention of professional artists and cannot generate images in real time for a large number of randomly appearing objects in the real world, thus failing to meet the needs of rapid interaction.

[0005] It is evident that improving the efficiency of generating anthropomorphic images of objects is a technical issue worthy of attention. Summary of the Invention

[0006] To overcome the problems existing in related technologies, this disclosure provides a method, apparatus, electronic device and storage medium for generating anthropomorphic images.

[0007] According to a first aspect of the present disclosure, a method for generating anthropomorphic images is provided, the method comprising: Obtain the target image containing the target object; Based on the target image, determine the characteristic features and geometric information of the target object; Based on geometric information, the facial features generation region of the target object is determined from the target image; Based on personality traits, generate facial feature line drawings of the target object; Based on the facial feature line drawing, texture features and material features of the facial feature generation area, an anthropomorphic image of the target object is generated.

[0008] In some alternative implementations, acquiring a target image containing the target object includes: Target detection is performed on an image containing a target object to obtain the mask regions of multiple objects in the image; For each of the multiple objects, the saliency judgment information of the object in the image is determined based on the mask region of the object. The saliency judgment information includes at least one of the following: area ratio, centering degree, and sharpness. Based on the saliency information of each object in the image, the mask region of the target object is determined from the mask regions of multiple objects, and a target image containing the target object is obtained.

[0009] In some alternative implementations, based on the target image, the character traits of the target object are determined, including: Extract visual semantic features of the target object from the target image; Based on visual semantic features, the personality traits of a target object are determined by pre-defined mapping information, where the mapping information represents the mapping relationship between visual semantic features and personality traits.

[0010] In some alternative implementations, the geometric information of the target object is determined based on the target image, including: A monocular depth estimation algorithm is used to determine the three-dimensional geometric information of the target object based on the target image; and Based on geometric information, the facial features generation regions of the target object are determined from the target image, including: Based on three-dimensional geometric information, the facial features generation region of the target object is determined from the target image.

[0011] In some optional implementations, the facial feature generation regions of the target object are determined from the target image based on three-dimensional geometric information, including: Based on three-dimensional geometric information, regions that meet preset conditions are determined from the target image; wherein the preset conditions include at least one of the following: the surface smoothness of the target object is less than or equal to a preset threshold; the angle between the orientation of the target object and the optical axis of the target image acquisition device is within a preset angle range. The areas that meet the preset conditions are identified as the facial features generation areas of the target object.

[0012] In some optional implementations, an anthropomorphic image of the target object is generated based on the facial feature line drawing, the texture features and material features of the facial feature generation region, including: Extract a local image patch from the area where facial features are generated; Extract texture and material features from local image patches; Using the facial feature line drawings as constraints, a generative adversarial network is used to redraw the facial feature generation area according to texture and material features, resulting in an anthropomorphic image of the target object.

[0013] In some optional implementations, after generating an anthropomorphic image of the target object based on the facial feature line drawing, texture features of the facial feature generation region, and material features, the method further includes: Get the audio to be played; Extract the pronunciation content and emotional features from the audio to be played; Based on the pronunciation content and emotional features, a video stream is generated in the facial feature generation area of ​​the anthropomorphic image, where the facial features change as the audio to be played changes.

[0014] According to a second aspect of the present disclosure, an apparatus for generating anthropomorphic images is provided, the apparatus comprising: The acquisition unit is configured to acquire a target image containing the target object; The first determining unit is configured to: determine the character features and geometric information of the target object based on the target image; The second determining unit is configured to: determine the facial feature generation region of the target object from the target image based on geometric information; The first generation unit is configured to generate facial feature lines of the target object based on personality traits. The second generation unit is configured to generate an anthropomorphic image of the target object based on the facial feature line drawing, the texture features and material features of the facial feature generation area.

[0015] In some alternative implementations, acquiring a target image containing the target object includes: Target detection is performed on an image containing a target object to obtain the mask regions of multiple objects in the image; For each of the multiple objects, the saliency judgment information of the object in the image is determined based on the mask region of the object. The saliency judgment information includes at least one of the following: area ratio, centering degree, and sharpness. Based on the saliency information of each object in the image, the mask region of the target object is determined from the mask regions of multiple objects, and a target image containing the target object is obtained.

[0016] In some alternative implementations, based on the target image, the character traits of the target object are determined, including: Extract visual semantic features of the target object from the target image; Based on visual semantic features, the personality traits of a target object are determined by pre-defined mapping information, where the mapping information represents the mapping relationship between visual semantic features and personality traits.

[0017] In some alternative implementations, the geometric information of the target object is determined based on the target image, including: A monocular depth estimation algorithm is used to determine the three-dimensional geometric information of the target object based on the target image; and Based on geometric information, the facial features generation regions of the target object are determined from the target image, including: Based on three-dimensional geometric information, the facial features generation region of the target object is determined from the target image.

[0018] In some optional implementations, the facial feature generation regions of the target object are determined from the target image based on three-dimensional geometric information, including: Based on three-dimensional geometric information, regions that meet preset conditions are determined from the target image; wherein the preset conditions include at least one of the following: the surface smoothness of the target object is less than or equal to a preset threshold; the angle between the orientation of the target object and the optical axis of the target image acquisition device is within a preset angle range. The areas that meet the preset conditions are identified as the facial features generation areas of the target object.

[0019] In some optional implementations, an anthropomorphic image of the target object is generated based on the facial feature line drawing, the texture features and material features of the facial feature generation region, including: Extract a local image patch from the area where facial features are generated; Extract texture and material features from local image patches; Using the facial feature line drawings as constraints, a generative adversarial network is used to redraw the facial feature generation area according to texture and material features, resulting in an anthropomorphic image of the target object.

[0020] In some alternative embodiments, the device further includes: The acquisition unit is configured to acquire the audio to be played. The extraction unit is configured to extract the pronunciation content and emotional features from the audio to be played. The third generation unit is configured to generate a video stream in which facial features change with the audio to be played, based on the pronunciation content and emotional characteristics, within the facial feature generation area of ​​the anthropomorphic image.

[0021] According to a third aspect of the present disclosure, an electronic device is provided, comprising: Memory, used to store computer programs; A processor is configured to execute a computer program stored in the aforementioned memory, and when the aforementioned computer program is executed, to implement the method of any embodiment of the method for generating anthropomorphic images according to the first aspect of this disclosure.

[0022] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method of any embodiment of the method for generating anthropomorphic images as described in the first aspect above.

[0023] According to a fifth aspect of the present disclosure, a computer program product is provided, including computer-readable code, which, when executed by a processor, implements the method of any embodiment of the method for generating anthropomorphic images of the first aspect described above.

[0024] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: In this embodiment, by acquiring a target image containing the target object and determining the target object's personality traits and geometric information based on the target image, the anthropomorphic image can conform to both the object's geometric characteristics and possess a personalized personality. By determining the facial feature generation area of ​​the target object based on geometric information, a more suitable position for generating facial features can be selected on the object's surface, ensuring that the generated facial features fit the object's surface more naturally and conform to perspective. By generating a line drawing of the target object's facial features based on personality traits, and generating an anthropomorphic image of the target object based on the line drawing of facial features, the texture features of the facial feature generation area, and the material features, the generated facial features can better integrate with the material and lighting conditions of the object's surface, eliminating the floating and incongruity of traditional AR sticker solutions, thereby improving visual realism. Furthermore, the above method can be used to generate anthropomorphic images of arbitrary, massive, and randomly appearing objects, thereby improving the generation efficiency of anthropomorphic images of arbitrary objects.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this disclosure, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0027] Figure 1 This is a flowchart illustrating a method for generating an anthropomorphic image according to an exemplary embodiment of the present disclosure.

[0028] Figure 2 This is a flowchart illustrating another method for generating anthropomorphic images according to an exemplary embodiment of this disclosure.

[0029] Figure 3 This is a flowchart illustrating yet another method for generating anthropomorphic images according to an exemplary embodiment of the present disclosure.

[0030] Figure 4This is a block diagram of an apparatus for generating anthropomorphic images according to an exemplary embodiment of the present disclosure.

[0031] Figure 5 This disclosure illustrates a block diagram for an electronic device according to an exemplary embodiment. Detailed Implementation

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0033] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0034] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0035] The embodiments of this disclosure will now be described in detail.

[0036] Figure 1 This is a flowchart illustrating a method for generating anthropomorphic images according to an exemplary embodiment. This method can be applied to one or more electronic devices such as smartphones, laptops, desktop computers, portable computers, and servers. Furthermore, the execution entity of this method can be hardware or software. When the execution entity is hardware, it can be one or more of the aforementioned electronic devices. For example, a single electronic device can execute this method, or multiple electronic devices can cooperate with each other to execute this method. When the execution entity is software, this method can be implemented as multiple software programs or software modules, or as a single software program or software module. No specific limitations are made herein.

[0037] like Figure 1 As shown, the method specifically includes: Step 101: Obtain the target image containing the target object.

[0038] In this embodiment, the target object can be any object, or it can be one object selected from multiple objects. In some alternative implementations, the target object can be a real-world object that needs to be anthropomorphized. As examples, the target object could be a trash can, a mechanical watch, a rusty hammer, a pink mug, etc.

[0039] The target image can be any image containing the target object. In some alternative implementations, the target image can be a 2D (two-dimensional) image or video frame containing the target object, or it can be an image region corresponding to the target detection bounding box containing the target object obtained through object detection.

[0040] Step 102: Based on the target image, determine the characteristics and geometric information of the target object.

[0041] In this embodiment, personality traits can represent the personality of the target object, such as irritability or cheerfulness. As an example, for a glass, due to its fragile nature, its personality trait can represent sensitivity; the personality of a rose can be outwardly proud but inwardly fragile.

[0042] In some alternative implementations, the target image can be input into a multimodal large language model (MLLM), which analyzes the target object to obtain a personified personality description, i.e., the personality traits of the target object.

[0043] In some alternative implementations, the target image can be input into a convolutional neural network (CNN) to determine the target object's character traits. This CNN can employ machine learning algorithms, trained on training samples containing sample images and character traits. The character traits corresponding to the sample images can be manually or automatically labeled.

[0044] In some optional implementations of this embodiment, the characteristic features of the target object can also be determined based on the target image in the following manner: The first step is to extract the visual semantic features of the target object from the target image.

[0045] Visual semantic features can represent the visual characteristics of a target object, such as color, texture (e.g., wood, metal, plush), age, shape (e.g., roundness, sharpness), and purpose.

[0046] In some alternative implementations, visual semantic features of the target object can be extracted from the target image using a multimodal large model or a convolutional neural network.

[0047] The second step is to determine the personality traits of the target object based on visual semantic features using pre-defined mapping information.

[0048] The mapping information represents the mapping relationship between visual semantic features and personality traits. Examples of mapping information could include: visual semantic feature “rusty hammer” -> personality traits “tough guy, weathered”; visual semantic feature “pink mug” -> personality traits “lively, young”.

[0049] It is understandable that by extracting the visual semantic features of the target object from the target image, the visual semantics of the object can be obtained; by using pre-set mapping information, the personality characteristics of the target object can be determined based on the visual semantic features, and different personality characteristics can be given to different visual semantics, so that the facial features of the anthropomorphic image can match the visual semantics of the target object, thereby enhancing the diversity and adaptability of the anthropomorphic image.

[0050] Geometric information can represent the geometric shape, geometric structure, etc. of a target object. As an example, the geometric information mentioned above can include at least one of the following: three-dimensional geometric information and two-dimensional geometric information.

[0051] In some alternative implementations, the three-dimensional structural information of the target object can be determined by the surface curvature, lighting conditions, and material texture of the target object in the target image, and then the three-dimensional structural information can be used as the geometric information of the target object.

[0052] In some optional implementations of this embodiment, the geometric information of the target object can be determined based on the target image in the following way: using a monocular depth estimation algorithm to determine the three-dimensional geometric information of the target object based on the target image.

[0053] Monocular depth estimation algorithms can be used to estimate scene depth from 2D images. As an example, a monocular depth estimation algorithm could be Depth Anything.

[0054] Three-dimensional geometric information can be the 3D structural information of the target object, such as surface depth and curvature.

[0055] Based on this, the facial features generation region of the target object can be determined from the target image based on geometric information in the following way: Based on three-dimensional geometric information, the facial features generation region of the target object can be determined from the target image.

[0056] In some optional implementations, a normal map of the target object's surface can be generated based on 3D geometric information; the target image is traversed using a sliding window approach, and the standard deviation of the direction of the normal vectors of all pixels within each window is calculated; continuous regions with a normal direction standard deviation less than a preset threshold are marked as candidate regions; from the candidate regions, the region with the largest area that meets the minimum size constraint (e.g., a size greater than a preset size) is selected and determined as the region for generating facial features of the target object. Thus, the standard deviation of the normal direction reflects the uniformity of the local surface orientation. The smaller the standard deviation, the more consistent the surface orientation of the region, effectively avoiding abrupt changes in curvature, sharp angles, or structural transitions (such as the junction of a bottle neck and body), ensuring that the generated facial features have controllable deformation and continuous edges under perspective projection.

[0057] In some optional implementations, pixel-level depth gradient magnitudes can be calculated based on the depth map in the 3D geometric information. Connected regions with depth gradient magnitudes consistently below a preset smoothing threshold are selected as initial candidates. Morphological closing operations are performed on the initial candidate regions to eliminate minor breaks caused by noise. The Euler number of the processed regions is calculated, and regions with a topological structure of simply connected domains (no holes, no breaks) and an area within a preset range (e.g., an area greater than a preset area) are selected as the facial feature generation regions of the target object. Thus, since the depth gradient magnitude characterizes the drastic change in surface depth, the Euler number in topology can quantify the structural integrity of the region. This allows regions with occlusion gaps, surface holes, or broken structures (such as the surface of a mesh-like object) to be excluded, ensuring that the facial feature generation regions possess geometric continuity and structural closure, providing a stable geometric base for subsequent redrawing.

[0058] It is understandable that by employing a monocular depth estimation algorithm, the three-dimensional geometric information of the target object can be determined based on the target image, allowing the acquisition of the spatial structure information of the object's surface without 3D reconstruction. By determining the facial feature generation area of ​​the target object from the target image based on the three-dimensional geometric information, it is possible to ensure that the selected area is suitable for generating facial features, so that the generated facial features can conform to the actual shape and perspective of the object. This changes the simple layer overlay mode of existing AR technology, and can reduce or even eliminate the floating and fake sticker feeling of facial features in anthropomorphic images, enabling the generated facial features to be better integrated with the target object.

[0059] In some application scenarios of the above-mentioned optional implementation methods, the facial features generation region of the target object can be determined from the target image based on three-dimensional geometric information in the following way: The first step is to determine the regions in the target image that meet the preset conditions based on the three-dimensional geometric information.

[0060] The preset conditions include at least one of the following: the surface smoothness of the target object is less than or equal to a preset threshold; and the angle between the orientation of the target object and the optical axis of the target image acquisition device falls within a preset angle range. For example, the preset angle range could be 170-180 degrees to ensure that the target object is close to a frontal viewing angle.

[0061] Surface smoothness can represent the degree of smoothness of the surface of a target object.

[0062] As an example, surface smoothness can be determined based on the standard deviation of local normal angles. For instance, a local window of a preset size (e.g., 5×5 pixels) is selected centered on the pixel to be evaluated; the unit normal vectors corresponding to all pixels within the window are extracted; the angles between all pairwise normal vectors within the window are calculated; and the standard deviation of all angle values ​​is taken as the surface smoothness of that local window. Here, the smaller the standard deviation, the more consistent the local surface normal directions are, and the higher the geometric continuity.

[0063] As another example, surface smoothness can also be determined based on the mean absolute value of Gaussian curvature. Specifically, a local window of a preset size can be selected centered on the pixel to be evaluated; the Gaussian curvature values ​​corresponding to all pixels within the window are extracted; and the arithmetic mean of the absolute values ​​of Gaussian curvature within the window is calculated as the surface smoothness of that local window. Here, the smaller the mean absolute value of Gaussian curvature, the closer the local surface is to a plane or cylinder (without severe bumps), and the higher the smoothness. This method is sensitive to non-smooth structures such as sharp edges and tiny pits, and is suitable for selecting continuous curved surface areas suitable for facial features.

[0064] The preset threshold can be a critical value for surface smoothness.

[0065] Orientation can be represented by the normal to the surface of the target object.

[0066] The second step is to identify the areas that meet the preset conditions as the areas for generating the facial features of the target object.

[0067] It is understandable that, based on three-dimensional geometric information, determining the region that meets the preset conditions from the target image can quantitatively assess whether the object's surface is suitable for generating facial features. By determining the region that meets the preset conditions as the facial feature generation region of the target object, it can be ensured that facial features are generated on a suitable surface of the object, avoiding the generation of facial features on edges, handles, or complex occlusions, making the facial features flat and in line with perspective, and improving the naturalness and realism of the anthropomorphic image.

[0068] Step 103: Based on geometric information, determine the facial features generation region of the target object from the target image.

[0069] In this embodiment, the facial feature generation region refers to the area in the target image suitable for generating facial features. As an example, the facial feature generation region can be the image area at the top third. As another example, the smooth surface of the object in the target image that faces the viewing angle can also be used as the facial feature generation region to avoid edges, handles, or complex occlusions, ensuring that the facial features are flat and conform to perspective.

[0070] Step 104: Based on personality traits, generate a line drawing of the facial features of the target object.

[0071] In this embodiment, the facial feature line drawing can be a sketch of the facial features determined based on the personality traits of the target object.

[0072] In some alternative implementations, the base library can be called to generate corresponding facial feature lines (such as inverted triangle eyes) based on personality traits (such as anger).

[0073] Step 105: Based on the facial feature line drawing, texture features and material features of the facial feature generation area, generate an anthropomorphic image of the target object.

[0074] In this embodiment, texture features can represent the visual texture attributes of the target object's surface, such as wood grain, matte finish, etc.

[0075] Material characteristics can represent the material properties of a target object, such as hardness and reflectivity.

[0076] Anthropomorphic images can represent images of target objects that have facial features. As an example, an anthropomorphic image could be: an image that retains the original texture of the target object but naturally grows facial features.

[0077] Here, this solution does not require the use of text or other methods to describe texture and material. Instead, it directly extracts a local image patch of the object itself as a visual prompt to determine the texture and material characteristics of the target object.

[0078] In some alternative implementations, texture and material features can be fused into facial feature lines to generate an anthropomorphic image of the target object.

[0079] In some alternative implementations, the facial features line drawings can be first fused into the target image to obtain a fused image; then the texture features and material features can be fused into the fused image to generate an anthropomorphic image of the target object.

[0080] In some optional implementations of this embodiment, the following method can be used to generate an anthropomorphic image of the target object based on the facial feature line drawing, the texture features and material features of the facial feature generation region: The first step is to extract local image blocks from the areas where facial features are generated.

[0081] Among them, the local image patch can be a local part of the image in the facial feature generation region.

[0082] The second step is to extract the texture and material features of local image patches.

[0083] In some optional implementations, local image patches can be used as visual cues. These patches fully preserve the original pixel information of the target object within the facial feature generation region, including surface microstructure and optical reflection properties. Specifically, the local image patch can be input into a pre-trained visual encoder (e.g., ViT-B / 16), which generates a high-dimensional (e.g., 512-dimensional) visual feature vector through forward computation. This vector encodes texture features (e.g., the periodic direction of wood grain, the mesh distribution of woven fabrics, and the random texture of scratches) and material features (e.g., the distribution of specular highlights on metal surfaces, the diffuse reflection characteristics of plush materials, and the specular reflection intensity of ceramics). The resulting visual feature vector is then input into the lightweight adapter layer built into the IP-Adapter. After linear projection and normalization, it is converted into a conditional embedding vector compatible with the cross-attention mechanism of generative models (e.g., Stable Diffusion U-Net). This conditional embedding vector serves as the key and value for cross-attention, continuously guiding the generative model to focus on the original visual attributes of the local image patch during subsequent redrawing.

[0084] The third step involves using the facial feature line drawing as a constraint, and then redrawing the facial feature generation area according to texture and material features using a generative adversarial network to obtain an anthropomorphic image of the target object.

[0085] Generative Adversarial Networks (GANs) can generate anthropomorphic images of objects based on facial features, texture features, and material features. For example, a GAN can be a Diffusion Model or a Generative Adversarial Network (GAN).

[0086] In some alternative implementations, the facial feature line drawing and the content of the target image within the facial feature generation region (after masking and cropping) can be concatenated along the channel dimension and used as the backbone input of the generator in the generative adversarial network, ensuring that the facial feature contours are strictly aligned with the line drawing. The extracted texture and material features are converted into conditional vectors through a lightweight projection layer; adaptive instance normalization is then injected into the intermediate layer of the decoder in the generator: the generated features are dynamically renormalized using the channel statistics (mean μ, variance σ) of the material features, ensuring that the reflection attributes of the output content match local image patches; texture features are fused with the corresponding layer of the encoder through skip connections, enhancing the ability to reproduce microscopic details (such as preserving wood grain direction and woven mesh). The generator can output the pixel content within the facial feature generation region, which is then weighted and fused to the original target image through soft masking to avoid abrupt boundary changes.

[0087] It is understandable that by extracting local image patches from the facial feature generation area, the original visual information of that area can be obtained. By extracting the texture and material features of the local image patches and using the facial feature line drawing as a constraint, the generative adversarial network can redraw the facial features in the facial feature generation area according to the texture and material features. This allows the material features of the object itself to be used directly to generate facial features, thus enabling the generated facial features to blend better with the object's material.

[0088] In some optional implementations of this embodiment, after generating an anthropomorphic image of the target object based on the facial feature line drawing, texture features, and material features of the facial feature generation region, the aforementioned execution entity may also perform the following steps: The first step is to obtain the audio to be played.

[0089] The audio to be played can be any audio file. Here, the audio to be played can be used to drive changes in the facial expressions of the anthropomorphic character. In addition, the audio to be played can be obtained directly through an audio acquisition device, or it can be obtained through text-to-speech.

[0090] The second step is to extract the pronunciation content and emotional features from the audio to be played.

[0091] The audio content can be the spoken content in the played audio.

[0092] Emotional features can represent the emotions in the audio to be played.

[0093] In some alternative implementations, methods such as Wav2Vec can be used to extract the pronunciation content and emotional features from the audio to be played.

[0094] The third step involves generating a video stream in which facial features change according to the audio to be played, based on the pronunciation content and emotional characteristics, within the facial feature generation area of ​​the anthropomorphic image.

[0095] In some alternative implementations, TPS (Thin-Plate Spline) or Dense Motion Field algorithms can be used to nonlinearly deform only the facial feature generation region in the target image, driving the mouth opening and closing and eyebrow movement, and weighting the personality (for example, if the personality is excited, the movement amplitude parameter is amplified; if the personality is lazy, the reaction delay is increased and the movement amplitude is reduced), thereby generating a video stream in which the facial features change with the audio to be played.

[0096] It is understandable that by acquiring the audio to be played and extracting the pronunciation content and emotional features from the audio, information that drives the facial expressions of the anthropomorphic character can be obtained. By generating a video stream in the facial feature generation area of ​​the anthropomorphic character image based on the pronunciation content and emotional features, the facial features change with the audio to be played. This allows for more natural driving of the target object, enabling the anthropomorphic character to produce facial expressions that are synchronized with the speech and consistent with the emotional state. Compared with the FaceReenactment technology, this reduces the dependence on the standard face topology and avoids image tearing or distortion caused by facial key point driving.

[0097] It should be noted that, where there is no conflict, the technical features described in different alternative implementations can be included in the same embodiment. For the sake of brevity, they will not be elaborated here.

[0098] Based on the embodiments of this disclosure, by acquiring a target image containing the target object and determining the target object's personality traits and geometric information based on the target image, the anthropomorphic image can conform to both the geometric characteristics of the object itself and possess a personalized personality. By determining the facial feature generation area of ​​the target object based on the geometric information, a more suitable position for generating facial features can be selected on the object's surface, ensuring that the generated facial features fit the object's surface more naturally and conform to perspective. By generating the facial feature line drawing of the target object based on personality traits, and generating an anthropomorphic image of the target object based on the facial feature line drawing, the texture features and material features of the facial feature generation area, the generated facial features can be better integrated with the material and lighting conditions of the object's surface, eliminating the floating and incongruity of traditional AR sticker solutions, thereby improving visual realism. Furthermore, the above method can be used to generate anthropomorphic images of arbitrary, massive, and randomly appearing objects, thereby improving the generation efficiency of anthropomorphic images of arbitrary objects.

[0099] Figure 2 This is a flowchart illustrating another method for generating anthropomorphic images according to an exemplary embodiment of this disclosure. Figure 2 As shown, the method specifically includes: Step 201: Perform target detection on the image containing the target object to obtain the mask regions of multiple objects in the image.

[0100] In this embodiment, the aforementioned image can be an image or video frame containing the target object. Here, the image can be obtained by capturing the target object in a real scene.

[0101] The mask region can be an object contour mask obtained through a segmentation algorithm.

[0102] In some alternative implementations, image detection algorithms such as Grounding DINO or YOLO-World can be used in combination with segmentation algorithms (such as SAM-Segment Anything Model) to perform object detection and instance segmentation on images containing target objects, thereby obtaining mask regions of multiple objects in the image.

[0103] Step 202: For the mask region of each of the multiple objects, determine the saliency judgment information of the object in the image based on the mask region of the object. The saliency judgment information includes at least one of the following: area ratio, centering degree and sharpness.

[0104] In this embodiment, the area ratio can represent the ratio of the area of ​​the image region where the object is located to the area of ​​the aforementioned image.

[0105] Centering degree represents the degree to which an object is centered in the image above. As an example, the centering degree d can be calculated using the following formula: Wherein, represents the centroid coordinates of the object; represents the center coordinates of the image, W represents the width of the image, and H represents the height of the image; or, represents the center coordinates of the target region, W represents the width of the target region, and H represents the height of the target region, where the target region can be the smallest region in the image that contains all detected objects.

[0106] Sharpness can represent how sharp an object is in the image above. As an example, sharpness can be determined by calculating the Sobel gradient in the horizontal and vertical directions for each object's mask region.

[0107] Step 203: Based on the saliency judgment information of each object in the image, determine the mask region of the target object from the mask regions of multiple objects to obtain the target image containing the target object.

[0108] In this embodiment, for each object, its area ratio, centering degree, and sharpness in the image can be weighted and summed. Based on the calculation results, the mask region of the target object can be determined from the mask regions of multiple objects to obtain the target image containing the target object.

[0109] Step 204: Based on the target image, determine the characteristics and geometric information of the target object.

[0110] In this embodiment, step 204 and Figure 1 Step 102 in the corresponding embodiment is basically the same, and will not be repeated here.

[0111] Step 205: Based on geometric information, determine the facial features generation region of the target object from the target image.

[0112] In this embodiment, step 205 and Figure 1 Step 103 in the corresponding embodiment is basically the same, and will not be repeated here.

[0113] Step 206: Based on personality traits, generate a line drawing of the facial features of the target object.

[0114] In this embodiment, step 206 and Figure 1 Step 104 in the corresponding embodiment is basically the same, and will not be repeated here.

[0115] Step 207: Based on the facial feature line drawing, texture features and material features of the facial feature generation area, generate an anthropomorphic image of the target object.

[0116] In this embodiment, step 207 and Figure 1 Step 105 in the corresponding embodiment is basically the same, and will not be repeated here.

[0117] It should be noted that, in addition to the contents described above, this embodiment may also include... Figure 1 The corresponding technical features described in the corresponding embodiments, thereby achieving Figure 1 For details on the technical effects of the anthropomorphic image generation method shown, please refer to [link / reference needed]. Figure 1 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0118] Based on the embodiments of this disclosure, by performing target detection on an image containing a target object, mask regions of multiple objects in the image can be obtained, which can accurately identify each independent object and its boundary in the scene; by determining the salience judgment information of each object in the image based on the mask region of the object, the priority of each object as an interactive subject can be quantitatively evaluated; by determining the mask region of the target object from the mask regions of multiple objects based on the salience judgment information of each object in the image, the most suitable object as an anthropomorphic object can be automatically selected without human intervention, enabling the system to generate a large number of randomly appearing objects in the real world in real time, meeting the needs of rapid interaction of embodied intelligent agents.

[0119] The following describes the embodiments of this disclosure by way of example. However, it should be noted that the following content is only used to understand the technical solutions of the embodiments of this disclosure and does not constitute a limitation on the protection scope of the embodiments of this disclosure.

[0120] In the fields of computer graphics, augmented reality, and digital entertainment, technologies for anthropomorphizing objects mainly fall into two categories: 1. 3D modeling and animation technology: Typically used in animated film production, this technology requires the complete digital reconstruction of real-world objects using 3D modeling software, redesigning rigging and skinning, and achieving anthropomorphic animation through manual key-frame or motion capture technology.

[0121] 2. Simple AR sticker technology: Using simple image recognition technology, preset cartoon stickers, such as common eye and mouth materials, are mechanically superimposed on the two-dimensional coordinates of an object's surface.

[0122] The above solution has the following problems: 1. High production cost and long cycle: 3D modeling technology requires the intervention of professional artists and cannot generate in real time for a large number of randomly appearing objects in the real world, thus failing to meet the needs of embodied intelligent agents for rapid interaction.

[0123] 2. Poor visual integration and strong sense of incongruity: Existing AR sticker technology usually only involves simple layer overlay, failing to perceive the surface curvature, lighting conditions, and material texture of objects. For example, sticking a bright, plastic-textured cartoon eye on a wooden chair looks extremely unnatural and lacks a sense of "nativeness".

[0124] 3. Lack of semantically personalized expression: Related technologies typically assign facial features randomly or in a fixed manner, failing to automatically deduce an object's "personality" based on its physical attributes (such as shape and purpose). For example, a trash can and a sophisticated mechanical watch should have completely different "eyes" and "micro-expressions," and related technologies cannot achieve this kind of identity-based semantic consistency.

[0125] 4. Single driving mechanism: Most related technologies are driven by human faces and lack general driving algorithms for non-human face topological objects. This results in stiff deformation effects when anthropomorphizing non-human face objects (such as cup handles as hands and bottle caps as heads).

[0126] In view of this, this solution proposes a method for anthropomorphizing all things based on multimodal large models and generative AI (Artificial Intelligence). This method does not perform 3D reconstruction of objects, but directly endows objects with "personality" through semantic understanding in 2D image / video streams, generating anthropomorphic facial features that are perfectly integrated with the object's material and lighting, and uses audio or text to drive these organs to perform movements that conform to physical laws.

[0127] Specifically, see Figure 3This solution includes the following steps: Step 301: Multi-target detection and salient object extraction.

[0128] Input: The original image or video frame of the real scene, i.e., the above-mentioned image.

[0129] Processing procedure: 1. Object detection: Using object detection algorithms (such as Grounding DINO or YOLO-World) combined with segmentation algorithms (such as SAM - Segment Anything Model) to identify all independent objects in the scene.

[0130] 2. Prominence determination: Calculate the area ratio, centering and clarity of the object in the image to filter out the main target of the current interaction, i.e. the target object.

[0131] Output: The mask region of the target object (i.e., the mask region mentioned above) and the object's category label.

[0132] Step 302: Object profiling and personality extraction based on multimodal large model.

[0133] Objective: Not just to identify "what this is", but to perform "facial analysis" of objects through multimodal large models.

[0134] Input: A cropped image of the target object, i.e., the target image.

[0135] Processing procedure: 1. Visual semantic analysis: Analyze the color, texture (such as wood, metal, plush), age, and shape characteristics (such as roundness, sharpness) of objects.

[0136] 2. Personality Mapping Reasoning: Based on a pre-set "object-personality mapping knowledge base", the personality of an object is inferred through the Prompt process.

[0137] Example: Rusty hammer -> Personality: Tough guy, weathered -> Matching facial features: Rough lines, one eye, scars.

[0138] Example: Pink mug -> Personality: Lively, youthful -> Matching facial features: Big eyes, Japanese anime style.

[0139] Output: Problem surface texture features, object characterization, and suggested facial feature style parameters.

[0140] Step 303: Material-adaptive anthropomorphic facial features generation and fusion.

[0141] Objective: To achieve a perfect fusion of facial features with object materials and lighting, eliminate the floating effect of "AR stickers," and achieve a naturally growing visual effect.

[0142] Core technologies: Stable Diffusion + ControlNet (structural control) + IP-Adapter (material feature transfer).

[0143] Algorithm flow: 1. Determining facial features based on geometric perception: Using a monocular depth estimation algorithm (such as Depth Anything) to identify the 3D structure of the object's surface (i.e., the aforementioned three-dimensional geometric information). The system automatically selects smooth areas on the object's surface that are directly facing the viewpoint as the areas for generating facial features, avoiding edges, handles, or complex obstructions to ensure that the facial features are flat and conform to perspective.

[0144] 2. Determine facial contours based on personality constraints: Based on the determined personality keywords (such as "anger"), the base library is used to generate corresponding facial feature line sketches (such as inverted triangle eyes). The ControlNet-Canny model is used as a strong constraint to lock the edge structure of the facial features in the generated image and prevent deformation.

[0145] 3. Texture Transfer-Based Material Fusion: Instead of using textual descriptions of materials, it directly extracts local image patches of the object itself as visual cues. Utilizing IP-Adapter technology, it extracts the object's original texture features (such as wood grain and metallic sheen) and drives the generated model to perform local inpainting within the facial contours. This automatically "fills" the facial features with the object's original material and automatically infers the light source direction based on surrounding pixels, generating correct reflections and shadows.

[0146] Output: A static, high-fidelity image that retains the original texture of the object but naturally grows facial features, i.e., an anthropomorphic image of the target object mentioned above.

[0147] Step 304: Audio / text-driven facial expression transfer and micro-motion generation.

[0148] Input: Audio to be played (or text-to-speech TTS), image synthesized in step 303, and object personality description output in step 302.

[0149] Algorithm flow: 1. Audio feature extraction: Use Wav2Vec and other software to extract pronunciation content and emotional features from audio.

[0150] 2. General Non-Facial Expression Driven Network: Unlike traditional face-driven networks, this invention trains a Motion Transfer network targeting a "generalized topology". Utilizing TPS (Thin-Plate Spline) or Dense Motion Field algorithms, it performs non-linear deformation only on the "facial feature generation region" in the image, driving mouth opening and closing and eyebrow movement.

[0151] 3. Personality-weighted: If the personality type is "Excited", increase the amplitude parameter of the movement. If the personality type is "lazy", increase reaction delay and reduce the range of motion.

[0152] Output: A video stream in which facial features move in sync with the speech.

[0153] Furthermore, in addition to using Diffusion Model to generate high-quality facial features, the above solutions can also employ GAN (Generative Adversarial Network) for more real-time extraction of texture features in the regions where stylized facial features are generated, making them suitable for edge devices. Moreover, for scenarios with limited computing power, traditional graphics-driven methods based on Blendshape can be used. This involves pre-generating a set of 2D deformation bases for general facial features and achieving animation through weighted blending, rather than pixel-level generation.

[0154] It should be noted that, in addition to the contents described above, this embodiment may also include the technical features described in the above embodiments, thereby achieving the technical effect of the above-described method for generating anthropomorphic images. For details, please refer to the above description. For the sake of brevity, it will not be elaborated here.

[0155] Based on the embodiments disclosed herein, the cumbersome industrial processes of 3D scanning, modeling, retopology, and skeletal binding of objects in traditional technologies are eliminated. A generation scheme based on 2D image streams is adopted, utilizing deep learning models to directly construct images in two-dimensional image space, thus lowering the threshold for content production of embodied intelligent interactions. Whether it's new equipment on a production line or old furniture in a user's home, no pre-modeling is required; images can be used immediately, improving processing efficiency from "days / hours" to "minutes / seconds." Furthermore, this solution changes the simple layer overlay mode of existing AR technologies. It introduces technologies such as determining facial feature positions based on geometric perception, determining facial feature contours based on personality constraints, and IP-Adapter. The algorithm can actively extract the texture features of the object itself (such as wood grain and metal rust) and redraw them by combining the surface curvature perceived by normal maps, thereby eliminating the floating effect and the feeling of fake stickers. The generated facial features appear to grow naturally from the object's surface, possessing a material feel, diffuse reflectivity, and light projection direction completely consistent with the original object, achieving photo-level visual realism. Moreover, this solution solves the semantic fragmentation problem caused by random allocation or fixed templates in related technologies. Using a Multimodal Large Model (MLLM) as its brain, this approach first understands the physical attributes (e.g., sharp / rounded) and social attributes (e.g., use / newness / oldness) of objects. Then, through reasoning, it maps logically consistent personality traits, thus endowing all things with unique personalities. For example, it can automatically generate weathered, rugged facial features for a rusty hammer and lively, cute features for a pink water glass. This semantically consistent anthropomorphism greatly enhances the immersiveness and credibility of user interaction with the intelligent agent. This solution breaks through the dependence of traditional Face Reenactment technology on standard facial topology. Employing a generalized topology-driven network, combined with TPS or dense motion field techniques, it directly drives pixel-level deformation of facial feature regions of arbitrary shapes, enabling natural manipulation of non-standard geometric shapes. Even on non-facial structures such as handles, bottle caps, and irregular stones, it can present actions such as opening and closing the mouth and blinking that conform to the laws of physical movement, avoiding image tearing or distortion caused by rigidly applying facial key points. Furthermore, this solution, through object profiling and personality extraction, ensures logical consistency between the object's external appearance and its internal attributes. For example, adding an anxious expression to an alarm clock or a lazy expression to a sofa greatly enhances the immersive and engaging experience of embodied intelligent interaction. The driving algorithm, based on TPS or dense motion fields, does not rely on standard facial landmarks and can adapt to various irregular surfaces such as chair backs, cups, and spheres, demonstrating strong generalization capabilities.

[0156] Figure 4 This is a block diagram illustrating an apparatus for generating anthropomorphic images according to an exemplary embodiment of the present disclosure. The apparatus specifically includes: The acquisition unit 401 is configured to: acquire a target image containing the target object; The first determining unit 402 is configured to: determine the character features and geometric information of the target object based on the target image; The second determining unit 403 is configured to: determine the facial feature generation region of the target object from the target image based on geometric information; The first generation unit 404 is configured to: generate facial feature line drawings of the target object based on personality traits; The second generation unit 405 is configured to generate an anthropomorphic image of the target object based on the facial feature line drawing, the texture features and material features of the facial feature generation area.

[0157] In some alternative implementations, acquiring a target image containing the target object includes: Target detection is performed on an image containing a target object to obtain the mask regions of multiple objects in the image; For each of the multiple objects, the saliency judgment information of the object in the image is determined based on the mask region of the object. The saliency judgment information includes at least one of the following: area ratio, centering degree, and sharpness. Based on the saliency information of each object in the image, the mask region of the target object is determined from the mask regions of multiple objects, and a target image containing the target object is obtained.

[0158] In some alternative implementations, based on the target image, the character traits of the target object are determined, including: Extract visual semantic features of the target object from the target image; Based on visual semantic features, the personality traits of a target object are determined by pre-defined mapping information, where the mapping information represents the mapping relationship between visual semantic features and personality traits.

[0159] In some alternative implementations, the geometric information of the target object is determined based on the target image, including: A monocular depth estimation algorithm is used to determine the three-dimensional geometric information of the target object based on the target image; and Based on geometric information, the facial features generation regions of the target object are determined from the target image, including: Based on three-dimensional geometric information, the facial features generation region of the target object is determined from the target image.

[0160] In some optional implementations, the facial feature generation regions of the target object are determined from the target image based on three-dimensional geometric information, including: Based on three-dimensional geometric information, regions that meet preset conditions are determined from the target image; wherein the preset conditions include at least one of the following: the surface smoothness of the target object is less than or equal to a preset threshold; the angle between the orientation of the target object and the optical axis of the target image acquisition device is within a preset angle range. The areas that meet the preset conditions are identified as the facial features generation areas of the target object.

[0161] In some optional implementations, an anthropomorphic image of the target object is generated based on the facial feature line drawing, the texture features and material features of the facial feature generation region, including: Extract a local image patch from the area where facial features are generated; Extract texture and material features from local image patches; Using the facial feature line drawings as constraints, a generative adversarial network is used to redraw the facial feature generation area according to texture and material features, resulting in an anthropomorphic image of the target object.

[0162] In some alternative embodiments, the device further includes: The acquisition unit (not shown in the figure) is configured to acquire the audio to be played; The extraction unit (not shown in the figure) is configured to extract the pronunciation content and emotional features from the audio to be played. The third generation unit (not shown in the figure) is configured to generate a video stream in which facial features change with the audio to be played, based on the pronunciation content and emotional characteristics, in the facial feature generation area of ​​the anthropomorphic image.

[0163] The anthropomorphic image generation device provided in this embodiment can execute the corresponding steps of the anthropomorphic image generation methods described above, thereby achieving the technical effects of the anthropomorphic image generation methods described above. The anthropomorphic image generation device and the anthropomorphic image generation methods can refer to and cite each other in terms of specific implementation and technical effects. For the sake of brevity, they will not be elaborated here.

[0164] Figure 5 This disclosure illustrates a block diagram for an electronic device according to an exemplary embodiment. Figure 5 The illustrated electronic device 500 includes at least one processor 501, a memory 502, at least one network interface 504, and other user interfaces 503. The various components in the electronic device 500 are coupled together via a bus system 505. It is understood that the bus system 505 is used to implement communication between these components. In addition to a data bus, the bus system 505 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 5 The general designated all buses as Bus System 505.

[0165] The user interface 503 may include a display, keyboard, or clicking device (e.g., mouse, trackball, touchpad, or touchscreen).

[0166] It is understood that the memory 502 in this embodiment of the present disclosure may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 502 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0167] In some implementations, memory 502 stores elements, executable units or data structures, or subsets thereof, or extended sets thereof: operating system 5021 and application program 5022.

[0168] The operating system 5021 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 5022 includes various applications, such as a media player and a browser, used to implement various application functions. The program implementing the method of this embodiment can be included in the application program 5022.

[0169] In this embodiment, by calling the program or instructions stored in memory 502, specifically the program or instructions stored in application program 5022, processor 501 executes the method steps provided in each method embodiment, including, for example: Obtain the target image containing the target object; Based on the target image, determine the characteristic features and geometric information of the target object; Based on geometric information, the facial features generation region of the target object is determined from the target image; Based on personality traits, generate facial feature line drawings of the target object; Based on the facial feature line drawing, texture features and material features of the facial feature generation area, an anthropomorphic image of the target object is generated.

[0170] The methods disclosed in the above embodiments of this disclosure can be applied to or implemented by processor 501. Processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware or by instructions in the form of software in processor 501. The processor 501 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software units may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 502. Processor 501 reads the information in memory 502 and, in conjunction with its hardware, completes the steps of the above method.

[0171] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described above, or combinations thereof.

[0172] For software implementation, the techniques described herein can be implemented by units that perform the functions described above. The software code can be stored in memory and executed by a processor. The memory can be implemented within the processor or external to the processor.

[0173] The electronic device provided in this embodiment may be as follows: Figure 5 The electronic device shown can execute all the steps of the above-described methods for generating anthropomorphic images, thereby achieving the technical effects of the above-described methods for generating anthropomorphic images. For details, please refer to the above descriptions. For the sake of brevity, further details are omitted here.

[0174] This disclosure also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; the memory may also include combinations of the above types of memory.

[0175] When one or more programs in the storage medium can be executed by one or more processors to implement the above-described method for generating anthropomorphic images executed on the electronic device side.

[0176] The processor described above is used to execute a program for generating anthropomorphic images stored in memory, to implement the following steps of the anthropomorphic image generation method executed on the electronic device side: Obtain the target image containing the target object; Based on the target image, determine the characteristic features and geometric information of the target object; Based on geometric information, the facial features generation region of the target object is determined from the target image; Based on personality traits, generate facial feature line drawings of the target object; Based on the facial feature line drawing, texture features and material features of the facial feature generation area, an anthropomorphic image of the target object is generated.

[0177] Furthermore, the computer program product provided in this disclosure embodiment may include computer-readable code that, when executed on a device, causes a processor in the device to implement the steps of the method for generating anthropomorphic images that is executed on the electronic device side: Obtain the target image containing the target object; Based on the target image, determine the characteristic features and geometric information of the target object; Based on geometric information, the facial features generation region of the target object is determined from the target image; Based on personality traits, generate facial feature line drawings of the target object; Based on the facial feature line drawing, texture features and material features of the facial feature generation area, an anthropomorphic image of the target object is generated.

[0178] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0179] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0180] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0181] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for generating anthropomorphic images, characterized in that, The method includes: Obtain the target image containing the target object; Based on the target image, determine the characteristic features and geometric information of the target object; Based on the geometric information, the facial features generation region of the target object is determined from the target image; Based on the personality traits, generate a line drawing of the facial features of the target object; Based on the facial feature line drawing, the texture features and material features of the facial feature generation area, an anthropomorphic image of the target object is generated.

2. The method according to claim 1, characterized in that, The step of acquiring a target image containing the target object includes: Target detection is performed on an image containing a target object to obtain mask regions of multiple objects in the image; For the mask region of each of the plurality of objects, based on the mask region of the object, the salience judgment information of the object in the image is determined, wherein the salience judgment information includes at least one of the following: area ratio, centering degree, and sharpness; Based on the saliency judgment information of each object in the image, the mask region of the target object is determined from the mask regions of multiple objects, and a target image containing the target object is obtained.

3. The method according to claim 1, characterized in that, Based on the target image, determine the personality traits of the target object, including: From the target image, extract the visual semantic features of the target object; Based on the visual semantic features, the personality traits of the target object are determined by using pre-defined mapping information, wherein the mapping information represents the mapping relationship between the visual semantic features and the personality traits.

4. The method according to claim 1, characterized in that, Based on the target image, the geometric information of the target object is determined, including: Using a monocular depth estimation algorithm, the three-dimensional geometric information of the target object is determined based on the target image; and The step of determining the facial feature generation region of the target object from the target image based on the geometric information includes: Based on the three-dimensional geometric information, the facial features generation region of the target object is determined from the target image.

5. The method according to claim 4, characterized in that, The step of determining the facial feature generation region of the target object from the target image based on the three-dimensional geometric information includes: Based on the three-dimensional geometric information, a region that meets preset conditions is determined from the target image; wherein, the preset conditions include at least one of the following: the surface smoothness of the target object is less than or equal to a preset threshold; the angle between the orientation of the target object and the optical axis of the target image acquisition device is within a preset angle range; The regions that meet the preset conditions are determined as the facial feature generation regions of the target object.

6. The method according to claim 1, characterized in that, The process of generating an anthropomorphic image of the target object based on the facial feature line drawing, the texture features and material features of the facial feature generation region, includes: Extract a local image patch from the facial feature generation region; Extract the texture and material features of the local image patch; Using the facial feature line drawings as constraints, a generative adversarial network is used to redraw the facial feature generation area according to the texture features and material features to obtain an anthropomorphic image of the target object.

7. The method according to any one of claims 1-6, characterized in that, After generating an anthropomorphic image of the target object based on the facial feature line drawing, the texture features and material features of the facial feature generation region, the method further includes: Get the audio to be played; Extract the pronunciation content and emotional features from the audio to be played; Based on the pronunciation content and the emotional features, a video stream is generated in the facial feature generation area of ​​the anthropomorphic image, where the facial features change with the audio to be played.

8. An apparatus for generating anthropomorphic images, characterized in that, The device includes: The acquisition unit is configured to acquire a target image containing the target object; The first determining unit is configured to: determine the character features and geometric information of the target object based on the target image; The second determining unit is configured to: determine the facial feature generation region of the target object from the target image based on the geometric information; The first generation unit is configured to: generate a line drawing of the facial features of the target object based on the personality traits; The second generation unit is configured to generate an anthropomorphic image of the target object based on the facial feature line drawing, the texture features and material features of the facial feature generation area.

9. An electronic device, comprising: Memory, used to store computer programs; A processor for executing a computer program stored in the memory, wherein when the computer program is executed, it implements the method described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-7.

Images