A virtual-real combined interactive social system and method and storage medium

Abstract

This invention discloses a virtual-real integrated interactive social system and its implementation method. The method includes: receiving a frontal portrait image uploaded by a user; generating a customized 2D cartoon-style image of the user based on a preset image generation model and a fine-tuning model; generating a corresponding 3D cartoon digital human model based on the 2D cartoon-style image using 3D reconstruction technology; associating the 3D cartoon digital human model with a physical interactive card, wherein the physical interactive card contains a near-field communication tag or QR code for storing the unique identification information of the 3D cartoon digital human model; and reading the identification information of the physical interactive card through a terminal device, loading and displaying the corresponding 3D cartoon digital human model, supporting real-time interactive operations between the user and the 3D cartoon digital human model. This invention solves the pain points of traditional 3D digital humans, such as high cost, low user acceptance, and limited interaction, through AI-driven personalized generation, virtual-real integrated interactive design, and efficient technology implementation, providing an innovative solution for consumer-grade virtual social scenarios.

Description

Technical Field

[0001] This invention belongs to the technical field of combining 3D digital humans with artificial intelligence, specifically involving an interactive social system that combines virtual and real elements and its implementation method. Background Technology

[0002] With the development of artificial intelligence technology and the optimization of human portrait model datasets, the technology of creating 3D digital humans from a single photograph has developed rapidly and is beginning to replace the generation and production of 3D digital humans by 3D scanning technology and traditional manual art methods. For example, technologies such as patent CN119693524A - Head Reconstruction Method Based on a Single Photograph and patent CN119762673A - Full-Body Reconstruction Method Based on a Single Photograph use a single photograph to generate a lifelike 3D digital human, that is, a digital clone of the user. In addition, image generation model technology, especially image generation methods based on diffusion models (such as Stable Diffusion), achieves high-quality image synthesis through a stepwise denoising process, making image processing of various styles possible.

[0003] 3D digital human technology based on artificial intelligence or other automated technologies, such as 3D digital humans quickly generated from a single photograph or through scanning a real person, has undergone some development. However, for consumer-grade 3D virtual humans, it is still limited by the "uncanny valley effect"—the technology from image creation to motion and expression driving. In real-world scenarios, we have found that users are not easily accepting a 3D digital human replica of themselves that is not yet perfect. Therefore, the development of consumer-grade applications based on this technology will be difficult to continue. These technical difficulties are expected to be difficult to overcome in the short term. As a result, the application of 3D digital humans is still mainly concentrated on a small number of digital human IPs created for enterprises, without truly leveraging the advantages of rapid customization, low cost, and intelligent access of 3D digital humans.

[0004] Therefore, 3D cartoon digital human technology will be a key development direction for the foreseeable future. For example, patent CN117542098A discloses a method for making a 3D cartoon digital human. The final effect of the cartoon digital human depends not only on the effect of the 3D reconstruction of the human's face, but also mainly on the relationship and production level of the pre-made neutral 3D face model and the pre-made 3D cartoon style template. It is still unclear whether the actual effect can achieve the cute cartoon feel that is very similar to the human. Furthermore, the workload is relatively large when updating different cartoon style templates. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for implementing an interactive social system that combines virtual and real elements, in order to solve the shortcomings of traditional 3D digital humans, such as high cost, low user acceptance, and limited interaction.

[0006] In accordance with the above objectives, a first aspect of the present invention provides a method for implementing a virtual-real integrated interactive social system, the method comprising:

[0007] It receives frontal portrait images uploaded by users and generates customized 2D cartoon-style images for users based on preset image generation models and fine-tuning models.

[0008] Based on the 2D cartoon-style image, a corresponding 3D cartoon digital human model is generated using 3D reconstruction technology;

[0009] The 3D cartoon digital human model is associated with a physical interactive card, which contains a near-field communication tag or QR code to store the unique identification information of the 3D cartoon digital human model.

[0010] The terminal device reads the identification information of the physical interactive card, loads and displays the corresponding 3D cartoon digital human model, and supports real-time interactive operation between the user and the 3D cartoon digital human model.

[0011] Furthermore, the specific steps for generating the user's customized 2D cartoon-style image include:

[0012] Load a base image generation model and at least one lightweight fine-tuning model, the fine-tuning model being used to enhance stylized features and detail representation;

[0013] Extract facial features from the user input image and edge structure information from the preset template image;

[0014] By using a facial recognition-guided model and a style fusion model, combined with text prompts and a control network, 2D cartoon images that conform to the target style are generated.

[0015] Furthermore, the generation of the corresponding 3D cartoon digital human model includes:

[0016] Based on the 2D cartoon-style image, a cartoon-style 3D head model is generated using 3D head reconstruction technology;

[0017] The 3D head model is shaped and transformed with a preset cartoon body standard model. The head and body are then fused together through vertex matching and coordinate mapping to generate a complete 3D cartoon digital human model.

[0018] Furthermore, the shaping transformation includes: separating the head portion of the standard model, adjusting the overlapping area of ​​its vertex positions with the 3D head model, establishing vertex number and coordinate mapping relationship, and migrating the texture map to the merged model based on the mapping relationship.

[0019] Furthermore, the plastic transformation specifically includes:

[0020] A pre-made standard digital human model is selected as the benchmark model. The benchmark model is a 3D model with a cute cartoonish figure, selected from the SMPLX model or CC4 model.

[0021] Adjust the size and position of the head region of the reference model so that it partially overlaps with the head or face region of the 3D head model;

[0022] Traverse the vertices of the head region of the 3D head model and the baseline model, and establish the correspondence between vertex numbers and the relative coordinate mapping relationship within the overlapping region;

[0023] Based on the vertex number correspondence and coordinate mapping relationship, the texture map and topology information of the 3D head model are transferred to the base model to achieve automatic texture map adaptation, and finally generate a fused complete 3D cartoon digital human model.

[0024] Furthermore, establishing the correspondence between vertex numbers within the overlapping area includes: determining the nearest neighbor pairs by calculating the Euclidean distance between the 3D head model and the head vertices of the reference model, and recording their numbers and coordinate offsets.

[0025] Furthermore, the fine-tuning model includes at least one of the following: sticker style conversion model, cartoon feature enhancement model, expression adjustment model, age control model, and cute style optimization model; the interactive operation includes: action animation switching, clothing switching, scene switching, and expression feedback.

[0026] A second aspect of the present invention provides an interactive social system that combines virtual and real elements, the system comprising:

[0027] The image generation module is used to receive user-input images and generate customized 2D cartoon-style images;

[0028] The 3D digital human generation module is used to generate a corresponding 3D cartoon digital human model based on the 2D cartoon-style image;

[0029] The physical interactive card generation module is used to associate the 3D cartoon digital human model with a near-field communication tag or QR code;

[0030] The terminal application module is used to load and display 3D cartoon digital human models by reading physical interactive cards, supporting user interaction.

[0031] Furthermore, the image generation module includes:

[0032] A basic image generation model is used to generate preliminary cartoon-style images based on user input images;

[0033] A fine-tuning model library, containing multiple lightweight fine-tuning models, is used to stylize and refine the details of the initial cartoon-style images;

[0034] The facial feature extraction unit is used to extract facial regions and key features from user input images;

[0035] The style fusion unit is used to combine the edge structure information and facial features of a preset template image, and guide the model to achieve style transfer through facial recognition.

[0036] The control network unit is used to control the contour and pose of the generated image based on the template edge information, so as to ensure the structural consistency between the generated image and the template.

[0037] In a third aspect, the present invention provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method for implementing a virtual-real integrated interactive social system as described in the first aspect.

[0038] Compared with the prior art, the method and system for implementing a virtual-real integrated interactive social system disclosed in the embodiments of the present invention achieve the following technical effects:

[0039] I. The method of this invention realizes an automatic generation process for high-fidelity, high-style reproduction, and controllable structure of cartoon 2D character images, which has good scalability and practical application value.

[0040] Second, the method of the present invention realizes an automatic generation process of cartoon 3D character images with high fidelity, high style reproduction and controllable structure. It breaks the "uncanny valley" effect of conventional real-life 3D digital human image creation and motion driving, and has high practical application value at the present stage.

[0041] Third, this invention automatically generates 2D cartoon-style images by combining user-uploaded real-life images with AI models (such as Stable Diffusion and Lora fine-tuning models), supporting multiple styles (stickers, chibi, and cute designs) to meet users' multi-dimensional personalized needs. Simultaneously, based on the 2D cartoon images, 3D cartoon digital humans are generated using 3D reconstruction technology, achieving full-process customization from appearance to movement.

[0042] Fourth, this invention employs a pre-trained model combined with a lightweight fine-tuning model (such as sticker style transfer and facial expression adjustment), significantly improving generation efficiency and avoiding the high costs of traditional manual modeling. Through vertex matching, coordinate mapping, and UV mapping transfer techniques, it achieves rapid fusion of the 3D head with the standard body model, reducing manual intervention.

[0043] V. This invention binds 3D digital humans to physical cards via NFC tags or QR codes. Users can quickly load the virtual avatar by "tapping" or "scanning," breaking down the boundaries between the physical and digital worlds. It supports real-time interactive operations (such as action switching, clothing changes, and scene linkage), enhancing user immersion and engagement.

[0044] VI. This invention uses standardized 3D models (such as SMPLX and CC4), supports common interfaces such as skeletal rigging and animation transfer, and is compatible with multiple terminal applications (such as WeChat mini programs and standalone apps). Users only need to upload a single front-facing photo and select a template, and the system will automatically generate 2D / 3D images and interactive cards, greatly simplifying the user experience. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating the implementation method of a virtual-real integrated interactive social system in an embodiment of the present invention.

[0046] Figure 2 This is a schematic diagram of the structure of an interactive social system that combines virtual and real elements in an embodiment of the present invention.

[0047] Figure 3 This is a schematic diagram of the structure of an electronic device for interactive social networking that combines virtual and real elements, as described in an embodiment of the present invention. Detailed Implementation

[0048] The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention. Certain terms are used in the specification and claims to refer to specific components. Those skilled in the art will understand that hardware or software manufacturers may use different names to refer to the same component. This specification and claims do not distinguish components based on differences in name, but rather on differences in function. The following descriptions of preferred embodiments of the present invention are intended to illustrate the general principles of the invention and are not intended to limit the scope of the invention. The scope of protection of the present invention shall be determined by the appended claims.

[0049] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0050] This invention provides a method, system, storage medium, and device for creating a virtual-real integrated interactive social system based on the core technologies of generating customized 2D cartoon images and 3D cartoon digital humans. By uploading a user's frontal portrait image to a 2D Q-version image server, a 2D cartoon-style image of the user is generated based on technologies such as a stable diffusion model. Using 3D virtual digital human technology, a stylized 3D digital human is generated based on the style image. A physical image card, created using the user's stylized 2D image and technologies such as NFC near-field communication and QR codes, can be associated with the user's stylized 3D digital human. Users can quickly access the user's customized 3D cartoon-style digital human on their own or anyone else's mobile phone or other device using convenient methods such as tapping or scanning a QR code, and interact and engage in social activities with it.

[0051] Example 1

[0052] Please see Figure 1 Embodiment 1 of the present invention provides a method for implementing an interactive social system that combines virtual and real elements, the method comprising:

[0053] Step S1: Receive the frontal portrait image uploaded by the user, and generate a customized 2D cartoon-style image for the user based on the preset image generation model and the fine-tuning model.

[0054] In step S1, the specific steps for generating the user's customized 2D cartoon-style image include:

[0055] S11. Load the base image generation model and at least one lightweight fine-tuning model, which is used to enhance stylized features and detail representation.

[0056] For example, DreamShaperXL, an open-source graphics model, can be used as the base model for image generation. This model possesses high realism and style transfer capabilities, enabling it to generate high-resolution, color-saturated, and cartoonish character images. To enrich stylistic diversity and enhance image detail, this invention introduces several lightweight LoRa fine-tuning models on top of the base model. These LoRa models include, but are not limited to: sticker style transfer models (such as StickersRedmond), cartoonization feature enhancement models (such as cartoon_style), facial expression adjustment models (smiling), age control models (age), and cuteness style optimization models. Different fine-tuning models have different advantages and can be used in combination.

[0057] S12. Extract facial features from the user input image and edge structure information from the preset template image.

[0058] After detecting the user-input image of a person and the style reference template image, the system extracts the facial region and key features from the input image, and extracts the edge structure information of the template image through Canny edge detection.

[0059] S13. By using a facial recognition-guided model and a style fusion model, combined with text prompts and a control network, a 2D cartoon image that conforms to the target style is generated.

[0060] Specifically, based on extracted facial features and edge information, a facial recognition guidance model (IPAdapter) and a style fusion model (Pulid) are applied to achieve facial recognition guidance and fuse template styles, enabling style transfer and redrawing of the character image while preserving its original appearance. The IPAdapter model, as a crucial sub-module for image cartoon style guidance and structure preservation, is primarily used to achieve accurate mapping between the input character image and the generated result in terms of identity consistency, facial features, and cartoon template style. It provides powerful visual guidance capabilities and an identity preservation mechanism. The Pulid model, on the other hand, undertakes the dual functions of "style fusion" and "facial refinement," and its deep semantic modeling capabilities and structure preservation mechanism are essential components for achieving high-quality customized cartoon character generation.

[0061] The system intelligently enhances the style consistency and thematic relevance of the generated images based on user-inputted prompts that match the template content; simultaneously, it introduces reverse prompts to suppress unwanted image features. The Canny edge map generated from the template image is used as guiding information input to the ControlNet model, aiming to control the contour and pose of the generated image, thereby improving the pose and structure of the generated result's high consistency with the cartoon template. The dpmp_2m sampler performs the image generation task, using iterative denoising with 50 iterations and a noise attenuation factor of 0.85 to generate high-resolution cartoon images. After sampling, an image VAE decoder is used to decode the sampled results into a visual image. Finally, background removal is performed on the resulting image, retaining only the main character area for subsequent image compositing or content editing.

[0062] Step S2: Based on the 2D cartoon-style image, generate a corresponding 3D cartoon digital human model using 3D reconstruction technology.

[0063] The generation of the corresponding 3D cartoon digital human model includes:

[0064] S21. Based on the 2D cartoon-style image, a cartoon-style 3D head model is generated using 3D head reconstruction technology.

[0065] Specifically, the 3D head reconstruction technology in this embodiment can adopt the applicant's prior patent application and publication, "CN119693524A - Head Reconstruction Method Based on a Single Photograph," or other methods based on 3DMM (3D Deformable Face Model, where the face structure is represented by a unified topological 3D mesh) to extract the face from the cartoon-styled 2D image generated in step S1 and perform 3D head reconstruction to obtain the corresponding cartoon-styled 3D head model and texture map. The result of the 3D head reconstruction includes the order of head points and corresponding topological information; that is, any point on any generated 3D head model has a fixed sequence number and corresponding semantic information, for example, the point with sequence number xxxx corresponds to the outer corner of the eye in this model. In step S21, a user-customized cartoon-styled 3D head model and texture map are obtained, but a cartoon-styled body model and processing of the junction between the head and body, such as the neck area, are still lacking.

[0066] S22. The 3D head model is shaped and transformed with a preset cartoon body standard model. The head and body are merged through vertex matching and coordinate mapping to generate a complete 3D cartoon digital human model. In order to make the head model and UV map of the digital human standard model consistent with the 3D head model, a shaped transformation is required. The basic process of the shaped transformation is as follows: by separating the head part of the standard model, adjusting its vertex position and the overlapping area with the 3D head model, establishing vertex sequence and coordinate mapping relationship, and transferring the texture map to the merged model based on the mapping relationship.

[0067] In this embodiment, the plastic transformation specifically includes the following steps:

[0068] S221. Select a pre-made digital human standard model as the reference model. The reference model is a 3D model with a cartoonish body shape, selected from the SMPLX model or CC4 model. Among them, the CC4 model can efficiently adapt to the shaping and transformation requirements of the head model due to its standardized skeletal binding, cross-platform compatibility and preset cartoon body template.

[0069] S222. Adjust the size and position of the head region of the reference model so that it partially overlaps with the head or face region of the 3D head model.

[0070] Assume a cartoon-styled 3D head model A1 has been generated using a cartoon-styled 2D image, and a standard digitized human model B0 with a cute, childlike body has been created. The system detects and compares the position and size of the head region of A1 and B0, then performs displacement and scaling operations on the standard digitized human model B0, and can use methods such as manual wrapping or tool wrapping to obtain a new standard digitized human model B1, such that the head or face portion of B1 coincides with the head or face portion of the 3D head model A1. Preferably, the heads and faces of the two models completely coincide. Taking the position encompassed by the vertices of the 3D head model A1 as a reference, the vertices of the head, neck, and shoulder portions of the standard digitized human model B1 corresponding to the position are selected, and the head model B1H of B1 is separated from B1. The B1 model is divided into a head model B1H and a body model B1B. Then, the head or face portion of B1H coincides with the head or face portion of the 3D head model A1.

[0071] S223. Traverse the vertices of the head region of the 3D head model and the reference model, and establish the correspondence between the vertex numbers and the relative coordinate mapping relationship within the overlapping region.

[0072] Traverse all vertices of the head model B1H of the 3D head model A1 and the standard digital human model B1, and find the correspondence between the nearest points of the overlapping models A1 and B1H, as well as the relative coordinate relationship between the nearest points. The above correspondence between the nearest points and the relative coordinate relationship constitute the model shaping transformation relationship between the 3D head model A1 and the standard digital human model B1. In an optional embodiment, establishing the vertex number correspondence within the overlapping area includes: determining the nearest point pairs by calculating the Euclidean distance between the vertices of the 3D head model and the reference model head, and recording their numbers and coordinate offsets.

[0073] Assume the point set of model A1 is: A1 = {a11, a12, ..., a1} m},in Let $\mathbf{i}$ represent the 3D coordinates of the $i$-th point in model A1, where $i \in \{1, 2, ..., m}$. The point set of model B1H is: $B1H = {b11, b12, ..., b1\mathbf{i}}$. n},in Let $\mathbf{j}$ represent the three-dimensional coordinates of the $j$-th point in model $B1H$, where $j \in \{1,2,…,n}$.

[0074] The query process is calculated using the Euclidean distance metric:

[0075]

[0076] Wherein d(a1) i b1 j () represents point a1 in model A1 i With point b1 in model B1H j The Euclidean distance between them.

[0077] For each point a1 in model A1 i By calculating d(a1) of all B1H points i b1 j This allows us to calculate point a1 in model A1. i The k nearest neighbors in model B1H. We set the distance point a1. i Let's take the nearest k points as an example (3 points) to illustrate the method for calculating relative coordinate coefficients.

[0078] For each vertex a1 in model A1 i Find its three nearest neighbors in model B1H. k =(bk x ,bk y ,bk z For each direction (x, y, z), k∈{1,2,3}, first calculate the minimum and maximum values ​​of the three nearest points as the difference interval. Take the x-direction as an example:

[0079] x1=min(b1 x b2 x b3 x );

[0080] x2=max(b1 x b2 x b3 x );

[0081] Determine vertex a1 i Does it fall within the interval [x1, x2]?

[0082] If a1 i If x ∈ [x1, x2], then linear interpolation is performed, and the interpolation coefficients are:

[0083]

[0084] like Then normalization is performed.

[0085] The normalization process is as follows:

[0086] When performing nearest neighbor search, there are local cases where the density of the point cloud is high or low. Therefore, when calculating coordinate differences, the coordinate differences in each direction are normalized. Specifically, in each direction (x, y, z), if vertex a1 i If the coordinates of a point exceed the minimum or maximum value of its nearest neighbor, then a normalization coefficient is calculated to convert the coordinate difference to the standardized range.

[0087] Assuming we need to normalize the coordinate differences in the x-direction, we first calculate the minimum x1 and maximum x2 of the nearest neighbor points in the x-direction, and then perform normalization:

[0088]

[0089] The same normalization process can be applied to the values ​​of the nearest neighbor points in the y and z directions in model B1H. That is, the minimum, maximum, and normalization coefficients of the nearest neighbor point in the y direction are b1, b2, and Δb, respectively; and the minimum, maximum, and normalization coefficients of the nearest neighbor point in the z direction are c1, c2, and Δc, respectively.

[0090] Therefore, a1, a2, Δa; b1, b2, Δb; c1, c2, Δc constitute each vertex a1 in model A1. i The mapping relationship between the vertices of model B1H and the model B1H.

[0091] It should be noted that a KD-tree data structure can also be used to more efficiently calculate the nearest neighbor between each point in model A1 and model B1H. A KD-tree is a structure used to accelerate nearest neighbor searches in multidimensional space, significantly improving the efficiency of point-to-point searches. For each point a1 in model A1... i Use a KD-tree to query the nearest neighbor set in the B1H model and find the k points that are closest to it.

[0092] S224. Based on the vertex number correspondence and coordinate mapping relationship, the texture map and topology information of the 3D head model are transferred to the base model to realize automatic texture map adaptation, and finally the fused complete 3D cartoon digital human model is generated.

[0093] Based on the aforementioned steps, a new cartoon-style 2D image is generated for the user. Then, a new cartoon-style 3D head model A2 is generated based on this cartoon-style 2D image. Then, based on the model shaping transformation relationship between the aforementioned 3D head model A1 and the digital human standard model B1H, that is, the corresponding relationship of the above point numbers and the relative coordinate relationship of the points, a new digital human standard model head model B2H can be calculated, so that the B2H head model is consistent with the 3D head model A2.

[0094] Specifically, for each point A2 in the new 3D head model A2 i The reconstructed coordinates (x, y, z) are calculated according to the following formula:

[0095] x-axis coordinate reconstruction:

[0096] Difference reconstruction (points within the interval):

[0097] Normalized reconstruction (points outside the interval):

[0098] Where R x =x max -x min , representing the coordinate range of model B1H in the x-direction.

[0099] The same reconstruction method was applied to the y and z directions. The final reconstructed target point was obtained as follows:

[0100]

[0101] The UV mapping method uses barycentric coordinate interpolation to project each vertex in model B1H onto the nearest neighboring facet of model A1. By calculating the UV coordinates of the projection point of each vertex in B1H onto the texture map of model A1, the UV mapping of model B1H on the texture map of model A1 is obtained.

[0102] The UV mapping calculation process is as follows:

[0103] For each triangular facet of model B1H, calculate its geometric center (or centroid). Assuming the vertices of the triangular facet are v0, v1, and v2, then its center C... i (C) can be calculated using the following formula. i Let v0, v1, and v2 be the center coordinates of face i, and v0, v1, and v2 be the three vertices of face i.

[0104]

[0105] Find the nearest neighbor face:

[0106] For each vertex of model B1H, we need to find the closest face in model A1. To do this, we first calculate the center of all faces in model A1 and then index these center points using the KD-tree algorithm. For each vertex p of model B1H... a Query the KD tree to obtain the center of the face of model A1 that is closest to the given point. Assume the distance from vertex p... a The nearest center of the dough is C i If the index of the face is i, then the index of the face is i.

[0107] Calculate the centroid coordinates:

[0108] For each query vertex p a We examine its geometric relationship with candidate face i and calculate the centroid coordinates of that point relative to face i. If the vertices of face i are v0, v1, v2, then the centroid coordinates u, v, w can be calculated using the following formula:

[0109]

[0110] w = 1 - uv

[0111] Where, d 00 =<v0-v1,v0-v1> It is the dot product of the vectors from vertex v0 to v1. d 01 =<v0-v1,v0-v2> It is the dot product of the vectors from vertex v0 to v2. d 11 =<v1-v2,v1-v2> It is the dot product of the vectors from vertices v1 to v2. d 20 = <p a -v0, v0-v1> are vertices p a to v o The inner product of the vector and the edge v0→v1. 21 = <p a -v0,v0-v2> is the vertex p a to v o The inner product of the vector and the edge v0→v2. It is the denominator, ensuring that the calculated centroid coordinates are valid.

[0112] UV coordinate interpolation:

[0113] Once the centroid coordinates u, v, and w are obtained, they can be used for UV coordinate interpolation calculations. Assuming the UV coordinates corresponding to facet i are uv0, uv1, and uv2, i.e., the UV coordinates of the three vertices of the facet, the interpolation formula is as follows:

[0114] uv = u·uv0 + v·uv1 + w·uv2

[0115] Where uv0, uv1, and uv2 are the UV coordinates of the three vertices of the facet, and u, v, and w are the calculated centroid coordinates.

[0116] Through the above steps, it is ensured that the vertices of model B1H can be accurately mapped to the UV coordinates on the surface of model A1, thus completing the new UV map of the standard model B1H.

[0117] Hairstyles, clothing, accessories, motion animations, and facial animations created on the standard digital human model B1 can be automatically transferred to the new 3D model B2, thus completing the creation of the new cartoonish and adorable 3D digital human.

[0118] Step S3: Associate the 3D cartoon digital human model with a physical interactive card. The physical interactive card contains a near-field communication tag or QR code, which is used to store the unique identification information of the 3D cartoon digital human model.

[0119] Specifically, the process involves printing a 2D image, writing and embedding the NFC near-field communication chip into the casing kit, and then printing and cropping a customized 2D cartoon image. Next, this digital human ID number, along with the WeChat NFC device schema, is written into an NFC tag or a QR code is created. This NFC tag or QR code is then combined with the casing kit / printed cartoon image and encapsulated to produce a cartoon image card.

[0120] Since the NFC serial number or the aforementioned QR code content is bound to terminal application software such as WeChat mini programs, the terminal and 3D digital human can be opened by "tap" or "scan" the cartoon image card in the terminal application.

[0121] Step S4: Read the identification information of the physical interactive card through the terminal device, load and display the corresponding 3D cartoon digital human model, and support the user to perform real-time interactive operations with the 3D cartoon digital human model.

[0122] On smart terminals with NFC or QR code scanning capabilities, the NFC serial number or digital human ID number on the physical cartoon image card can be read via "tap" or "scan," opening the terminal application and loading the 3D digital human, scene, and motion animations. Users can interact with the digital human in real time by touching or swiping the screen, including motion animation switching, clothing switching, scene switching, and facial expression feedback, interacting with the corresponding 3D digital human on the physical cartoon image card. For example, when a user swipes the screen, the terminal application calls a skeletal animation interpolation algorithm to smoothly transition to the target action; when the user touches the clothing icon, the digital human's clothing texture is updated in real time through a texture replacement interface.

[0123] Example 2

[0124] Reference Figure 2 As shown, corresponding to Embodiment 1 above, Embodiment 2 of the present invention provides an interactive social system that combines virtual and real elements, the system comprising:

[0125] The image generation module receives user input images and generates customized 2D cartoon-style images. For example, the image generation module uses a pre-trained Stable Diffusion model, combined with a LoRA fine-tuned model (such as cartoon_style, StickersRedmond) for style transfer, with a weight configuration of 0.6. The 3D digital human generation module is based on the head reconstruction method of prior patent CN119693524A, and generates a cartoon-style 3D model through topological matching.

[0126] The 3D digital human generation module is used to generate a corresponding 3D cartoon digital human model based on the 2D cartoon-style image. For example, the 2D cartoon image output by the image generation module is transmitted to the 3D digital human generation module through a feature extraction interface (such as facial key point data) to drive head model reconstruction.

[0127] The physical interactive card generation module is used to associate the 3D cartoon digital human model with a near-field communication tag or QR code. For example, the physical interactive card generation module encodes the digital human ID in NDEF format and writes it into the NFC tag, or generates a QR code containing a cryptographic hash value to ensure data security and terminal compatibility.

[0128] The terminal application module loads and displays a 3D cartoon digital human model by reading a physical interactive card, supporting user interaction. For example, the terminal application module supports gesture recognition (such as swiping to switch actions) and AR scene fusion, calling the Unity animation engine to render digital human feedback in real time. The terminal application module is configured to be compatible with WeChat mini-programs and native iOS / Android applications.

[0129] The image generation module includes:

[0130] A basic image generation model is used to generate a preliminary cartoon-style image based on a user-input image; a fine-tuning model library contains multiple lightweight fine-tuning models for stylistic enhancement and detail adjustment of the preliminary cartoon-style image;

[0131] The facial feature extraction unit is used to extract facial regions and key features from the user input image; the style fusion unit is used to combine the edge structure information and facial features of the preset template image to achieve style transfer through the facial recognition-guided model.

[0132] The control network unit is used to control the contour and pose of the generated image based on the template edge information, so as to ensure the structural consistency between the generated image and the template.

[0133] The interactive social system in Embodiment 2 of the present invention is used to execute the interactive social method in Embodiment 1. For details not covered herein, please refer to the method in Embodiment 1, which will not be repeated here.

[0134] Example 3

[0135] Please see Figure 3 Embodiment 2 of the present invention provides an electronic device for virtual-real integrated interactive social networking. The electronic device includes at least one processor 201 and at least one memory 202. The processor 201 and the memory 202 are directly connected to each other, or communicate with each other through a communication interface 203, or are electrically connected through one or more communication buses or signal lines to realize data transmission or interaction. The memory 202 stores program instructions that can be executed by the processor 201. The processor 201 calls the program instructions to execute the virtual-real integrated interactive social networking system implementation method disclosed in Embodiment 1. For example, the system can: receive a frontal portrait image uploaded by a user; generate a customized 2D cartoon-style image for the user based on a preset image generation model and a fine-tuning model; generate a corresponding 3D cartoon digital human model based on the 2D cartoon-style image using 3D reconstruction technology; associate the 3D cartoon digital human model with a physical interactive card, the physical interactive card containing a near-field communication tag or QR code for storing the unique identification information of the 3D cartoon digital human model; and read the identification information of the physical interactive card through a terminal device, load and display the corresponding 3D cartoon digital human model, supporting real-time interactive operations between the user and the 3D cartoon digital human model.

[0136] The memory 202 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable read-only memory (EPROM), electrically erasable read-only memory (EEPROM), etc.

[0137] The processor 201 can be an integrated circuit chip with signal processing capabilities. The processor 201 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0138] Understandable Figure 3 The structure shown is for illustrative purposes only; the electronic device may also include components that are more advanced than those shown. Figure 3 The more or fewer components shown, or having the same Figure 3 The different configurations shown. Figure 3 The components shown can be implemented using hardware, software, or a combination thereof.

[0139] Example 4

[0140] This invention also provides a computer-readable storage medium storing a computer program that, when executed by processor 201, implements the method for implementing the virtual-real integrated interactive social system described in Embodiment 1. For example, it implements: receiving a frontal portrait image uploaded by a user; generating a customized 2D cartoon-style image of the user based on a preset image generation model and a fine-tuning model; generating a corresponding 3D cartoon digital human model based on the 2D cartoon-style image using 3D reconstruction technology; associating the 3D cartoon digital human model with a physical interactive card, the physical interactive card containing a near-field communication tag or QR code for storing the unique identification information of the 3D cartoon digital human model; and reading the identification information of the physical interactive card through a terminal device, loading and displaying the corresponding 3D cartoon digital human model, supporting real-time interactive operations between the user and the 3D cartoon digital human model.

[0141] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0142] It is worth noting that, for those skilled in the art, it will be apparent that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0143] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for implementing an interactive social system that combines virtual and real elements, characterized in that, The method includes: It receives frontal portrait images uploaded by users and generates customized 2D cartoon-style images for users based on preset image generation models and fine-tuning models. Based on the 2D cartoon-style image, a corresponding 3D cartoon digital human model is generated using 3D reconstruction technology, including: Based on the 2D cartoon-style image, a cartoon-style 3D head model is generated using 3D head reconstruction technology; The 3D head model is shaped and transformed with a preset cartoon body standard model. The head and body are then fused through vertex matching and coordinate mapping to generate a complete 3D cartoon digital human model. The shaping and transformation includes: separating the head portion of the standard model, adjusting its vertex positions to overlap with the 3D head model, establishing vertex indices and coordinate mapping relationships, and migrating texture maps to the fused model based on these mapping relationships. The step of separating the head portion of the standard model and adjusting its vertex positions to overlap with the 3D head model includes: A pre-made standard digital human model is selected as the benchmark model, which is a 3D model with a cute cartoonish figure; Adjust the size and position of the head region of the reference model so that it partially overlaps with the head or face region of the 3D head model; Traverse the vertices of the head region of the 3D head model and the reference model, and establish the correspondence between the vertex numbers and the relative coordinate mapping relationship in the overlapping area, including: by calculating the Euclidean distance between the head vertices of the 3D head model and the reference model, determine the nearest neighbor pairs, and record their numbers and coordinate offsets. Based on the vertex number correspondence and coordinate mapping relationship, the texture map and topology information of the 3D head model are transferred to the base model to achieve automatic texture map adaptation, and finally generate a fused complete 3D cartoon digital human model. The 3D cartoon digital human model is associated with a physical interactive card, which contains a near-field communication tag or QR code to store the unique identification information of the 3D cartoon digital human model. The terminal device reads the identification information of the physical interactive card, loads and displays the corresponding 3D cartoon digital human model, and supports real-time interactive operation between the user and the 3D cartoon digital human model.

2. The implementation method according to claim 1, characterized in that, The specific steps for generating the user's customized 2D cartoon-style image include: Load a base image generation model and at least one lightweight fine-tuning model, the fine-tuning model being used to enhance stylized features and detail representation; Extract facial features from the user input image and edge structure information from the preset template image; By combining facial recognition-guided models and style fusion models with text prompts and a control network, 2D cartoon images that conform to the target style are generated.

3. The implementation method according to claim 1, characterized in that, The fine-tuning model includes at least one of the following: sticker style conversion model, cartoon feature enhancement model, expression adjustment model, age control model, and cute style optimization model. The interactive operations include: action animation switching, clothing switching, scene switching, and expression feedback.

4. The implementation method according to claim 1, characterized in that, The baseline model is selected from either the SMPLX model or the CC4 model.

5. A virtual-real integrated interactive social system, characterized in that, The system includes: The image generation module is used to receive user-input images and generate customized 2D cartoon-style images; A 3D digital human generation module is used to generate a corresponding 3D cartoon digital human model based on the 2D cartoon-style image, including: Based on the 2D cartoon-style image, a cartoon-style 3D head model is generated using 3D head reconstruction technology; The 3D head model is shaped and transformed with a preset cartoon body standard model. The head and body are then fused through vertex matching and coordinate mapping to generate a complete 3D cartoon digital human model. The shaping and transformation includes: separating the head portion of the standard model, adjusting its vertex positions to overlap with the 3D head model, establishing vertex indices and coordinate mapping relationships, and migrating texture maps to the fused model based on these mapping relationships. The step of separating the head portion of the standard model and adjusting its vertex positions to overlap with the 3D head model includes: A pre-made standard digital human model is selected as the benchmark model, which is a 3D model with a cute cartoonish figure; Adjust the size and position of the head region of the reference model so that it partially overlaps with the head or face region of the 3D head model; Traverse the vertices of the head region of the 3D head model and the reference model, and establish the correspondence between the vertex numbers and the relative coordinate mapping relationship in the overlapping area, including: by calculating the Euclidean distance between the head vertices of the 3D head model and the reference model, determine the nearest neighbor pairs, and record their numbers and coordinate offsets. Based on the vertex number correspondence and coordinate mapping relationship, the texture map and topology information of the 3D head model are transferred to the base model to achieve automatic texture map adaptation, and finally generate a fused complete 3D cartoon digital human model. The physical interactive card generation module is used to associate the 3D cartoon digital human model with a near-field communication tag or QR code; The terminal application module is used to load and display 3D cartoon digital human models by reading physical interactive cards, supporting user interaction.

6. The interactive social system according to claim 5, characterized in that, The image generation module includes: A basic image generation model is used to generate preliminary cartoon-style images based on user input images; A fine-tuning model library, containing multiple lightweight fine-tuning models, is used to stylize and refine the details of the initial cartoon-style images; The facial feature extraction unit is used to extract facial regions and key features from user input images; The style fusion unit is used to combine the edge structure information and facial features of a preset template image, and guide the model to achieve style transfer through facial recognition. The control network unit is used to control the contour and pose of the generated image based on the template edge information, so as to ensure the structural consistency between the generated image and the template.

7. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the virtual-real integrated interactive social system implementation method as described in any one of claims 1 to 3.

Images