Interaction method and cloud server

Abstract

The application provides an interaction method and a cloud server. The method of the application obtains attribute information of a target object, an object 3D model and interaction content information; generates a personification 3D material suitable for the target object automatically according to the attribute information of the target object, and synthesizes the personification 3D material to the object 3D model to obtain a personification 3D model of the target object, without manually designing different personification images for the object; further, the personification 3D model of the object is displayed as a whole virtual image in an interaction page, and the personification 3D model is driven to perform an interaction behavior matched with the interaction content information, so that the personification 3D model can make a matched interaction behavior, without manually designing different animation actions for the object, with high efficiency, short cycle, and the ability to automatically adapt to different objects and different interaction contents, and more diversified and personified actions.

Description

Technical Field

[0001] This application relates to fields such as artificial intelligence, deep learning, machine learning, virtual reality, augmented reality, and human-computer interaction in computer technology, and in particular to an interaction method and a cloud server. Background Technology

[0002] Existing three-dimensional (3D) animation employs anthropomorphic techniques, giving objects life and enabling them to act, think, and express emotions like humans, providing viewers with a completely different experience from the object's perspective. For example, anthropomorphic 3D animation created on real-world objects can present users with anthropomorphic 3D animations of those objects.

[0003] Currently, both traditional methods of manually creating anthropomorphic 3D models by artists and methods of automatically generating anthropomorphic 3D models involve constructing 3D models of real-world objects and then creating / rendering two-dimensional (2D) anthropomorphic patterns (such as virtual facial features and virtual limb models) on these 3D models to form anthropomorphic 3D models and animations. The anthropomorphic patterns such as facial features and limbs added to the objects are all 2D materials. These are played repeatedly according to pre-set animation movements to form the animation. The movements in the animation are monotonous and repetitive. To improve the diversity of anthropomorphic animations for different objects, separate animation movements need to be set for each object, resulting in low efficiency, long cycles, and fixed movements with poor interactivity. Summary of the Invention

[0004] This application provides an interaction method and a cloud server to solve the problems of low efficiency, long cycle, fixed actions, and poor interactivity in the anthropomorphism of objects in traditional interaction scenarios.

[0005] Firstly, this application provides an interaction method, including:

[0006] In response to an interaction request for a target object, the system acquires the target object's attribute information, 3D model, and interaction content information.

[0007] Based on the attribute information of the target object, generate anthropomorphic 3D materials suitable for the target object, and composite the anthropomorphic 3D materials onto the 3D model of the object to obtain the anthropomorphic 3D model of the target object;

[0008] The anthropomorphic 3D model is displayed on the interactive page, and the anthropomorphic 3D model is driven to perform interactive behaviors that match the interactive content information.

[0009] Secondly, this application provides an interaction method, including:

[0010] In response to the end device's request for anthropomorphism of the target object, the attribute information, 3D model of the object, and broadcast content of the target object are obtained;

[0011] Based on the attribute information of the target object, generate anthropomorphic 3D materials suitable for the target object, and composite the anthropomorphic 3D materials onto the 3D model of the object to obtain the anthropomorphic 3D model of the target object;

[0012] The anthropomorphic 3D model is displayed on the interactive page, and the anthropomorphic 3D model is driven to complete the interactive behavior of broadcasting the broadcast content and generate the broadcast animation.

[0013] In response to the confirmation operation of the broadcast animation, the broadcast animation is provided to the terminal device.

[0014] Thirdly, this application provides an interaction method, including:

[0015] During real-time interaction, in response to an interaction request with the target object, the system acquires the target object's attribute information, 3D model, and interaction content information.

[0016] Based on the attribute information of the target object, generate anthropomorphic 3D materials suitable for the target object, and composite the anthropomorphic 3D materials onto the 3D model of the object to obtain the anthropomorphic 3D model of the target object;

[0017] The anthropomorphic 3D model is displayed on the real-time interactive page, and the anthropomorphic 3D model is driven to perform interactive behaviors that match the interactive content information.

[0018] Fourthly, this application provides a cloud server, including: a processor, and a memory communicatively connected to the processor;

[0019] The memory stores computer-executed instructions;

[0020] The processor executes computer execution instructions stored in the memory to implement the method described in any of the above aspects.

[0021] Fifthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in any of the above aspects.

[0022] The interactive method and cloud server provided in this application obtain the attribute information, 3D model, and interactive content information of the target object; based on the attribute information of the target object, automatically generate anthropomorphic 3D materials suitable for the target object, and composite the anthropomorphic 3D materials onto the 3D model of the object to obtain an anthropomorphic 3D model of the target object; display the anthropomorphic 3D model on the interactive page, and drive the anthropomorphic 3D model to perform interactive behaviors matching the interactive content information. By automatically generating suitable anthropomorphic 3D materials based on the object's attribute information and compositing them onto the 3D model of the object, the corresponding... Anthropomorphic 3D models eliminate the need for manually designing different anthropomorphic images for each object. The anthropomorphic 3D model of an object, as a unified virtual image, automatically drives itself based on interactive content information, enabling the anthropomorphic 3D model to perform matching interactive behaviors. This eliminates the need for manually designing different animations for each object, resulting in high efficiency, short cycle time, and automatic adaptation to different objects and interactive content. This makes the interactive behaviors based on the anthropomorphic image of objects more diverse and anthropomorphic, with a higher degree of matching with the interactive content, thus enhancing the expressiveness of the anthropomorphized object. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0024] Figure 1 An exemplary interactive system architecture diagram applicable to this application;

[0025] Figure 2 A flowchart illustrating an exemplary embodiment of this application;

[0026] Figure 3 An example diagram illustrating interactive behavior based on anthropomorphic objects, provided for an exemplary embodiment of this application;

[0027] Figure 4 An example diagram of real-time interaction provided for an exemplary embodiment of this application;

[0028] Figure 5 An interactive flowchart of a customized object anthropomorphic broadcasting animation scene provided for an exemplary embodiment of this application;

[0029] Figure 6 A flowchart illustrating a real-time interactive method based on anthropomorphic objects, provided as an exemplary embodiment of this application;

[0030] Figure 7 A schematic diagram of the structure of an interactive device provided in an exemplary embodiment of this application;

[0031] Figure 8This is a schematic diagram of the structure of a cloud server provided in an embodiment of this application.

[0032] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0033] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0034] First, let me explain the terms used in this application:

[0035] AI-Generated Content (AIGC) refers to content automatically generated using artificial intelligence (AI) algorithms, such as generating new content (like text or images) based on known text, audio, and images. The AI ​​algorithm model used to generate this content is called a generative model (AIGM), also known as a generative AI model.

[0036] Augmented Reality (AR): AR technology refers to augmented reality technology, which can connect real-world information with virtual-world information, such as adding virtual images to a captured image.

[0037] AI-driven digital humans: AI algorithms are used to "understand" content information such as text, voice or music, and then make decisions and generate the expressions and movements of digital humans, making digital humans come alive.

[0038] Multimodal interaction: This refers to human-computer interaction through various means such as text, voice, vision, action, and environment, fully simulating the interaction between people.

[0039] Traditional methods for constructing 3D models of real-world objects involve creating / rendering 2D anthropomorphic patterns on the 3D model of the object based on preset animation actions, generating anthropomorphic 3D models and animations, and then monotonously repeating the preset animation actions in the animations to achieve interaction based on the anthropomorphic objects. This method requires designing and creating corresponding animation actions for each object, which is inefficient, time-consuming, and results in fixed actions and poor interactivity.

[0040] This application provides an interactive method that acquires the attribute information, 3D model, and interactive content information of a target object; automatically generates anthropomorphic 3D materials suitable for the target object based on its attribute information, and composites these materials onto the object's 3D model to obtain an anthropomorphic 3D model of the target object; displays the anthropomorphic 3D model on an interactive page, and drives the anthropomorphic 3D model to perform interactive behaviors matching the interactive content information. By automatically generating suitable anthropomorphic 3D materials based on the object's attribute information and compositing them onto the object's 3D model, a corresponding anthropomorphic 3D model of the object is obtained. This eliminates the need for manually designing different anthropomorphic images for each object, and the anthropomorphic 3D model of the object, as a whole virtual image, can automatically drive the anthropomorphic 3D model based on the interactive content information, enabling the anthropomorphic 3D model to perform matching interactive behaviors. This method is highly efficient, has a short cycle time, is applicable to various types of objects, supports user-defined interactive content information, supports multimodal interaction, and has good interactivity.

[0041] The interactive method provided in this application can be applied to scenarios such as e-commerce, live video streaming, and film / animation production. For example, it can be used in e-commerce for product promotion, such as generating anthropomorphic product announcement videos; or in live video streaming, it can use anthropomorphic images of objects to replace the anchor / virtual anchor and complete the interaction with the viewers; or in animation production, it can automatically generate anthropomorphic images of object characters, drive the anthropomorphic images of objects to perform corresponding behaviors based on given script content, and record and store them as animated videos. The interactive method provided in this application can also be applied to intelligent customer service scenarios in e-commerce, or other scenarios that require generating anthropomorphic images of objects and driving the anthropomorphic images of objects to perform certain interactive behaviors, which will not be listed here.

[0042] Figure 1 This is an exemplary interactive system architecture diagram applicable to this application. Figure 1 As shown, Figure 1 As shown, the system architecture may specifically include cloud servers, edge devices, and data service devices.

[0043] Specifically, the cloud server can be a server cluster located in the cloud. The cloud server has a communication link with each end-device, enabling communication between the cloud server and each end-device. The cloud server includes three modules: image generation, dialogue model, and AI-driven engine. The image generation module automatically generates a human-like 3D model of the object based on its attribute information. The dialogue model enables real-time question-and-answer and dialogue functions, obtaining interactive content information based on user input data. In some scenarios, the interactive content information can be directly provided to the cloud server by the user through the end-device, or obtained by the cloud server from data service devices. The AI-driven engine drives the human-like 3D model of the object to perform interactive behaviors that match the interactive content information, enabling user interaction with the human-like object. Figure 1 The dashed lines in the text indicate that different methods for obtaining interactive content information are optional. In different application scenarios, cloud servers can use different methods to obtain interactive content information.

[0044] Specifically, edge devices can be hardware devices used by various users that have network communication, computing, and information display functions, including but not limited to smartphones, tablets, desktop computers, AR devices, and IoT devices.

[0045] Data service devices can be those capable of providing cloud servers with 3D models of objects, attribute information, and multimodal knowledge data from at least one source. These devices provide an Application Programming Interface (API) for acquiring relevant data about the target object, allowing the server to obtain the required data by calling the API. For example, a data service device could be a server storing product-related data and 3D models on an e-commerce platform or live streaming platform, or a user-specified device.

[0046] In one possible application scenario, this solution can be used to generate a human-like 3D model of a target object based on a user-specified target object and broadcast content, and then generate a broadcast animation matching the broadcast content based on this human-like 3D model. The user can send a human-like request for the target object to the cloud server through their device, carrying information about the target object and the broadcast content. Based on the received human-like request, the cloud server identifies and determines the target object, and obtains its attribute information, 3D model, and broadcast content. The cloud server generates human-like 3D material suitable for the target object based on its attribute information, and composites this material onto the 3D model to obtain the human-like 3D model of the target object. This human-like 3D model is then displayed on an interactive page, driving it to complete the interactive behavior of broadcasting the content and generating a broadcast animation. The cloud server provides the broadcast animation to the device. The device can then deliver the broadcast animation based on the human-like 3D model of the target object through various means.

[0047] In one possible application scenario, this solution can be applied to real-time interactive scenarios. The cloud server provides a real-time interactive interface. During real-time interaction, users can send interaction requests to the cloud server via their edge devices, carrying information about the target object. Based on the received anthropomorphic requests for the target object, the cloud server identifies and determines the target object, and obtains its attribute information, 3D model, and interactive content information. The interactive content information can be pre-set announcements corresponding to the target object, or responses generated by the cloud server based on user input data. The cloud server generates anthropomorphic 3D materials suitable for the target object based on its attribute information, and then composites these materials onto the object's 3D model to obtain an anthropomorphic 3D model of the target object. This anthropomorphic 3D model is displayed on a real-time interactive page, and it is driven to perform interactive behaviors that match the interactive content. This allows the target object to interact with the user in a real-time interactive scenario through an anthropomorphic image, engaging in real-time dialogue from a "first-person" perspective. This brings the originally static object to life, making it more interactive and engaging, thus enhancing the user's interaction with the target object and increasing the fun of the interaction.

[0048] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0049] Figure 2This is a flowchart of an interactive method provided for an exemplary embodiment of this application. The execution entity of this embodiment is the cloud server in the above-described interactive system architecture. The method provided in this embodiment is used to automatically generate anthropomorphic 3D materials that match the attributes of the target object, automatically anchor the position of the anthropomorphic 3D materials on the target object, composite the anthropomorphic 3D materials onto the object's 3D model, generate an anthropomorphic 3D model of the target object, and drive the anthropomorphic 3D model to perform interactive behaviors that match the interactive content information, thereby realizing an interactive method based on the anthropomorphic image of an object.

[0050] like Figure 2 As shown, the specific steps of this method are as follows:

[0051] Step S201: In response to the interaction request for the target object, obtain the target object's attribute information, 3D model of the object, and interaction content information.

[0052] In this context, an interaction request to a target object can be a request to initiate interaction with the target object. In different application scenarios, the specific form of an interaction request to a target object can vary.

[0053] For example, a merchant user might initiate a request to generate a human-like promotional video for a product. This request could include at least one of the following: a product image and descriptive information. The product image could be a photograph taken by the user containing the product, or a pre-created image of the product's appearance. The product description could include identification information (such as product name, code, etc.) and category information (such as product class). Based on this description, the cloud server can identify the target product to obtain its attribute information, 3D model, and interactive content. The cloud server then generates a promotional video of the target product and returns it to the user.

[0054] For example, in a live streaming scenario, the host / virtual host initiates a request for a designated target object to interact with the user using a human-like appearance. The host / virtual host might issue a pre-set message like "Please welcome object XXX," where "XXX" refers to the object's name or code. Based on this identification information, the cloud server can identify the target object (which could be a product in the live stream or the object that will act as the virtual host) to obtain its attribute information, 3D model, and interactive content. The cloud server then generates and drives the human-like 3D model of the target object, enabling real-time interaction between the human-like appearance of the object and the user through the live streaming interface.

[0055] For example, in an intelligent customer service scenario, a user sends a link to a target object to the cloud server. Based on this link, the cloud server can identify and determine the target object, thereby obtaining its attribute information, 3D model, and interactive content. The cloud server then generates and drives a human-like 3D model of the target object, enabling real-time interaction between the target object and the user from a first-person perspective, based on the customer service interface. This enhances the interactivity and engagement of the customer service system.

[0056] In this embodiment, interactive content information refers to the content information output by the target object to the user based on the interactive request. Interactive content information can be provided by the user through the terminal device, or it can be the response content information automatically generated by the cloud server in response to user input data.

[0057] Step S202: Based on the attribute information of the target object, generate anthropomorphic 3D material suitable for the target object, and composite the anthropomorphic 3D material onto the object's 3D model to obtain the anthropomorphic 3D model of the target object.

[0058] The attribute information of the target object includes, but is not limited to, the target object's identification information, color, shape, and category.

[0059] In this embodiment, the cloud server automatically generates anthropomorphic 3D materials that match the attributes of the target object based on the target object's attribute information. Further, the cloud server automatically anchors the anthropomorphic 3D materials onto the target object based on the target object's attribute information and pre-set scaling parameters, and composites the anthropomorphic 3D materials onto the corresponding positions on the object's 3D model to generate an anthropomorphic 3D model of the target object.

[0060] For example, a cloud server can determine the connection positions of anthropomorphic 3D materials of a target object based on its outline and shape, the central axis (horizontal or vertical), and the aspect ratio of the object's 3D model. The connection position of the anthropomorphic 3D material can be a point (called a connection point) or a region (called a connection region). For example, determining the connection points for the upper limbs, the lower limbs, and the connection regions for the eyes, nose, and mouth. The connection points for the limbs can be symmetrically set on both sides of the central axis of the target object's 3D model, and their specific positions need to be determined according to the aspect ratio. For example, for a vertically oriented cuboid target object, such as a lipstick, the connection points for the upper limbs need to be adjusted upwards, and the connection points for the lower limbs need to be adjusted downwards. For a horizontally oriented cuboid target object, such as a tissue box, the connection points for the upper limbs need to be adjusted in the middle. These adjustments make the proportions of the final anthropomorphic image (anthropomorphic 3D model) more harmonious and reasonable.

[0061] Step S203: Display an anthropomorphic 3D model on the interactive page and drive the anthropomorphic 3D model to perform interactive behaviors that match the interactive content information.

[0062] After generating a human-like 3D model of the target object, the cloud server can display the human-like 3D model of the target object in the current or new interactive page, and use AI-driven algorithms to drive the human-like 3D model to make interactive behaviors that match the interactive content information, including facial expressions, mouth shapes, body movements, etc., to achieve interaction based on the human-like image of the object.

[0063] Among these features, AI-driven algorithms can automatically adapt to different objects and interactive content, making interactive actions based on anthropomorphic object images more diverse and human-like, with a higher degree of matching with the interactive content. The animations of the target object anthropomorphic images presented by AI-driven algorithms are dynamically generated based on the object's image characteristics and interactive content. The animations produced by this mode are not simply monotonous repetitions of a few pre-set actions, but possess high diversity and can better match the object, enhancing the expressiveness of the anthropomorphized object.

[0064] For example, Figure 3 This is an example diagram illustrating an interactive behavior based on the generation of anthropomorphic images of objects, as provided in this embodiment. Figure 3 Taking a strawberry as an example, 3D models of facial features and limbs are synthesized onto a 3D model of the strawberry, creating an anthropomorphic strawberry image. Driven by AI algorithms, the anthropomorphic strawberry image performs complex and diverse movements, rather than simply repeating a few preset actions. It should be noted that... Figure 3 The images in the animation are just multiple frames extracted from the interactive behavior generated by the driver. There are also multiple frames between two adjacent images, which makes the actions in two adjacent images coherent and smooth.

[0065] In this embodiment, the system acquires the target object's attribute information, 3D model, and interactive content information. Based on the target object's attribute information, it automatically generates anthropomorphic 3D materials suitable for the target object and composites these materials onto the object's 3D model to obtain an anthropomorphic 3D model of the target object. The anthropomorphic 3D model is then displayed on the interactive page, and it is driven to perform interactive behaviors matching the interactive content information. This system can automatically generate suitable anthropomorphic 3D materials based on the object's attribute information and composite them onto the object's 3D model to obtain the corresponding anthropomorphic 3D model of the object, eliminating the need for manually designing different anthropomorphic images for each object. Furthermore, by treating the anthropomorphic 3D model of the object as a unified 3D virtual image and driving it according to the interactive content information, the anthropomorphic 3D model can perform matching interactive behaviors, eliminating the need for manually designing different animation actions for each object. This method is highly efficient, has a short cycle time, and is applicable to various types of objects. It supports user-defined interactive content information, multimodal interaction, and better interactivity.

[0066] In one optional embodiment, the user submits an image of the target object to the cloud server. Based on the image, the cloud server identifies the target object's attribute information and obtains its 3D model. In step S201 above, the acquisition of the target object's attribute information and 3D model in response to an interaction request is specifically implemented as follows:

[0067] In response to an interaction request with the target object, acquire an image of the target object; based on the image of the target object, identify the attribute information of the target object, and acquire a 3D model of the target object.

[0068] Specifically, in response to an interaction request with the target object, the system receives an image of the target object sent by the terminal device, or retrieves an image of the target object carried in the interaction request. The image of the target object can be an image taken by the user or an image created by the owner of the target object.

[0069] Furthermore, based on the image of the target object, the attribute information of the target object is identified, wherein the attribute information includes at least one of the following: color, shape, and category.

[0070] Specifically, by performing visual inspection on the image of the target object, the color of the target object is determined, such as its main color or the areas covered by different colors. Based on the image of the target object, the target object and background are automatically identified, and image segmentation is performed to completely separate the target object from the background. The contour points of the target object are then calculated to obtain its contour information. The shape of the target object can be determined based on this contour information. Classification and recognition of the target object image can determine its category, such as identifying its SKU (Stock Keeping Unit) identifier, category, etc.

[0071] Furthermore, obtaining the 3D model of the target object can be achieved in the following ways:

[0072] Based on the image of the target object, its contour and 3D geometric information are detected. A 3D model of the target object is then generated based on this information. This method of 3D modeling, which detects the object's contour and 3D geometric information, allows for the automatic generation of a 3D model for objects without a pre-existing model. This approach can be applied to any object, broadening the scope of interactive methods based on anthropomorphic object representations. 3D modeling of target objects can be achieved using existing modeling software, which will not be elaborated upon here.

[0073] Optionally, 3D models of various object categories can be pre-built and stored. When obtaining the 3D model of the target object, the 3D model corresponding to the category of the target object can be obtained from the pre-built 3D models of each category. This saves time in 3D object modeling, improves the efficiency of generating anthropomorphic images of the target object, and thus improves the efficiency of interaction based on anthropomorphic images of objects.

[0074] Optionally, when obtaining the 3D model of the target object, the 3D model corresponding to the category of the target object can be obtained from a pre-built set of 3D models corresponding to each category. If no 3D model corresponding to the category of the target object is found, the contour information and 3D geometric information of the target object can be detected from the image of the target object using 3D modeling methods; and a 3D model of the target object can be generated based on the contour information and 3D geometric information.

[0075] The solution in this embodiment supports application scenarios where users only provide images of the target object to the cloud server.

[0076] In another alternative embodiment, the user submits a description of the target object to the cloud server. Exemplarily, the description of the target object may include at least one of the following: name, category, and details page link. In response to an interaction request with the target object, the cloud server retrieves the target object's attribute information and 3D model based on the description of the target object.

[0077] Specifically, retrieving the attribute information and 3D model of a target object can be done through either local retrieval or external retrieval. For example, the attribute information and 3D models of different objects can be pre-stored on a cloud server. In this case, the cloud server can directly retrieve the attribute information and 3D model of the target object from local storage. Alternatively, the attribute information and 3D model of different objects can be pre-stored on another data service device (such as a server or other device). The cloud server can then retrieve the attribute information and 3D model corresponding to the target object's identification information (such as name or category) from the data service device.

[0078] The solution in this embodiment supports application scenarios where users provide one or more descriptive information about a target object to the cloud server.

[0079] In an optional embodiment, the attribute information of the target object obtained in step S201 includes category and color. In step S202, based on the attribute information of the target object, an anthropomorphic 3D material suitable for the target object is generated, which can be implemented in the following way:

[0080] Obtain basic anthropomorphic 3D assets corresponding to the target object; adjust the configuration parameters of the basic anthropomorphic 3D assets according to the category and color of the target object to obtain anthropomorphic 3D assets suitable for the target object. The anthropomorphic 3D assets include: facial feature 3D models and / or limb 3D models.

[0081] For example, a set of anthropomorphic 3D assets (including 3D models of facial features and limbs) can be set as the default basic anthropomorphic 3D assets, that is, as the basic anthropomorphic 3D assets corresponding to any object. Alternatively, different basic anthropomorphic 3D assets can be set for different categories of objects. Or, objects can be divided into several major categories, and different basic anthropomorphic 3D assets can be set for objects in different major categories, with different objects in the same major category corresponding to the same basic anthropomorphic 3D assets.

[0082] Optionally, the cloud server can display preset anthropomorphic 3D assets for users to select the desired anthropomorphic 3D asset as the base anthropomorphic 3D asset for the target object. In response to the user's selection of any preset anthropomorphic 3D asset, the selected anthropomorphic 3D asset is used as the base anthropomorphic 3D asset for the target object.

[0083] Furthermore, the cloud server can adjust the configuration parameters of the basic anthropomorphic 3D material based on the category and color of the target object. This includes, but is not limited to, adjusting the material (such as textures, reflections, etc.) and shape of the basic anthropomorphic 3D material, so that the adjusted anthropomorphic 3D material can better match the category, color, and atmosphere of the target object. For example, the shape can be adjusted by performing mesh deformation on the basic anthropomorphic 3D material using a mesh deformer.

[0084] This embodiment allows users to fine-tune the automatically generated anthropomorphic 3D model of a target object. After adjusting the configuration parameters of the basic anthropomorphic 3D material according to the target object's category and color to obtain anthropomorphic 3D material suitable for the target object, the cloud server can also display the configuration parameters of the anthropomorphic 3D material. Users can edit the displayed configuration parameters of the anthropomorphic 3D material. In response to the editing operation of the configuration parameters of the anthropomorphic 3D material, the cloud server updates the anthropomorphic 3D material of the target object according to the edited configuration parameters. In addition, one or more configuration parameters with different characteristics (such as long legs, cute, etc.) can be preset, and users can adjust the configuration parameters of the basic anthropomorphic 3D material to one of them.

[0085] In an optional embodiment, in step S202 above, the anthropomorphic 3D material is composited onto the object's 3D model to obtain the anthropomorphic 3D model of the target object. This can be achieved in the following way:

[0086] Based on the shape and outline information of the object's 3D model, determine the connection positions of the anthropomorphic 3D material on the object's 3D model; based on the connection positions, composite the anthropomorphic 3D material onto the corresponding connection positions to obtain the anthropomorphic 3D model of the target object.

[0087] In this embodiment, the cloud server can automatically anchor the connection position of the anthropomorphic 3D material on the target object according to the attribute information of the target object and the pre-set scale parameters, and synthesize the anthropomorphic 3D material onto the corresponding connection position on the 3D model of the object to obtain the anthropomorphic 3D model of the target object.

[0088] Specifically, based on the different shapes of the objects, several shape types are defined, and different anchoring algorithms / rules are set for different shape types. Each anchoring algorithm / rule includes a set of proportional parameters. When anchoring the connection position of the anthropomorphic 3D material on the target object's 3D model, the shape type of the target object is determined based on its shape. The connection position of the anthropomorphic 3D material on the object's 3D model is then determined according to the anchoring algorithm / rules corresponding to the target object's shape type, that is, according to the preset proportional parameters corresponding to the target object's shape type.

[0089] Shape types include, but are not limited to, at least one of the following: vertical tubular objects, horizontal tubular objects, spheres, near-spheres, squares, and near-squares. These shape types can be categorized based on the aspect ratio of the object's bounding box. The object's bounding box can be a hexahedral bounding box, which can be understood as a limiting frame or boundary box. The geometric center of the hexahedral bounding box is also the center of the object. In other words, the geometric center of the object is determined as the center of the object's 3D model. The length and width of the object's 3D model refer to the length (horizontal) and width (vertical) of the front face (corresponding to the object's frontal view) of the object's 3D model's hexahedral bounding box. The aspect ratio of the object's 3D model refers to the ratio of the length (horizontal) to the width (vertical) of the front face (corresponding to the object's frontal view) of the object's 3D model's hexahedral bounding box. The front face of the hexahedral bounding box is also the front face of the object, and can be set according to different objects. For objects without a clearly defined front face, one face of the hexahedral bounding box can be randomly selected as the front face, or the face with the larger area can be set as the front face.

[0090] Among them, the preset proportional parameters corresponding to any shape type include the proportional range of the distance between the connection position of each anthropomorphic 3D material on the object 3D model and the side of the bounding box of the object 3D model relative to the length of the bounding box, and the proportional range of the distance between the connection position and the top and bottom surfaces of the bounding box of the object 3D model relative to the width of the bounding box.

[0091] Optionally, different anchoring algorithms / rules can be set for different shape types. An anchoring algorithm / rule includes a set of proportional parameters and can also include preset anchoring rules. For example, the connecting area of ​​facial features cannot cover preset patterns on the object, such as trademarks or other identifying patterns, or unique functional areas of the object. The color difference between the corresponding connecting area and the surrounding area of ​​the anthropomorphic 3D material and the color of the anthropomorphic 3D material reaches a preset difference threshold, resulting in a significant color contrast between the anthropomorphic 3D material and its surrounding area. The preset difference threshold can be set according to the needs of the actual application scenario. Different preset difference thresholds can be set for anthropomorphic 3D materials of different categories of objects, and different anthropomorphic 3D materials can also correspond to different preset difference thresholds.

[0092] For example, for vertical tubular objects such as lipsticks and beverage bottles, the connection area of ​​the facial features 3D model can be selected on the lower half of the lipstick, choosing a facial feature 3D model with a color that contrasts significantly with the lower half of the lipstick bottle; the connection area of ​​the facial features 3D model should avoid the beverage bottle's logo. For horizontal tubular objects such as tissue boxes, the connection points of the upper limbs would be relatively far apart on the sides of the tissue box, so the connection points of the upper limbs can be placed relatively close to the center. For objects with small length and width ratios (such as spheres, near-spheres, squares, and near-squares), such as strawberries and apples, the connection area of ​​the facial features 3D model can be selected in the center and evenly distributed on the front of the object's 3D model.

[0093] In addition, any anchoring algorithm / rule includes preset ratio parameters and preset anchoring rules, and supports user-defined ratio parameters and anchoring rules, or the generation of new algorithms / rules by adjusting the preset ratio parameters and anchoring rules.

[0094] In this embodiment, the size of the object is the same as the size of its 3D model, or the size of the object is proportionally reduced or enlarged compared to the size of its 3D model. Optionally, before compositing the anthropomorphic 3D material onto the corresponding connection position, the size of the anthropomorphic 3D material can be adjusted according to the connection position to match the size of the anthropomorphic 3D material with the size of the object's 3D model, thereby making the size of the anthropomorphic 3D material on the generated anthropomorphic 3D model of the object more reasonable and coordinated.

[0095] Optionally, before obtaining the anthropomorphic 3D model of the target object by compositing the anthropomorphic 3D material onto the corresponding connection position, the cloud server can also display the connection position of the anthropomorphic 3D material on the object's 3D model, allowing users to adjust the connection position of any anthropomorphic 3D material on the object's 3D model. Furthermore, in response to the adjustment operation of the connection position of the anthropomorphic 3D material on the object's 3D model, the cloud server updates the connection position of the anthropomorphic 3D material on the object's 3D model.

[0096] Optionally, based on the connection position, the anthropomorphic 3D material is composited onto the corresponding connection position. Before obtaining the anthropomorphic 3D model of the target object, the cloud server can also adjust the color of the area surrounding the connection position of the anthropomorphic 3D material on the object's 3D model, and automatically adjust the color of the connection position and / or the anthropomorphic 3D material.

[0097] For example, by adjusting the color of the area surrounding the connection point of the anthropomorphic 3D material on the 3D model of the object, the color of the anthropomorphic 3D material and the area surrounding the corresponding connection point can be made to have a large contrast.

[0098] For example, by adjusting the color of the anthropomorphic 3D material based on the color of the area surrounding the connection point on the object's 3D model, a greater contrast can be achieved between the anthropomorphic 3D material and the color of the area surrounding the corresponding connection point.

[0099] Optionally, based on the connection positions, anthropomorphic 3D materials are composited onto the corresponding connection positions to obtain an anthropomorphic 3D model of the target object. Afterward, the cloud server can also display the anthropomorphic 3D model and support users adjusting the connection positions of the anthropomorphic 3D materials on the object's 3D model. Furthermore, in response to adjustments to the connection positions of the anthropomorphic 3D materials on the anthropomorphic 3D model, the cloud server updates the anthropomorphic 3D model.

[0100] In this embodiment, after obtaining the final anthropomorphic 3D model of the target object, the anthropomorphic 3D model, as a whole virtual image, is driven to perform corresponding behaviors. When the camera or scene rotates on the interactive page, the anthropomorphic 3D material and the object's 3D are automatically tracked and aligned, and the anthropomorphic 3D model, as a whole virtual image, follows the rotation of the camera or scene.

[0101] In an optional embodiment, in a real-time interaction scenario, step S201 above, in response to an interaction request with the target object, obtains the interaction content information of the target object, which can be implemented in the following way:

[0102] In response to a first interaction request with a target object, the system acquires input dialogue content and multimodal knowledge data of the target object from at least one source. The first interaction request is a real-time interaction request. This request includes information about the target object and dialogue content input by the user.

[0103] The cloud server acquires dialogue content input by the user, such as receiving dialogue text sent by the user through a client device; this dialogue text constitutes the dialogue content. Alternatively, it receives dialogue audio sent by the user through a client device and converts the audio into dialogue text to obtain the dialogue content. Based on the target object's information, the cloud server can determine the target object, acquire its attribute information and 3D model, generate anthropomorphic 3D materials suitable for the target object based on its attribute information, and composite these anthropomorphic 3D materials onto the target object's 3D model to obtain the anthropomorphic 3D model of the target object. The cloud server can also acquire multimodal knowledge data of the target object from at least one source, based on its information. The cloud server can obtain multimodal knowledge data of the target object from at least one source from local storage space or other data service platforms.

[0104] Furthermore, in step S203 above, displaying a humanoid 3D model on the interactive page and driving the humanoid 3D model to perform interactive behaviors that match the interactive content information can be specifically implemented in the following way:

[0105] The system invokes a dialogue model to generate response information based on multimodal knowledge data of the target object from at least one source. A human-like 3D model of the target object is displayed on the first interactive page, and the human-like 3D model is driven to perform actions that broadcast the response information based on the feedback. These actions include at least one of facial expressions, lip movements, and body language.

[0106] Among them, the multimodal knowledge data of the target object from at least one source includes, but is not limited to: frequently-asked questions (FAQ) knowledge base, attribute knowledge, titles, knowledge graphs, and comment data.

[0107] The dialogue model used to generate responses for the dialogue can be any dialogue model that supports multimodal interaction. For example, a unified dialogue model based on knowledge, personality, emotion, and memory, such as a generative model, can be used. This unified dialogue model collects massive amounts of data, possesses a certain degree of general dialogue capability, and also has multi-turn, multi-source heterogeneous knowledge dialogue capability. It can access various multi-source heterogeneous knowledge such as FAQ knowledge bases related to the target object, attribute knowledge, titles, knowledge graphs, and comment data for question answering and dialogue.

[0108] For example, Figure 4 An example diagram of real-time interaction is shown, such as Figure 4 As shown, the target object is the product "pumpkin seeds". The dialogue model generates response information based on multi-source heterogeneous multimodal indication data such as FAQ, product attributes, product title, point of interest (POI) tags, and product reviews.

[0109] Furthermore, after obtaining the response information from the dialogue, a human-like 3D model of the target object is displayed on the first interactive page. An AI-driven engine then drives the human-like 3D model to broadcast the response information based on the input text or speech. The response information can be text or speech, and the AI-driven engine can drive the human-like 3D model to make appropriate facial expressions, lip movements, and body gestures based on the input text or speech. The AI-driven engine can be implemented using AI-driven algorithms that support text and / or speech-driven virtual characters, which will not be elaborated here. The AI-driven algorithm drives the human-like 3D model of the target object to perform interactive behaviors matching the input response information. For the same target object, it can adapt to different response information, allowing the human-like 3D model to perform different interactive behaviors. The output of the AI-driven algorithm can automatically adapt to different objects and different styles of interactive content for the same object, making the interactive behavior of the human-like image more diverse and human-like, and with a higher degree of matching with the interactive content.

[0110] In an optional embodiment, in the scenario of a customized object anthropomorphic broadcast video, step S201 above, in response to an interaction request with the target object, obtains the interaction content information of the target object, which can be implemented in the following way:

[0111] In response to a second interaction request with the target object, the system retrieves the input broadcast content. The second interaction request is an interaction request to retrieve a human-like broadcast video of the target object. This request includes information about the target object and the broadcast content input by the user.

[0112] The cloud server can identify the target object based on its information, obtain its attribute information and 3D model, generate anthropomorphic 3D materials suitable for the target object based on its attribute information, and composite these anthropomorphic 3D materials onto the target object's 3D model to obtain the anthropomorphic 3D model of the target object. The cloud server can also obtain the broadcast content input by the user, such as receiving text broadcast content sent by the user through a terminal device, or receiving audio broadcast content sent by the user through a terminal device.

[0113] Furthermore, in step S203 above, displaying a humanoid 3D model on the interactive page and driving the humanoid 3D model to perform interactive behaviors that match the interactive content information can be specifically implemented in the following way:

[0114] The second interactive page displays a human-like 3D model of the target object, and drives the human-like 3D model to perform actions that broadcast the broadcast content, based on the broadcast content. These actions include at least one of facial expressions, lip movements, and body language.

[0115] In this embodiment, after acquiring the broadcast content and generating a humanoid 3D model of the target object, the humanoid 3D model of the target object is displayed on the second interactive page. An AI-driven engine then drives the humanoid 3D model to perform actions that broadcast the content, based on the broadcast content. The broadcast content can be text or speech, and the AI-driven engine can drive the humanoid 3D model to make reasonable facial expressions, lip movements, and body movements based on the input text or speech. The AI-driven engine can be implemented using AI-driven algorithms that support text and / or speech-driven virtual characters, which will not be elaborated here. The AI-driven algorithm drives the humanoid 3D model of the target object to perform interactive behaviors matching the broadcast content. For the same target object, it can adapt to different broadcast content, allowing the humanoid 3D model to perform different interactive behaviors. The output of the AI-driven algorithm can automatically adapt to different objects and different styles of interactive content for the same object, making the interactive actions of the humanoid image in the broadcast video more diverse and human-like, and with a higher degree of matching with the broadcast content.

[0116] In an optional embodiment, in a video scenario where a customized, anthropomorphic image of an object dances to specified audio data (such as music), step S201 above, in response to an interaction request with the target object, obtains the interaction content information of the target object. This can be implemented in the following way:

[0117] In response to a third interaction request with the target object, input audio data is acquired. This third interaction request is an interaction request to acquire a video of an anthropomorphic image of the target object dancing in sync with specified audio data. This request includes information about the target object and the audio data input by the user. The audio data can be a piece of music or a recorded human voice.

[0118] The cloud server can identify the target object based on its information, obtain the target object's attribute information and 3D model, generate anthropomorphic 3D materials suitable for the target object based on the target object's attribute information, and then composite the anthropomorphic 3D materials onto the target object's 3D model to obtain the anthropomorphic 3D model of the target object.

[0119] Furthermore, in step S203 above, displaying a humanoid 3D model on the interactive page and driving the humanoid 3D model to perform interactive behaviors that match the interactive content information can be specifically implemented in the following way:

[0120] The third interactive page displays an anthropomorphic 3D model of the target object and drives the anthropomorphic 3D model to perform dance movements that match the audio data. The dance movements include at least one of facial expressions, lip movements, and body movements.

[0121] In this embodiment, after acquiring the audio data and generating a humanoid 3D model of the target object, the humanoid 3D model is displayed on the third interactive page. An AI-driven engine then drives the humanoid 3D model to perform dance movements matching the audio data. The AI-driven engine can be implemented using an AI-driven algorithm that supports audio data-driven virtual character animation; details will not be elaborated here. By using the AI-driven algorithm to drive the humanoid 3D model of the target object to perform dance movements matching the input audio data, the function of music-driven humanoid representation of the target object dancing is achieved.

[0122] Based on the solution provided by any of the above method embodiments, the target object can be one or more, and the interactive content information of any target object can be determined based on user input data or based on the output data of other virtual images (such as virtual characters or anthropomorphic images of objects) to achieve the effect of linkage and interaction of multiple objects.

[0123] In one optional embodiment, by using a multimodal AI-driven engine that combines deep learning and motion graph algorithms, reasonable facial expressions, lip movements, and body movements can be generated from inputs such as text, voice, video, or random signals. These movements include directional indications, interactive dialogue behaviors (greetings, smiling), and dancing to music. This AI-driven engine can automatically adapt to different objects and different modalities of interactive content, making the interactive actions based on anthropomorphic object images more diverse and human-like, with a higher degree of matching with the interactive content. The animation of the target object's anthropomorphic image presented by the AI-driven engine is dynamically generated based on the characteristics of the object image and the input information (text, images, videos, settings). The animation produced by this mode is not a monotonous repetition of a few pre-set actions, but has high diversity and can better match the object, improving the expressiveness of the anthropomorphized object.

[0124] Figure 5 An interactive flowchart illustrating a customized object-anthropomorphic broadcast animation scene provided as an exemplary embodiment of this application. (For example...) Figure 5 As shown, the specific steps of this method are as follows:

[0125] Step S501: In response to the end device's request for anthropomorphism of the target object, obtain the target object's attribute information, 3D model of the object, and broadcast content.

[0126] In this embodiment, the user sends a request for anthropomorphism of a target object to the cloud server through their edge device. This request may be for obtaining an anthropomorphic broadcast animation of the target object. The user inputs the broadcast content and sends it to the cloud server through the edge device. This request for anthropomorphism of the target object carries information about the target object, such as its name, code, and category.

[0127] In response to a request from the edge device to anthropomorphize a target object, the cloud server obtains information about the target object, identifies the target object based on this information, and acquires its attribute information and 3D model. The cloud server then receives broadcast content sent by the edge device. This broadcast content can be text, voice, video, etc. The object's attribute information includes, but is not limited to, its color, category, and shape.

[0128] Step S502: Based on the attribute information of the target object, generate anthropomorphic 3D material suitable for the target object, and composite the anthropomorphic 3D material onto the object's 3D model to obtain the anthropomorphic 3D model of the target object.

[0129] The specific implementation method of this step is the same as that of step S202 above. The implementation method of step S202 provided in any of the foregoing embodiments can be adopted. For details, please refer to the relevant content of step S202 in the foregoing embodiments. It will not be repeated here.

[0130] Step S503: Display the anthropomorphic 3D model on the interactive page, drive the anthropomorphic 3D model to complete the interactive behavior of broadcasting the broadcast content, and generate the broadcast animation.

[0131] The specific implementation method of displaying an anthropomorphic 3D model on the interactive page and driving the anthropomorphic 3D model to complete the interactive behavior of broadcasting the broadcast content in this step is consistent with the implementation method of step S203 above. The implementation method of step S203 provided in any of the foregoing embodiments can be adopted. For details, please refer to the relevant content of step S203 in the foregoing embodiments, which will not be repeated here.

[0132] In this embodiment, during the process of displaying an anthropomorphic 3D model on the interactive page and driving the anthropomorphic 3D model to complete the interactive behavior of broadcasting the broadcast content, the screen of the interactive behavior of the anthropomorphic 3D model of the target object on the interactive page is saved as video data by means of screenshot or screen recording, so as to obtain the broadcast animation.

[0133] Step S504: In response to the confirmation operation of the broadcast animation, provide the broadcast animation to the end device.

[0134] After generating the broadcast animation, the cloud server can display it on another interactive page for users to watch. Once the user confirms that the broadcast animation meets their needs, they can trigger a confirmation action to retrieve the broadcast animation from the cloud server.

[0135] In response to the confirmation action of the broadcast animation, the cloud server provides the broadcast animation to the end device. For example, the cloud server sends the broadcast animation to the end device; or, the cloud server provides a download link for the broadcast animation, allowing the end device to download and obtain the broadcast animation according to the download link.

[0136] This embodiment addresses a scenario where users customize anthropomorphic broadcast animations of objects via a terminal device. Based on this solution, the cloud server can automatically generate anthropomorphic 3D models of various objects and drive these models to perform corresponding interactive behaviors according to the user-input broadcast content. The resulting broadcast video is then provided to the terminal device. This eliminates the need for manually designing different animation actions for each object, resulting in high efficiency, short cycle time, and applicability to various types of objects. It also supports user-defined broadcast content. This embodiment's AI-driven anthropomorphic broadcast animation of target objects is dynamically generated based on the object's attributes and the input broadcast content. The resulting animation is not a monotonous repetition of a few pre-set actions but possesses high diversity and better matches the object's characteristics, enhancing the expressiveness of the anthropomorphized object. This makes the broadcast animation content based on anthropomorphic object images richer, more diverse, and more vivid, improving the efficiency and quality of anthropomorphic object broadcast animation generation.

[0137] Figure 6 A flowchart illustrating a real-time interactive method based on anthropomorphic object representation, provided as an exemplary embodiment of this application. Figure 6 As shown, the specific steps of this method are as follows:

[0138] Step S601: Provide a real-time interactive page.

[0139] In this embodiment, the cloud server provides a real-time interactive page, enabling human-computer interaction between the user (through the terminal device used) and the interactive system. The real-time interactive page can be an interactive page used in real-time interactive scenarios such as intelligent customer service or live streaming.

[0140] Step S602: During real-time interaction, in response to the interaction request with the target object, obtain the target object's attribute information, 3D model of the object, and interaction content information.

[0141] In different real-time interactive scenarios, the interaction request with the target object can be different. For example, in an intelligent customer service scenario, the interaction request could be that the user sends a link to the target object, asks a question related to the target object, sends an image of the target object, or sends a link to the target object, a question, and various information from the image. In a live streaming scenario, the interaction request could be initiated by the host / virtual host, who would then have a designated target object interact with the user using a human-like appearance. For example, the host / virtual host might issue a pre-set script, such as "Please welcome the XXX object," or "Look at this XXX in my hand," where "XXX" refers to the object's name or identification information. For instance, the host / virtual host might make a pre-set gesture towards the target object in the live streaming scenario, such as pointing at an object.

[0142] The interaction request with the target object can carry information about the target object, such as its name, code, category, and image. Based on this information, the cloud server can identify and determine the target object, and obtain its attribute information and 3D model.

[0143] In this embodiment, interactive content information refers to the content information output by the target object to the user based on the interactive request. Interactive content information is the response content information generated by the cloud server based on the input data in response to the user's input data. The user's input data may include at least one of the following: an image of the target object, or a description of the target object. For a detailed explanation of how to obtain the interactive content information of the target object based on the input data, please refer to the implementation method of step S201 in the aforementioned embodiment for obtaining interactive content information in a real-time interactive scenario; it will not be repeated here.

[0144] Step S603: Based on the attribute information of the target object, generate anthropomorphic 3D material suitable for the target object, and composite the anthropomorphic 3D material onto the object's 3D model to obtain the anthropomorphic 3D model of the target object.

[0145] The specific implementation method of this step is the same as that of step S202 above. It can adopt the implementation method of step S202 in the real-time interaction scenario in the aforementioned embodiment. For details, please refer to the relevant content in the aforementioned embodiment, which will not be repeated here.

[0146] Step S604: Display the anthropomorphic 3D model on the real-time interactive page, and drive the anthropomorphic 3D model to perform interactive behaviors that match the interactive content information.

[0147] In this embodiment, after acquiring the interactive content information and generating a humanoid 3D model of the target object, the humanoid 3D model is displayed on the real-time interactive page. The AI-driven engine drives the humanoid 3D model to perform interactive behaviors that match the interactive content information, so that the interactive behavior of the target object can adaptively match different content interactive information. The AI-driven output can automatically adapt to different objects and different styles of interactive content information of the same object, making the interactive behavior of the humanoid image of the object more diverse and humanoid, and with a higher degree of matching with the interactive content information, thereby improving the interaction quality based on the humanoid image of the object.

[0148] Figure 7 This is a schematic diagram of the structure of an interactive device provided in an exemplary embodiment of this application. The interactive device provided in this embodiment can execute the processing flow provided in the interactive method embodiment. Figure 7 As shown, the interactive device 70 includes: an information acquisition module 71, an anthropomorphic processing module 72, and a driving module 73.

[0149] The information acquisition module 71 is used to acquire the target object's attribute information, 3D model, and interactive content information in response to an interaction request for the target object.

[0150] The anthropomorphic processing module 72 is used to generate anthropomorphic 3D materials suitable for the target object based on the attribute information of the target object, and to synthesize the anthropomorphic 3D materials onto the object's 3D model to obtain the anthropomorphic 3D model of the target object.

[0151] The driver module 73 is used to display an anthropomorphic 3D model on the interactive page and drive the anthropomorphic 3D model to perform interactive behaviors that match the interactive content information.

[0152] In an optional embodiment, when implementing the acquisition of attribute information and 3D model of the target object in response to an interaction request with the target object, the information acquisition module 71 is further configured to:

[0153] In response to an interaction request with the target object, acquire an image of the target object; based on the image of the target object, identify the attribute information of the target object, and acquire a 3D model of the target object.

[0154] In an optional embodiment, when acquiring the 3D model of the target object, the information acquisition module 71 is further configured to:

[0155] Based on the image of the target object, the contour information and 3D geometric information of the target object are detected; based on the contour information and 3D geometric information of the target object, a 3D model of the target object is generated, thus obtaining the 3D model of the object.

[0156] In an optional embodiment, when implementing the acquisition of attribute information and 3D model of the target object in response to an interaction request with the target object, the information acquisition module 71 is further configured to:

[0157] In response to an interaction request with the target object, obtain the target object's description information; based on the target object's description information, retrieve the target object's attribute information and 3D model.

[0158] In an optional embodiment, the attribute information of the target object includes category and color. When generating anthropomorphic 3D materials suitable for the target object based on its attribute information, the anthropomorphic processing module 72 is further configured to:

[0159] Obtain basic anthropomorphic 3D material corresponding to the target object; adjust the configuration parameters of the basic anthropomorphic 3D material according to the category and color of the target object to obtain anthropomorphic 3D material suitable for the target object. The anthropomorphic 3D material includes: facial 3D model and / or limb 3D model.

[0160] In an optional embodiment, after adjusting the configuration parameters of the basic anthropomorphic 3D material according to the category and color of the target object to obtain anthropomorphic 3D material suitable for the target object, the anthropomorphic processing module 72 is further configured to:

[0161] In response to editing the configuration parameters of the anthropomorphic 3D material of the target object, update the anthropomorphic 3D material of the target object according to the edited configuration parameters.

[0162] In an optional embodiment, when compositing anthropomorphic 3D materials onto a 3D model of an object to obtain an anthropomorphic 3D model of the target object, the anthropomorphic processing module 72 is further configured to:

[0163] Based on the shape and outline information of the object's 3D model, determine the connection positions of the anthropomorphic 3D material on the object's 3D model; based on the connection positions, composite the anthropomorphic 3D material onto the corresponding connection positions to obtain the anthropomorphic 3D model of the target object.

[0164] In an optional embodiment, before compositing the anthropomorphic 3D material onto the corresponding connection position to obtain the anthropomorphic 3D model of the target object according to the connection position, the anthropomorphic processing module 72 is further configured to:

[0165] In response to the adjustment operation of the connection position of the anthropomorphic 3D material on the object's 3D model, update the connection position of the anthropomorphic 3D material on the object's 3D model.

[0166] In an optional embodiment, in response to an interaction request with a target object, when acquiring interaction content information of the target object, the information acquisition module 71 is further configured to:

[0167] In response to a first interaction request with a target object, the system acquires input dialogue content and multimodal knowledge data of the target object from at least one source.

[0168] When displaying an anthropomorphic 3D model on an interactive page and driving the anthropomorphic 3D model to perform interactive behaviors that match the interactive content information, the driving module 73 is also used for:

[0169] The dialogue model is invoked to generate response information for the dialogue content based on multimodal knowledge data of the target object from at least one source; an anthropomorphic 3D model of the target object is displayed on the first interactive page, and the anthropomorphic 3D model is driven to perform an action of broadcasting the response information based on the response information, wherein the action of broadcasting the response information includes at least one of facial expressions, mouth shapes, and body movements.

[0170] In an optional embodiment, in response to an interaction request with a target object, when acquiring interaction content information of the target object, the information acquisition module 71 is further configured to:

[0171] In response to a second interaction request with the target object, obtain the input broadcast content.

[0172] When displaying an anthropomorphic 3D model on an interactive page and driving the anthropomorphic 3D model to perform interactive behaviors that match the interactive content information, the driving module 73 is also used for:

[0173] The second interactive page displays an anthropomorphic 3D model of the target object, and drives the anthropomorphic 3D model to perform actions that broadcast the broadcast content according to the broadcast content. The actions that broadcast the broadcast content include at least one of facial expressions, mouth shapes, and body movements.

[0174] In an optional embodiment, in response to an interaction request with a target object, when acquiring interaction content information of the target object, the information acquisition module 71 is further configured to:

[0175] In response to a third interaction request with the target object, acquire the input audio data.

[0176] When displaying an anthropomorphic 3D model on an interactive page and driving the anthropomorphic 3D model to perform interactive behaviors that match the interactive content information, the driving module 73 is also used for:

[0177] The third interactive page displays an anthropomorphic 3D model of the target object and drives the anthropomorphic 3D model to perform dance movements that match the audio data. The dance movements include at least one of facial expressions, lip movements, and body movements.

[0178] The apparatus provided in this application embodiment can be specifically used to execute the interactive method provided in any of the above method embodiments, and the specific functions and technical effects achieved will not be elaborated here.

[0179] Figure 8 This is a schematic diagram of the structure of a cloud server provided in an embodiment of this application. Figure 8 As shown, the cloud server includes a memory 801 and a processor 802. The memory 801 stores computer-executed instructions and can be configured to store various other data to support operations on the cloud server. The processor 802 is communicatively connected to the memory 801 and executes the computer-executed instructions stored in the memory 801 to implement the technical solutions provided in any of the above method embodiments. Their specific functions and the technical effects they achieve are similar and will not be repeated here.

[0180] Optional, such as Figure 8 As shown, the cloud server also includes other components such as a firewall 803, a load balancer 804, a communication component 805, and a power supply component 806. Figure 8 The diagram only shows some components and does not mean that a cloud server includes only these components. Figure 8 The components shown.

[0181] This application also provides a computer-readable storage medium storing computer-executable instructions. When executed by a processor, the computer-executable instructions are used to implement the solution provided in any of the above method embodiments. The specific functions and technical effects to be achieved are not described here.

[0182] This application also provides a computer program product, comprising: a computer program stored in a readable storage medium; at least one processor of a cloud server can read the computer program from the readable storage medium; and the at least one processor executes the computer program to cause the cloud server to perform the solution provided in any of the above method embodiments. Specific functions and achievable technical effects are not detailed here. This application also provides a chip, comprising: a processing module and a communication interface; the processing module is capable of executing the technical solution of the cloud server in the aforementioned method embodiments. Optionally, the chip further includes a storage module (e.g., a memory), which stores instructions; the processing module executes the instructions stored in the storage module, and the execution of the instructions stored in the storage module causes the processing module to perform the technical solution provided in any of the aforementioned method embodiments.

[0183] The aforementioned storage can be object storage (OSS).

[0184] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0185] The aforementioned communication components are configured to facilitate wired or wireless communication between the device containing the communication components and other devices. The device containing the communication components can access wireless networks based on communication standards, such as mobile hotspots (WiFi), second-generation (2G), third-generation (3G), fourth-generation (4G) / Long Term Evolution (LTE), fifth-generation (5G), or combinations thereof. In one exemplary embodiment, the communication components receive broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication components also include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0186] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0187] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, read-only optical disc storage (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0188] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0189] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0190] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0191] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0192] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0193] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0194] It should be noted that the user information (including but not limited to user device information, user attribute information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0195] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The sequence numbers are merely used to distinguish different operations, and the sequence number itself does not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types. "Multiple" means two or more, unless otherwise explicitly specified.

[0196] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0197] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0198] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An interaction method, characterized in that, include: In response to an interaction request for a target object, the system acquires the target object's attribute information, 3D model, and interaction content information. The attribute information of the target object includes its category and color; Obtain the basic anthropomorphic 3D material corresponding to the target object; Based on the category and color of the target object, the configuration parameters of the basic anthropomorphic 3D material are adjusted to obtain anthropomorphic 3D material suitable for the target object. The anthropomorphic 3D material includes: facial 3D model and / or limb 3D model. Based on the shape and outline information of the object's 3D model, determine the connection position of the anthropomorphic 3D material on the object's 3D model; Based on the connection positions, the anthropomorphic 3D material is synthesized onto the corresponding connection positions to obtain the anthropomorphic 3D model of the target object; The anthropomorphic 3D model is displayed on the interactive page, and the anthropomorphic 3D model is driven to perform interactive behaviors that match the interactive content information.

2. The method according to claim 1, characterized in that, In response to an interaction request with a target object, the system acquires the target object's attribute information and 3D model, including: In response to an interaction request with the target object, acquire an image of the target object; Based on the image of the target object, identify the attribute information of the target object and obtain the 3D model of the target object.

3. The method according to claim 2, characterized in that, The process of obtaining the 3D model of the target object includes: Based on the image of the target object, the contour information and 3D geometric information of the target object are detected; Based on the contour information and 3D geometric information of the target object, a 3D model of the target object is generated, thus obtaining the 3D model of the object.

4. The method according to claim 1, characterized in that, The step of responding to an interaction request with a target object and obtaining the target object's attribute information and 3D model includes: In response to an interaction request with the target object, obtain the description information of the target object; Based on the description information of the target object, retrieve the attribute information and 3D model of the target object.

5. The method according to claim 1, characterized in that, After adjusting the configuration parameters of the basic anthropomorphic 3D material according to the category and color of the target object to obtain anthropomorphic 3D material suitable for the target object, the method further includes: In response to the editing operation of the configuration parameters of the anthropomorphic 3D material of the target object, the anthropomorphic 3D material of the target object is updated according to the edited configuration parameters.

6. The method according to claim 1, characterized in that, Before compositing the anthropomorphic 3D material onto the corresponding connection position to obtain the anthropomorphic 3D model of the target object according to the connection position, the method further includes: In response to the operation of adjusting the connection position of the anthropomorphic 3D material on the object 3D model, update the connection position of the anthropomorphic 3D material on the object 3D model.

7. The method according to any one of claims 1-4, characterized in that, In response to an interaction request with a target object, the interaction content information of the target object is obtained, including: In response to a first interaction request with a target object, the system acquires input dialogue content and multimodal knowledge data of the target object from at least one source. Displaying the anthropomorphic 3D model on the interactive page and driving the anthropomorphic 3D model to perform interactive behaviors that match the interactive content information includes: The dialogue model is invoked to generate response information for the dialogue content based on multimodal knowledge data of the target object from at least one source. The first interactive page displays an anthropomorphic 3D model of the target object, and drives the anthropomorphic 3D model to perform an action of broadcasting the response information based on the response information, wherein the action of broadcasting the response information includes at least one of facial expressions, mouth shapes, and body movements.

8. The method according to any one of claims 1-4, characterized in that, In response to an interaction request with a target object, the interaction content information of the target object is obtained, including: In response to a second interaction request with the target object, obtain the input broadcast content; Displaying the anthropomorphic 3D model on the interactive page and driving the anthropomorphic 3D model to perform interactive behaviors that match the interactive content information includes: The second interactive page displays an anthropomorphic 3D model of the target object, and drives the anthropomorphic 3D model to perform actions to broadcast the broadcast content according to the broadcast content. The actions to broadcast the broadcast content include at least one of facial expressions, mouth shapes, and body movements.

9. The method according to any one of claims 1-4, characterized in that, In response to an interaction request with a target object, the interaction content information of the target object is obtained, including: In response to a third interaction request with the target object, acquire the input audio data; Displaying the anthropomorphic 3D model on the interactive page and driving the anthropomorphic 3D model to perform interactive behaviors that match the interactive content information includes: The third interactive page displays an anthropomorphic 3D model of the target object and drives the anthropomorphic 3D model to perform dance movements that match the audio data, wherein the dance movements include at least one of facial expressions, lip movements, and body movements.

10. An interaction method, characterized in that, include: In response to the end device's request for anthropomorphism of the target object, the attribute information, 3D model of the object, and broadcast content of the target object are obtained; The attribute information of the target object includes its category and color; Obtain the basic anthropomorphic 3D material corresponding to the target object; Based on the category and color of the target object, the configuration parameters of the basic anthropomorphic 3D material are adjusted to obtain anthropomorphic 3D material suitable for the target object. The anthropomorphic 3D material includes: facial 3D model and / or limb 3D model. Based on the shape and outline information of the object's 3D model, determine the connection position of the anthropomorphic 3D material on the object's 3D model; Based on the connection positions, the anthropomorphic 3D material is synthesized onto the corresponding connection positions to obtain the anthropomorphic 3D model of the target object; The anthropomorphic 3D model is displayed on the interactive page, and the anthropomorphic 3D model is driven to complete the interactive behavior of broadcasting the broadcast content and generate the broadcast animation. In response to the confirmation operation of the broadcast animation, the broadcast animation is provided to the terminal device.

11. An interaction method, characterized in that, include: During real-time interaction, in response to an interaction request with the target object, the system acquires the target object's attribute information, 3D model, and interaction content information. The attribute information of the target object includes its category and color; Obtain the basic anthropomorphic 3D material corresponding to the target object; Based on the category and color of the target object, the configuration parameters of the basic anthropomorphic 3D material are adjusted to obtain anthropomorphic 3D material suitable for the target object. The anthropomorphic 3D material includes: facial 3D model and / or limb 3D model. Based on the shape and outline information of the object's 3D model, determine the connection position of the anthropomorphic 3D material on the object's 3D model; Based on the connection positions, the anthropomorphic 3D material is synthesized onto the corresponding connection positions to obtain the anthropomorphic 3D model of the target object; The anthropomorphic 3D model is displayed on the real-time interactive page, and the anthropomorphic 3D model is driven to perform interactive behaviors that match the interactive content information.

12. A cloud server, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-11.

Images