Three-dimensional object generation method, device and equipment based on large model and intelligent agent

Abstract

The invention provides a three-dimensional object generation method and device based on a large model, equipment and an intelligent agent, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models and the like, and can be applied to scenes of virtual reality, 3D clothing generation and the like. The three-dimensional object generation method based on the large model comprises the following steps: performing morphological transformation on an initial graph for a target object to obtain an object morphological graph representing that the object has a specified morphological; performing feature fusion on the object texture feature and the object form graph of the initial graph by using a visual large model to obtain a basic target object graph; performing multi-view conversion on the basic target object graph to obtain a plurality of expanded target object graphs related to a plurality of preset views; and generating a three-dimensional object based on the plurality of extended target object maps.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to technical fields such as computer vision, deep learning, and large models, and can be applied to scenes such as virtual reality and 3D clothing generation. Background Art

[0002] With the development of artificial intelligence technology, in the fields of Internet e-commerce, animation games, video production, etc., it is possible to interact with users through three-dimensional objects. For example, in the field of Internet e-commerce, three-dimensional clothing models can be used for display and fitting to improve the display effect of clothing. Summary of the invention

[0003] The present disclosure provides a method, apparatus, device, medium, program product and intelligent agent for generating a three-dimensional object based on a large model.

[0004] According to one aspect of the present disclosure, a method for generating a three-dimensional object based on a large model is provided, comprising: performing morphological transformation on an initial image for a target object to obtain an object morphological map representing an object having a specified morphology; using a visual large model to perform feature fusion on object texture features of the initial image and the object morphological map to obtain a basic target object map; performing multi-perspective transformation on the basic target object map to obtain a plurality of extended target object maps associated with a plurality of preset perspectives; and generating a three-dimensional object based on the plurality of extended target object maps.

[0005] According to another aspect of the present disclosure, a three-dimensional object generation device based on a large model is provided, including: a morphological transformation module, used to perform morphological transformation on an initial image for a target object to obtain an object morphological map representing an object with a specified morphology; a fusion module, used to use the visual large model to perform feature fusion on the object texture features of the initial image and the object morphological map to obtain a basic target object map; a perspective transformation module, used to perform multi-perspective transformation on the basic target object map to obtain multiple extended target object maps related to multiple preset perspectives; and a generation module, used to generate a three-dimensional object based on the multiple extended target object maps.

[0006] According to another aspect of the present disclosure, there is provided an artificial intelligence agent, comprising: an input module for receiving input information; a processing module for determining a target task based on the input information received by the input module, determining a visual big model based on the target task, and obtaining output information by calling the visual big model to execute the above-mentioned method; and an output module for outputting the output information obtained by the processing module.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method as described above.

[0009] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method described above when executed by a processor.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 An exemplary system architecture to which a method and device for generating a three-dimensional object based on a large model can be applied according to an embodiment of the present disclosure is schematically shown;

[0013] Figure 2 A flowchart of a method for generating a three-dimensional object based on a large model according to an embodiment of the present disclosure is schematically shown;

[0014] Figure 3 The schematic diagram of the method for generating a three-dimensional object based on a large model according to an embodiment of the present disclosure is exemplarily shown;

[0015] Figure 4 A schematic diagram of a visual macro model according to an embodiment of the present disclosure is exemplarily shown;

[0016] Figure 5 A schematic diagram of a visual macro model according to another embodiment of the present disclosure is exemplarily shown;

[0017] Figure 6 A block diagram of a large model-based three-dimensional object generation device according to an embodiment of the present disclosure is schematically shown;

[0018] Figure 7 A block diagram schematically shows a structure of an artificial intelligence agent according to an embodiment of the present disclosure; and

[0019] Figure 8 A block diagram of an electronic device suitable for implementing a large model-based three-dimensional object generation method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0020] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0021] In the technical solution disclosed in the present invention, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations, necessary confidentiality measures are taken, and do not violate public order and good morals.

[0022] With the development of artificial intelligence technology, it is possible to interact with users by constructing three-dimensional objects. However, the inventors have found that in the related art, images from multiple perspectives are required to construct three-dimensional objects, and the generation conditions of three-dimensional objects are relatively harsh.

[0023] In this regard, the embodiments of the present disclosure provide a method, device, equipment, medium, program product and intelligent agent for generating a three-dimensional object based on a large model. A method for generating a three-dimensional object based on a large model includes: performing morphological transformation on an initial image for a target object to obtain an object morphological image representing an object with a specified morphology; using a visual large model to perform feature fusion on the object texture features of the initial image and the object morphological image to obtain a basic target object image; performing multi-view transformation on the basic target object image to obtain multiple extended target object images related to multiple preset viewpoints; and generating a three-dimensional object based on the multiple extended target object images.

[0024] According to the embodiments of the present disclosure, by performing morphological transformation on a single initial image without specific shooting requirements, the object morphological image representing the specified morphology of the target object can realize standardized and complete display of the morphology of the target object, avoiding the diversification of the initial morphology of the target object in the initial image, which leads to morphological deformity of the subsequent three-dimensional object. By using the visual large model to process the object texture features and the object morphological image to perform feature fusion, the basic target object image can accurately display the object texture of the target object under the specified morphological conditions, and then by performing multi-perspective transformation on the basic target object image to obtain multiple extended target objects related to multiple preset perspectives, it is possible to accurately display the texture and morphology of the target object from multiple perspectives, so that the three-dimensional object generated based on the multiple extended target object images can more accurately represent the target object in the three-dimensional space, increase the amount of data required for the construction of the target object and ensure the matching degree between the three-dimensional object and the target object.

[0025] According to the embodiments of the present disclosure, a large model may be a model having a large-scale parameter structure, and the parameter magnitude is generally above 10 million or 100 million, and may reach 10 billion or 10 billion. In terms of the network structure of the large model, for example, a network structure such as UFO (Unified Feature Optimization) may be adopted.

[0026] Figure 1 An exemplary system architecture to which a large model-based three-dimensional object generation method and apparatus can be applied according to an embodiment of the present disclosure is schematically shown.

[0027] It should be noted that Figure 1 The examples shown are only examples of system architectures to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios. For example, in another embodiment, the exemplary system architecture to which the large-model-based three-dimensional object generation and device can be applied may include a terminal device, but the terminal device may implement the large-model-based three-dimensional object generation method and device provided in the embodiments of the present disclosure without interacting with the server.

[0028] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0029] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software, etc. (only as examples).

[0030] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0031] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for the content browsed by the user using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device.

[0032] The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server can also be a server of a distributed system, or a server combined with blockchain.

[0033] It should be noted that the three-dimensional object generation method based on the large model provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the three-dimensional object generation device based on the large model provided in the embodiment of the present disclosure can generally be set in the server 105. The three-dimensional object generation method based on the large model provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Correspondingly, the three-dimensional object generation device based on the large model provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0034] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0035] Figure 2 The flowchart of the method for generating a three-dimensional object based on a large model according to an embodiment of the present disclosure is schematically shown.

[0036] like Figure 2 As shown, the method 200 includes operations S210 to S240.

[0037] In operation S210 , a morphology transformation is performed on an initial graph for a target object to obtain a morphology graph representing an object having a specified morphology.

[0038] In operation S220, the visual macro model is used to perform feature fusion on the object texture features of the initial image and the object morphology image to obtain a basic target object image.

[0039] In operation S230, a multi-view conversion is performed on the basic target object graph to obtain a plurality of extended target object graphs associated with a plurality of preset viewpoints.

[0040] In operation S240, a three-dimensional object is generated based on the plurality of extended target object graphs.

[0041] According to an embodiment of the present disclosure, the target object may be any type of object in the initial image, such as clothing, people, animals, buildings, tools, vehicles, etc. The target object may represent a real object such as a person or an animal, or the target object may also represent a virtual object such as an animated character.

[0042] According to an embodiment of the present disclosure, the initial image for the target object may be a single in-the-wild image without specific shooting requirements. Here, "in-the-wild images" may be understood as images taken in an uncontrolled, natural environment, and the images are usually diverse and complex, including changes in different shooting angles, lighting conditions, image quality, etc. The morphology of the target object in the initial image may be diverse, complex, and non-standardized.

[0043] According to an embodiment of the present disclosure, the specified form can fully and standardizedly display the form of the target object. The specified form may include, for example, a standard view form of the target object in a frontal view direction.

[0044] According to an embodiment of the present disclosure, the initial graph for the target object can be morphologically transformed to obtain an object morphological graph representing a specified morphology. It is understandable that the initial morphology of the target object in the initial graph may be diverse and non-standardized. By performing morphological transformation on the initial graph, the morphology of the target object in the initial graph can be converted into a specified morphology, so that the object morphological graph representing the specified morphology of the target object can realize a standardized and complete display of the morphology of the target object, avoiding the morphological deformity of the subsequent three-dimensional object caused by the diversity of the initial morphology of the target object in the initial graph.

[0045] According to the embodiments of the present disclosure, the visual macro model can be used to fuse the object texture features of the initial image with the object morphology image to obtain a basic target object image in which the target object has a specified shape and represents the object texture features.

[0046] Exemplarily, the visual macro model can be used to process the initial image to obtain object texture features that characterize the target object. The visual macro model can be used to perform feature fusion of the object texture features with the object morphology image to obtain a basic target object image.

[0047] According to the embodiments of the present disclosure, since the object texture features are determined based on the initial image, the object texture features can relatively completely retain the low-level features such as the edge contour and texture structure of the target object, which is beneficial to improving the accuracy and reliability of the basic target object image. Since the object morphology image represents the specified morphology of the target object, the morphology standardization of the basic target object image can be ensured, which is beneficial to the subsequent generation of three-dimensional objects. By using the visual large model to fuse the object texture features and the object morphology image, a basic target object image in which the target object is in the specified morphology and represents the object texture features can be obtained, so that the basic target object image can accurately display the texture features of the target object under the specified morphological conditions.

[0048] According to an embodiment of the present disclosure, a vision big model (Vision Transformer) may be a visual processing model with large-scale parameters and computing power, and the vision big model may extract useful features and information from visual data such as images and videos. The vision big model may be applied to technical fields such as image recognition, target detection, semantic segmentation, and image generation. The vision big model may be a model with a model structure having large-scale parameters, and the parameter magnitude is generally above the tens of millions or hundreds of millions, and may reach the billions or tens of billions. The vision big model may include multimodal big models with visual understanding capabilities such as the VideoChat model and the Video-LlaMA model. The vision big model involved in the embodiment of the present disclosure may be a general big model, or it may be an expert big model obtained after fine-tuning (Fine Tune) based on sample object description information and sample images, and the embodiment of the present disclosure does not limit this. The vision big model can be used to process information of any modality such as images, text, video, and audio.

[0049] According to an embodiment of the present disclosure, a basic target object graph can be converted into multiple perspectives to obtain multiple extended target object graphs related to multiple preset views. On this basis, a three-dimensional object can be generated based on the multiple extended target object graphs. The three-dimensional object can represent the three-dimensional structure of the target object.

[0050] According to an embodiment of the present disclosure, multiple preset views can represent multiple viewing angles, and multiple extended target object graphs can represent target objects under multiple preset views. Multiple extended target object graphs can accurately display the texture and morphology of target objects from multiple viewing angles, so that the three-dimensional objects generated based on the multiple extended target object graphs can more accurately represent the target objects in three-dimensional space, thereby increasing the amount of data required for target object construction and ensuring the matching degree between the three-dimensional object and the target object.

[0051] It should be understood that the target object can be represented based on the two-dimensional image regions in the multiple extended target object graphs. The three-dimensional object can be represented based on the three-dimensional object elements (eg, three-dimensional space points, three-dimensional grid elements) in the three-dimensional space.

[0052] According to the embodiments of the present disclosure, by performing morphological transformation on a single initial image without specific shooting requirements, the object morphological image representing the specified morphology of the target object can realize standardized and complete display of the morphology of the target object, avoiding the diversification of the initial morphology of the target object in the initial image, which leads to morphological deformity of the subsequent three-dimensional object. By using the visual large model to process the object texture features and the object morphological image to perform feature fusion, the basic target object image can accurately display the object texture of the target object under the specified morphological conditions, and then by performing multi-perspective transformation on the basic target object image to obtain multiple extended target objects related to multiple preset perspectives, it is possible to accurately display the texture and morphology of the target object from multiple perspectives, so that the three-dimensional object generated based on the multiple extended target object images can more accurately represent the target object in the three-dimensional space, increase the amount of data required for the construction of the target object and ensure the matching degree between the three-dimensional object and the target object.

[0053] It should be noted that the method provided according to the embodiment of the present disclosure can be applied to multiple application scenarios such as e-commerce platforms, virtual reality, and clothing design. The three-dimensional object generation method based on a large model provided by the embodiment of the present disclosure does not limit the application scenario.

[0054] In one example, a single clothing picture uploaded by a user on an e-commerce platform can be processed based on the large model-based three-dimensional object generation method provided in an embodiment of the present disclosure to generate a 360-degree rotatable three-dimensional clothing model, thereby enhancing the user's shopping experience.

[0055] In one example, a single character image, prop image, or clothing image can be processed based on the large model-based three-dimensional object generation method provided in an embodiment of the present disclosure, so as to generate three-dimensional objects such as virtual three-dimensional character models, virtual three-dimensional prop models, and virtual three-dimensional clothing models in a virtual reality scene, so as to provide users with an immersive experience in virtual reality or augmented reality application scenarios.

[0056] In one example, a two-dimensional clothing design drawing is processed using a three-dimensional object generation method based on a large model provided by an embodiment of the present disclosure to obtain a three-dimensional clothing object model, thereby simplifying the production process of clothing design and production and improving clothing production efficiency.

[0057] According to an embodiment of the present disclosure, the visual big model includes an isomorphic image generative big model and a texture generative big model. The image generative big model and the texture generative big model can be understood as isomorphic models. The image generative big model and the texture generative big model can be, for example, generative big models with the same network structure, and the model parameters of the image generative big model and the texture generative big model can be different.

[0058] According to an embodiment of the present disclosure, the object texture features of the initial image are subjected to feature fusion with the object morphology image by using a visual big model to obtain a basic target object image, including: using an image generation big model to perform multi-level feature fusion on the object morphology image and multi-level object texture features to obtain a target basic object image.

[0059] According to an embodiment of the present disclosure, multi-level object texture features are determined by performing multi-level feature extraction on an initial image using a texture generative large model.

[0060] According to one embodiment of the present disclosure, the visual big model may include an image generative big model and a texture generative big model with the same or similar network structure. Exemplarily, the image generative big model and the texture generative big model can be constructed based on a generative model algorithm. For example, it can be constructed based on any type of generative model algorithm such as a GAN (Generative Adversaria Networks) model, a VAE (Variational auto-encoder) model, a Flow-based Models, a Diffusion Models, etc. However, it is not limited to this, and a texture generation network can also be constructed based on other types of deep learning algorithms. For example, a texture generation network can be constructed based on a convolutional neural network algorithm.

[0061] According to the embodiments of the present disclosure, a texture generative large model can be used to perform multi-level feature extraction on the initial image to obtain multi-level object texture features. By using a texture generative large model to perform multi-level feature extraction on the initial image, the texture features of the target object in the initial image can be captured and extracted from different levels, and multi-level object texture features of different granularities can be obtained, which can help the target basic object image to accurately characterize the texture features of the target object.

[0062] For example, the initial image may be subjected to feature extraction based on an image feature extraction algorithm to obtain multi-level object texture features. The image feature extraction algorithm may include any type of deep learning algorithm such as a convolutional neural network algorithm, and the embodiments of the present disclosure do not limit the specific type of the image feature extraction algorithm.

[0063] According to the embodiments of the present disclosure, based on the attention mechanism, an image generation model can be used to perform multi-level feature fusion on the object morphology map and the multi-level object texture features to obtain the target basic object map.

[0064] According to one embodiment of the present disclosure, the object morphology map and the multi-level object texture features can be processed based on the cross-attention network algorithm to fully integrate the texture features of the target object represented by the multi-level object texture features with the specified morphology of the target object represented by the object morphology map.

[0065] According to the embodiments of the present disclosure, by performing multi-level feature fusion of the object morphology map and the multi-level object texture features, the correspondence between the object texture features and the object morphology map can be controlled at different levels, so that the texture features of the target object can be more accurately and fully integrated into the image pixels of the object morphology map, so as to generate a basic target object map that can accurately display the texture features of the target object under specified morphological conditions. In this way, the basic target object map can more accurately represent the texture semantic attributes of the target object in the initial image, and the texture information of the basic target object map can match the texture information of the target object in the initial image, thereby improving the characterization accuracy of the basic target object map in representing the target object.

[0066] Figure 3 The schematic diagram of the method for generating a three-dimensional object based on a large model according to an embodiment of the present disclosure is exemplified.

[0067] As an example, Figure 3 As shown, the target object may be clothing, and the initial image 302 for the target object may be an image of a person wearing clothing. Alternatively, the initial image 302 may be an image segmented from the original image 301 and containing only the person wearing clothing.

[0068] As an example, Figure 3 As shown, the specified form may be a standard front view of a T-shirt, for example, a standard front view of a T-shirt. The object form image 303 may be a mask image of a front view view of a T-shirt.

[0069] As an example, Figure 3 As shown, the visual model N1 may include an image generation model N11 and a texture generation model N12 with the same network structure. The image generation model N11 and the texture generation model N12 may be selected as diffusion models with the same network structure, and each network layer of the image generation model N11 may correspond to each network layer of the texture generation model N12 one by one, and the model parameters of the image generation model N11 and the texture generation model N12 may be different.

[0070] As an example, the texture generative model N12 can be used to perform multi-level feature extraction on the initial image to determine the multi-level object texture features. The object morphology image 303 and the multi-level object texture features are multi-level feature fused using the image generative model N11 to obtain the target basic object image 304.

[0071] In one example, the texture generative model and the image generative model can be constructed based on the attention network algorithm, and both the texture generative model and the image generative model can include multiple attention network layers. Using the image generative model to perform multi-level feature fusion of the object morphology map and the multi-level object texture features can include using the current attention network layer of the image generative model to perform attention fusion of the feature vector output by the previous attention network layer with the object texture features output by the attention network layer corresponding to the texture generative model, and obtain the feature vector output by the current attention network layer, until the image is reconstructed based on the feature vector output by the last attention network layer of the image generative model to obtain the target basic object map.

[0072] As an example, Figure 3 As shown, the basic target object graph 304 may be, for example, a T-shirt pattern graph with texture information of the target object in the initial graph.

[0073] As an example, Figure 3 As shown, multiple extended target object graphs related to multiple preset perspectives can respectively represent the target objects under the front view perspective, the rear view perspective, the left view perspective, and the right view perspective. The basic target object graph 304 can represent the target object under the front view perspective. The basic target object graph 304 can be processed by the diffusion model N2, and the front view perspective, the rear view perspective, the left view perspective, and the right view perspective are used as multi-view image generation conditions. The diffusion model N2 can output multiple extended target object graphs 305. On this basis, a three-dimensional model can be reconstructed based on the extended target object graphs 305 corresponding to the multiple preset perspectives to generate a three-dimensional object 306 that matches the target object.

[0074] According to the method provided by the embodiments of the present disclosure, spatial consistency supervision is performed by extending the target object graph with multiple preset perspectives that intersect, so as to ensure that the display effect of the generated three-dimensional object at different perspectives in the three-dimensional space remains consistent with the preset three-dimensional space, thereby avoiding strange deformations or discontinuous surfaces of the three-dimensional object.

[0075] According to an embodiment of the present disclosure, the image generative large model includes N levels of feature fusion layers, and the texture generative large model includes N levels of texture feature extraction layers; wherein, using the image generative large model to perform multi-level feature fusion on the object morphology map and the multi-level object texture features to obtain the target basic object map includes: using the n-th level feature fusion layer to process the n-th level object texture features and the n-1-th level intermediate fusion features to obtain the n-th level intermediate fusion features, and the 1st level intermediate fusion features are determined by using the 1st level feature fusion layer to process the 1st level object texture features and the object morphology map; and in the case of N=n, determining the basic target object map based on the N-th level intermediate fusion features, wherein N≥n>1, and N and n are integers.

[0076] Figure 4 The schematic diagram of a visual big model according to an embodiment of the present disclosure is exemplarily shown.

[0077] According to one embodiment of the present disclosure, the image generation model may include 4 levels of feature fusion layers, and the texture generation model may include 4 levels of texture feature extraction layers.

[0078] Exemplarily, using the image generation model to perform multi-level feature fusion of the object morphology map and the multi-level object texture features to obtain the target basic object map may include the following operations:

[0079] For example, the first level texture feature extraction layer N411 may be used to process the initial image 401 to determine the first level object texture features 4011. The first level feature fusion layer N421 may be used to process the first level object texture features 4011 and the object morphology image 402 to determine the first level intermediate fusion features 4021.

[0080] For example, when n=2, the first-level object texture feature 4011 can be processed by the second-level texture feature extraction layer N421 to determine the second-level object texture feature 4012. The second-level object texture feature 4012 and the first-level intermediate fusion feature 4021 can be processed by the second-level feature fusion layer N422 to obtain the second-level intermediate fusion feature 4022.

[0081] For example, when n=3, the third-level texture feature extraction layer N413 can be used to process the second-level object texture feature 4012 to determine the third-level object texture feature 4013. The third-level feature fusion layer N423 can be used to process the third-level object texture feature 4013 and the second-level intermediate fusion feature 4022 to obtain the third-level intermediate fusion feature 4023.

[0082] For example, when n=4, the 4th level texture feature extraction layer N414 can be used to process the 3rd level object texture feature 4013 to determine the 4th level object texture feature 4014. The 4th level feature fusion layer N424 can be used to process the 4th level object texture feature 4014 and the 3rd level intermediate fusion feature 4023 to obtain the 4th level intermediate fusion feature 4024. The basic target object graph 403 can be determined based on the 4th level intermediate fusion feature 4024.

[0083] According to one embodiment of the present disclosure, the first level texture feature extraction layer and the second level texture feature extraction layer can be implemented as encoders of a diffusion model, and the third level texture feature extraction layer and the fourth level texture feature extraction layer can be implemented as decoders of a diffusion model.

[0084] According to one embodiment of the present disclosure, the first-level feature fusion layer and the second-level feature fusion layer can be implemented as an encoder of a diffusion model, and the third-level feature fusion layer and the fourth-level feature fusion layer can be implemented as a decoder of the diffusion model.

[0085] According to an embodiment of the present disclosure, using the n-th level feature fusion layer to process the n-th level object texture features and the n-1-th level intermediate fusion features includes: using the n-th level feature fusion layer to process the n-th level object texture features, the n-1-th level intermediate fusion features and the initial image semantic features, wherein the initial image semantic features are determined by extracting image semantic features from the initial image.

[0086] Figure 5 A schematic diagram of a visual big model according to another embodiment of the present disclosure is exemplarily shown.

[0087] According to one embodiment of the present disclosure, the image generation model may include 4 levels of feature fusion layers, and the texture generation model may include 4 levels of texture feature extraction layers.

[0088] According to an embodiment of the present disclosure, the initial image can be subjected to image semantic feature extraction using a large model to determine the initial image semantic features. For example, the initial image can be processed using a multimodal large model to determine the initial image semantic features that characterize the semantic attributes of the target object. Among them, the multimodal large model can be used for joint learning of image modality and text modality to enable the initial image semantic features to characterize the deep-level semantic features of the target object. The multimodal large model may, for example, include a CLIP model (Contrastive Language-Image Pre-training). In the process of generating three-dimensional objects, the high-level image semantic features obtained by processing the initial image using the CLIP model can be used to fuse low-level object texture features and object morphological features to ensure that the texture, pattern, and color of the three-dimensional object are truly reflected from all perspectives.

[0089] Exemplarily, using the image generation model to perform multi-level feature fusion of the object morphology map and the multi-level object texture features to obtain the target basic object map may include the following operations:

[0090] For example, the first level texture feature extraction layer N511 may be used to process the initial image 501 to determine the first level object texture features 5011. The first level feature fusion layer N521 may be used to process the first level object texture features 5011, the object morphology image 502 and the initial image semantic features 503 to determine the first level intermediate fusion features 5021.

[0091] For example, when n=2, the first-level object texture feature 5011 can be processed by the second-level texture feature extraction layer N521 to determine the second-level object texture feature 5012. The second-level object texture feature 5012, the first-level intermediate fusion feature 5021 and the initial image semantic feature 503 can be processed by the second-level feature fusion layer N522 to obtain the second-level intermediate fusion feature 5022.

[0092] For example, when n=3, the third-level texture feature extraction layer N513 can be used to process the second-level object texture feature 5012 to determine the third-level object texture feature 5013. The third-level feature fusion layer N523 can be used to process the third-level object texture feature 5013, the second-level intermediate fusion feature 5022 and the initial image semantic feature 503 to obtain the third-level intermediate fusion feature 5023.

[0093] For example, when n=4, the fourth-level texture feature extraction layer N514 may be used to process the third-level object texture feature 5013 to determine the fourth-level object texture feature 5014.

[0094] The 4th level feature fusion layer N524 may be used to process the 4th level object texture feature 5014, the 3rd level intermediate fusion feature 5023 and the initial image semantic feature 503 to obtain the 4th level intermediate fusion feature 5024. The basic target object graph 504 may be determined based on the 4th level intermediate fusion feature 5024.

[0095] According to the embodiments of the present disclosure, by fusing high-level initial image semantic features with low-level object texture features and intermediate fusion features, the intermediate fusion features of each level can be effectively enriched, thereby ensuring that the basic target object graph can more accurately represent the target object, thereby improving the accuracy and representation precision of the basic target object graph.

[0096] According to an embodiment of the present disclosure, performing morphological transformation on an initial image for a target object to obtain a morphological image representing an object having a specified morphology includes: masking the initial image to obtain an initial morphological mask image representing the initial morphology of the target object; and performing morphological transformation on the initial morphological mask image to obtain an object morphological image.

[0097] According to an embodiment of the present disclosure, the initial image shows the initial form of the target object. The initial image can be masked to extract the corresponding area of ​​the target object in the initial image, and obtain an initial form mask image representing the initial form of the target object.

[0098] According to the embodiments of the present disclosure, the initial morphological mask map can be morphologically transformed to obtain an object morphological map. By morphologically transforming the initial morphological mask map, the morphology of the target object in the initial morphological mask map can be transformed into a specified morphology, so that the object morphological map can fully display the morphology of the target object in a standardized manner, avoiding the morphological deformity of the subsequent three-dimensional object caused by the diversity of the initial morphology of the target object in the initial map.

[0099] In one example, the object morphology map can be a mask image that represents the specified morphology of the target object, so that the mask that represents the specified morphology of the target object can be used to control the area where the object morphology map and the object texture features are fused to correspond to the specified morphology of the target object, thereby enabling the basic target object map to accurately display the texture features of the target object under the specified morphological conditions, which is beneficial to improving the characterization accuracy of the basic target object map.

[0100] According to an embodiment of the present disclosure, generating a three-dimensional object based on multiple extended target object graphs includes: performing feature extraction on the multiple extended target object graphs to obtain extended object shape features based on undirected distance field representation, and extended object texture features based on texture radiation field representation; constructing a three-dimensional object based on the extended object shape features and extended object texture features corresponding to multiple preset viewing angles.

[0101] According to an embodiment of the present disclosure, a field is a quantity (scalar) defined for at least one of all (continuous) space and time coordinates, such as an electromagnetic field, a gravitational field, etc. Field is a continuous concept that maps a high-dimensional vector to a scalar. Among them, the above-mentioned undirected distance field is a neural field, and a neural field is a field that is fully or partially parameterized by a neural network.

[0102] In the field of vision, the neural field can be considered as a process that uses spatial coordinates and information of other dimensions (time, camera pose, etc.) as input, simulates the target function through a multilayer perceptron (MLP) network, and generates a target scalar (such as color, depth, etc.).

[0103] According to an embodiment of the present disclosure, an unsigned distance field (UDF) is an implicit expression of the surface of a target object in a three-dimensional space, which can record the distance from each point in the space to the nearest surface, but does not contain sign information.

[0104] Specifically, the undirected distance field of the target object can be represented by a distance field function. By inputting the coordinates of any point in the three-dimensional space into the distance field function, the distance field function can calculate the orthogonal distance between the point and the boundary set of the target object in the three-dimensional space (or the closest distance between the point and the surface of the target object). Since the undirected distance field only records the closest distance from a point in the space to the surface of the object and does not record the sign, there is no need to divide the inside and outside of the space, so open boundary objects can be accurately modeled.

[0105] According to an embodiment of the present disclosure, texture radiance fields are applied to texture representation in the fields of computer graphics and image generation, and textures with subtle geometric details can be captured and synthesized from multi-view images. For example, an implicit color radiance field (Color-MLP) can be used to express the texture information of the target object, and an undirected distance field and a texture radiance field can be expressed using a fully connected neural network.

[0106] According to an embodiment of the present disclosure, the extended object shape feature can characterize the three-dimensional morphology and structural features of the target object, for example, the shape and size of the target object in a specified morphology. The extended object texture feature can characterize the texture information of the surface of the target object.

[0107] According to an embodiment of the present disclosure, multiple extended target object graphs are related to multiple preset viewing angles of the target object, and feature extraction can be performed on the multiple extended target object graphs to obtain extended object shape features and extended object texture features corresponding to the multiple preset viewing angles.

[0108] As an example, shape feature extraction can be performed based on a first machine learning network, the machine learning network has a number of network weight parameters, its input is a plurality of extended target object graphs, and its output is an undirected distance field. As an example, texture feature extraction can be performed based on a second machine learning network, its input is a plurality of extended target object graphs, and its output is a texture radiation field. Exemplarily, the first machine learning network can be a multi-layer perceptron network for outputting an undirected distance field, and the second machine learning network can be a multi-layer perceptron network for outputting a texture radiation field.

[0109] According to an embodiment of the present disclosure, volume rendering technology can be used to generate N reconstructed images of a target object based on extended object shape features and extended object texture features. The reconstructed image can be a two-dimensional image of the target object observed at a specified viewing angle. Exemplarily, shape information and texture information can be extracted from an undirected distance field and a texture radiation field respectively to synthesize N reconstructed images.

[0110] According to an embodiment of the present disclosure, the extended object shape features and the extended object texture features can be iteratively optimized based on the differences between N reconstructed images of the target object and multiple extended target object graphs to improve the characterization accuracy and the modeling effect of the three-dimensional object. For example, multiple extended target object graphs can be used as references, and the extended object shape features and the extended object texture features can be iteratively optimized based on the differences between the N reconstructed images and the multiple extended target object graphs using a machine learning model, so that the undirected distance field and the texture radiation field gradually approach the actual geometric structure and texture information of the target object to ensure the model accuracy of the three-dimensional object.

[0111] According to an embodiment of the present disclosure, when the extended object shape features and the extended object texture features are optimized, a three-dimensional object can be constructed based on the extended object shape features and the extended object texture features corresponding to a plurality of preset viewing angles.

[0112] As an example, when the extended object shape features and the extended object texture features are optimized, the explicit mesh surface of the target object can be extracted from the extended object shape features and the extended object texture features corresponding to multiple preset perspectives. For example, after the optimization process converges, a mesh extraction algorithm such as a mesh-UDF algorithm can be used to extract an explicit mesh surface (mesh) from the optimized undirected distance field and texture radiation field to obtain the model data of the three-dimensional model of the target object, which can characterize the texture information and shape information of the target object in three-dimensional space. Among them, the mesh-UDF algorithm directly meshes the deep UDF into an open surface with a marching cube extension by locally detecting surface intersections. For example, a three-dimensional model rendering engine can be used to construct a three-dimensional object based on the model data of the three-dimensional model of the target object.

[0113] Figure 6 A block diagram of a large model-based three-dimensional object generation device according to an embodiment of the present disclosure is schematically shown.

[0114] like Figure 6 As shown, the large model-based three-dimensional object generation device 600 may include a morphology conversion module 610 , a fusion module 620 , a perspective conversion module 630 and a generation module 640 .

[0115] The morphology conversion module 610 is used to perform morphology conversion on the initial graph for the target object to obtain a morphology graph representing an object with a specified morphology.

[0116] The fusion module 620 is used to use the visual macro model to perform feature fusion on the object texture features of the initial image and the object morphology image to obtain a basic target object image.

[0117] The perspective conversion module 630 is used to perform multi-perspective conversion on the basic target object graph to obtain multiple extended target object graphs related to multiple preset perspectives.

[0118] The generating module 640 is used to generate a three-dimensional object based on a plurality of extended target object graphs.

[0119] According to an embodiment of the present disclosure, the visual big model includes a homogeneous image generation big model and a texture generation big model; wherein the fusion module 620 may include a multi-level feature fusion submodule.

[0120] The multi-level feature fusion submodule is used to use the image generation model to perform multi-level feature fusion on the object morphology map and the multi-level object texture features to obtain the target basic object map. The multi-level object texture features are determined by performing multi-level feature extraction on the initial map using the texture generation model.

[0121] According to an embodiment of the present disclosure, the image generation large model includes N levels of feature fusion layers, and the texture generation large model includes N levels of texture feature extraction layers; wherein the multi-level feature fusion submodule may include a first processing submodule and a second processing submodule.

[0122] The first processing submodule is used to use the nth level feature fusion layer to process the nth level object texture feature and the n-1th level intermediate fusion feature to obtain the nth level intermediate fusion feature. The first level intermediate fusion feature is determined by using the first level feature fusion layer to process the first level object texture feature and the object morphology map.

[0123] The second processing submodule is used to determine the basic target object graph based on the Nth level intermediate fusion feature when N=n, wherein N≥n>1, and N and n are integers.

[0124] According to an embodiment of the present disclosure, the first processing submodule may include a multi-level feature fusion unit.

[0125] The multi-level feature fusion unit is used to process the n-th level object texture feature, the n-1-th level intermediate fusion feature and the initial image semantic feature by using the n-th level feature fusion layer, wherein the initial image semantic feature is determined by extracting the image semantic feature of the initial image.

[0126] According to an embodiment of the present disclosure, the morphology conversion module 610 may include a mask submodule and a morphology conversion submodule.

[0127] The mask submodule is used to mask the initial image to obtain an initial morphological mask image representing the initial morphology of the target object.

[0128] The morphological transformation submodule is used to perform morphological transformation on the initial morphological mask image to obtain an object morphological image.

[0129] According to an embodiment of the present disclosure, the generation module 640 may include a feature extraction submodule and a construction submodule.

[0130] The feature extraction submodule is used to extract features from multiple extended target object graphs to obtain extended object shape features based on undirected distance field representation and extended object texture features based on texture radiation field representation.

[0131] The construction submodule is used to construct a three-dimensional object based on the extended object shape features and the extended object texture features corresponding to multiple preset viewing angles.

[0132] Figure 7 The structural block diagram of an artificial intelligence agent according to an embodiment of the present disclosure is schematically shown.

[0133] In the embodiments of the present disclosure, Figure 7 As shown, the AI ​​agent 700 may include an input module 710 , a processing module 720 , and an output module 730 .

[0134] The input module 710 is used to receive input information.

[0135] The processing module 720 is used to determine the target task based on the input information received by the input module, determine the visual large model based on the target task, and obtain output information by calling the visual large model to execute the above-mentioned large-model-based three-dimensional object generation method.

[0136] The output module 730 is used to output the output information obtained by the processing module.

[0137] According to an embodiment of the present disclosure, the input module 710 is responsible for receiving or sensing information such as queries, requests, instructions, signals or data from the outside world (e.g., users or external environment), and converting them into a format that can be understood and processed by the AI ​​agent 700. The input module 710 is the primary link for the AI ​​agent 700 to interact with the outside world, which enables the AI ​​agent 700 to efficiently and accurately obtain the necessary "sensory" information from the outside world and respond to this information.

[0138] In an example, the input module 710 may input the initial graph for the target object described above.

[0139] In the example, the processing module 720 is the core support for the AI ​​agent 700 to process complex tasks. The processing module 720 can execute the large model-based three-dimensional object generation method described above.

[0140] In the example, the performance of the processing module 720 may be closely related to the large model on which the AI ​​agent 700 is based. In order to give full play to the capabilities of the large model, the internal structure of the processing module 720 may be designed to be highly configurable and extensible so as to cope with various types of tasks and requirements in real scenarios.

[0141] In the example, after the AI ​​agent 700 obtains the initial image, the processing module 720 can perform a morphological transformation on the initial image for the target object to obtain an object morphological image representing the specified morphology. The processing module 720 can use the visual macro model to perform feature fusion on the object texture features of the initial image and the object morphological image to obtain a basic target object image. The processing module 720 can perform a multi-perspective transformation on the basic target object image to obtain multiple extended target object images related to multiple preset perspectives, and can generate a three-dimensional object based on the multiple extended target object images, and pass the three-dimensional object to the output module 730.

[0142] It is understandable that although the large language model has excellent language understanding and generation capabilities, it is just like a human being and can only perform very limited tasks without any tools. When AI agent 700 is given the ability to call tools, it can perform tasks such as using a calculator to perform mathematical operations, using Python to perform data analysis, and using a search engine to perform weather forecasts.

[0143] In an example, the output module 730 may output the three-dimensional object described above.

[0144] The AI ​​agent 700 according to the embodiment of the present disclosure can simply and effectively improve the level of intelligence, and improve flexibility and versatility.

[0145] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0146] According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0147] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method as described above.

[0148] According to an embodiment of the present disclosure, a computer program product includes a computer program, and when the computer program is executed by a processor, the computer program implements the method as described above.

[0149] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0150] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0151] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0152] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a three-dimensional object generation method based on a large model. For example, in some embodiments, the three-dimensional object generation method based on a large model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the three-dimensional object generation method based on the large model described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the large model-based three-dimensional object generation method in any other appropriate manner (for example, by means of firmware).

[0153] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0154] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0155] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0156] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0157] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0158] A computer system may include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises through computer programs running on respective computers and having a client-server relationship to each other. The server may be a cloud server, a server in a distributed system, or a server combined with a blockchain.

[0159] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0160] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating a three-dimensional object based on a large model, comprising: Performing morphological transformation on the initial graph for the target object to obtain a morphological graph representing the object with a specified morphology; Using a large visual model, feature fusion is performed on the object texture features of the initial image and the object morphology image to obtain a basic target object image; Performing multi-view conversion on the basic target object graph to obtain multiple extended target object graphs associated with multiple preset viewpoints; and A three-dimensional object is generated based on the plurality of extended target object graphs.

2. The method according to claim 1, wherein: The visual big model includes an isomorphic image generation big model and a texture generation big model; The step of using the visual macro model to perform feature fusion on the object texture features of the initial image and the object morphology image to obtain the basic target object image includes: The object morphology map is subjected to multi-level feature fusion with the multi-level object texture features by using the image generative large model to obtain the target basic object map. The multi-level object texture features are determined by performing multi-level feature extraction on the initial map by using the texture generative large model.

3. The method according to claim 2, wherein: The image generation model includes N levels of feature fusion layers, and the texture generation model includes N levels of texture feature extraction layers; Wherein, the step of using the image generation model to perform multi-level feature fusion on the object morphology map and the multi-level object texture features to obtain the target basic object map comprises: Processing the n-th level object texture feature and the n-1-th level intermediate fusion feature using the n-th level feature fusion layer to obtain the n-th level intermediate fusion feature, wherein the first level intermediate fusion feature is determined by processing the first level object texture feature and the object morphology map using the first level feature fusion layer; and In the case of N=n, the basic target object graph is determined based on the Nth level intermediate fusion features, wherein N≥n>1, and N and n are integers.

4. The method according to claim 3, wherein: The method of processing the n-th level object texture feature and the n-1-th level intermediate fusion feature by using the n-th level feature fusion layer comprises: The n-th level feature fusion layer is used to process the n-th level object texture features, the n-1-th level intermediate fusion features and the initial image semantic features, wherein the initial image semantic features are determined by extracting image semantic features from the initial image.

5. The method according to claim 1, wherein: The performing of morphological transformation on the initial graph for the target object to obtain a morphological graph representing the object having a specified morphology comprises: Masking the initial image to obtain an initial morphology mask image representing the initial morphology of the target object; and The initial morphological mask image is morphologically transformed to obtain the object morphological image.

6. The method according to claim 1, wherein: Generating a three-dimensional object based on a plurality of the extended target object graphs comprises: Performing feature extraction on the plurality of extended target object graphs to obtain extended object shape features based on undirected distance field representation and extended object texture features based on texture radiation field representation; The three-dimensional object is constructed based on the extended object shape features and the extended object texture features corresponding to the plurality of preset viewing angles.

7. A three-dimensional object generation device based on a large model, comprising: A morphological transformation module, used for performing morphological transformation on an initial image for a target object to obtain a morphological image representing an object having a specified morphology; A fusion module, used for performing feature fusion on the object texture features of the initial image and the object morphology image using a large visual model to obtain a basic target object image; A perspective conversion module, used for performing multi-perspective conversion on the basic target object graph to obtain multiple extended target object graphs related to multiple preset perspectives; as well as A generating module is used to generate a three-dimensional object based on a plurality of the extended target object graphs.

8. The device according to claim 7, wherein: The visual big model includes an isomorphic image generation big model and a texture generation big model; Wherein, the fusion module includes: The multi-level feature fusion submodule is used to use the image generation model to perform multi-level feature fusion on the object morphology map and the multi-level object texture features to obtain the target basic object map. The multi-level object texture features are determined by performing multi-level feature extraction on the initial map using the texture generation model.

9. The device according to claim 8, wherein: The image generation model includes N levels of feature fusion layers, and the texture generation model includes N levels of texture feature extraction layers; Wherein, the multi-level feature fusion submodule includes: a first processing submodule, for processing the nth level object texture feature and the n-1th level intermediate fusion feature by using the nth level feature fusion layer to obtain the nth level intermediate fusion feature, wherein the first level intermediate fusion feature is determined by processing the first level object texture feature and the object morphology map by using the first level feature fusion layer; and The second processing submodule is used to determine the basic target object graph based on the Nth level intermediate fusion feature when N=n, wherein N≥n>1, and N and n are integers.

10. The device according to claim 9, wherein: The first processing submodule includes: A multi-level feature fusion unit is used to process the n-level object texture features, the n-1-th level intermediate fusion features and the initial image semantic features using the n-th level feature fusion layer, wherein the initial image semantic features are determined by extracting image semantic features from the initial image.

11. The device according to claim 7, wherein: The morphology conversion module comprises: a mask submodule, configured to mask the initial image to obtain an initial morphological mask image representing the initial morphology of the target object; and The morphology conversion submodule is used to perform morphology conversion on the initial morphology mask image to obtain the object morphology image.

12. The device according to claim 7, wherein: The generation module comprises: A feature extraction submodule, used for extracting features from the plurality of extended target object graphs to obtain extended object shape features based on undirected distance field representation and extended object texture features based on texture radiation field representation; The construction submodule is used to construct the three-dimensional object based on the extended object shape features and the extended object texture features corresponding to the plurality of preset viewing angles.

13. An artificial intelligence agent, comprising: An input module, used for receiving input information; a processing module, configured to determine a target task based on the input information received by the input module, determine a visual macro model based on the target task, and obtain output information by executing the method according to any one of claims 1 to 6 by calling the visual macro model; An output module is used to output the output information obtained by the processing module.

14. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

16. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images