Three-dimensional content generation method for text travel digital scene

Abstract

The invention provides a three-dimensional content generation method for a text travel digital scene, and relates to the field of digital content generation. The method comprises the steps of obtaining attribute description information input by a user in response to an operation of creating target three-dimensional content by the user; inputting the attribute description information into an attribute extraction model for feature extraction so as to output attribute description corresponding to the target three-dimensional content; inputting the attribute description into a component generator to output a first component and a second component for constructing the target three-dimensional content; based on the geometric parameters and the constraint relationship, performing fusion optimization on the first component and the second component to obtain a fusion model; and performing rendering optimization on the fusion model based on the color information and the pattern information to generate the target three-dimensional content. According to the method and the device, the problem that a parameter template-based three-dimensional content combination method cannot meet the requirements of the user on diversification and personalized customization of the three-dimensional content when the personalized expression complexity of the user is relatively high or the customization requirement is remarkably enhanced is solved.

Description

Technical Field

[0001] This application relates to the field of digital content generation, and particularly to a method for generating three-dimensional content for cultural and tourism digital scenarios. Background Art

[0002] Cultural and tourism scenarios are often presented in the form of pictures or pre-recorded videos, which are difficult to stimulate users' deep participation and emotional resonance. In order to improve the immersion, expressiveness, and interactivity of the cultural and tourism industry in the digital process, how to achieve efficient, flexible, and semantic understanding-enabled three-dimensional display has become the mainstream research direction.

[0003] In traditional methods, three-dimensional content for cultural and tourism digital scenarios can be generated through a three-dimensional content combination method based on parameter templates. This method relies on predefined model libraries and parametric assembly rules, and generates specific three-dimensional content by selecting templates and adjusting limited parameters such as size, color, etc. Although the above method can improve the generation efficiency to a certain extent, it cannot dynamically understand and extract semantic attributes according to the user's natural language description, and automatically generate three-dimensional components and overall models that conform to the described semantics, resulting in the generated three-dimensional models being limited within the template range. Therefore, when the complexity of users' personalized expressions is relatively high or the customization requirements are significantly enhanced, it cannot meet the diverse and personalized customization needs of users.

[0004] Therefore, there is an urgent need for a method for generating three-dimensional content for cultural and tourism digital scenarios. Summary of the Invention

[0005] This application provides a method for generating three-dimensional content for cultural and tourism digital scenarios, which solves the problem that the three-dimensional content combination method based on parameter templates cannot meet the diverse and personalized customization needs of users for three-dimensional content when the complexity of users' personalized expressions is relatively high or the customization requirements are significantly enhanced.

[0006] In the first aspect of this application, a method for generating three-dimensional content for cultural and tourism digital scenarios is provided. The method includes: in response to an operation by a user to create target three-dimensional content, obtaining attribute description information input by the user, where the attribute description information includes the types of components, the number of components, the colors of components, the materials of components, and the geometric dimensions of components required for the target three-dimensional content; inputting the attribute description information into an attribute extraction model for feature extraction to output an attribute description corresponding to the target three-dimensional content, where the attribute description includes geometric parameters, constraint relationships, color information, and pattern information; inputting the attribute description into a component generator to output target components for constructing the target three-dimensional content, where the target components include first components and second components; based on the geometric parameters and constraint relationships, performing fusion optimization on the first components and the second components to obtain a fusion model; and based on the color information and pattern information, performing rendering optimization on the fusion model to generate the target three-dimensional content.

[0007] Optionally, before inputting the attribute description information into the attribute extraction model for feature extraction to output the attribute description corresponding to the target 3D content, it is necessary to construct the attribute extraction model, which specifically includes: constructing an attribute pre-extraction model, where the attribute pre-extraction model includes a preprocessing module and a bidirectional long short-term memory network; inputting the attribute description information into the preprocessing module to obtain a description sequence; training the bidirectional long short-term memory network with the description sequence to construct the attribute extraction model.

[0008] Optionally, the attribute description information includes speech description information and text description information. Inputting the attribute description information into the preprocessing module to obtain a description sequence specifically includes: extracting the Mel-frequency cepstral coefficients in the speech description information through the preprocessing module and segmenting them according to a preset time window to form a speech sequence; constructing a vocabulary, and based on the vocabulary, mapping the text description information into a digital sequence through the preprocessing module; using the speech sequence and the digital sequence as the description sequence.

[0009] Optionally, before inputting the attribute description into the component generator to output the target components for constructing the target 3D content, it is also necessary to construct the component generator, which specifically includes: constructing a 3D model set, where the 3D model set includes multiple standard 3D models and multiple standard 3D descriptions, and one standard 3D model corresponds to one standard 3D description; adding a first noise to the standard 3D model to obtain a noisy point cloud, where the first noise is obtained by sampling from a standard normal distribution; constructing a diffusion model, where the diffusion model includes a 3D description extraction structure, a point cloud description extraction structure, a time description extraction structure, a conversion network, and a predicted noise structure; inputting the standard 3D description, the noisy point cloud, and the diffusion time into the diffusion model to output a second noise; calculating the training loss based on the first noise and the second noise, and updating the parameters of the diffusion model according to the training loss to construct the component generator.

[0010] Optionally, inputting the standard 3D description, the noisy point cloud, and the diffusion time into the diffusion model to output a second noise specifically includes: inputting the attribute description of the standard 3D description into the corresponding 3D description extraction structure to obtain a 3D description block, where the 3D description block contains the representation of the attribute description of the standard 3D description in a high-dimensional space; inputting the noisy point cloud into the point cloud description extraction structure to obtain a point cloud description block, where the point cloud description block is used to reflect the characteristics and distribution of the noisy point cloud; inputting the diffusion time into the time description extraction structure to obtain a time description block, where the time description block is used to store the information of the current diffusion time; inputting the 3D description block, the point cloud description block, and the time description block into the conversion network and outputting a conversion description; inputting the conversion description into the predicted noise structure in the diffusion model to output a second noise.

[0011] Optionally, based on geometric parameters and constraint relationships, the first component and the second component are fusion-optimized to obtain a fusion model, which specifically includes: performing a first geometric transformation on the first component and the second component based on geometric parameters, where the first geometric transformation operation includes a scaling operation and a mirror operation; determining a first splicing area of the first component and a second splicing area of the second component, where the first splicing area is the connector of the first component and the second splicing area is the connector of the second component; splicing the first component and the second component based on the constraint relationship, where the constraint relationship is to make the centroid of the first splicing area coincide with the centroid of the second splicing area; evaluating the similarity between the first splicing area and the second splicing area based on the overlapping area between the first splicing area and the second splicing area to obtain a similarity evaluation result, where the overlapping area includes the contact area and the overlapping volume between the first splicing area and the second splicing area; performing a second geometric transformation operation on the first component and the second component according to the similarity evaluation result and determining an adjustment parameter set, where the second geometric transformation operation includes a translation operation and a rotation operation; selecting a target adjustment parameter from the adjustment parameter set and fusing the first component and the second component according to the target adjustment parameter to obtain a fusion model.

[0012] Optionally, based on color information and pattern information, the fusion model is rendered and optimized to generate target three-dimensional content, which specifically includes: performing a first rendering optimization and a second rendering optimization on the fusion model to generate target three-dimensional content, where the first rendering optimization is used to adjust the color information and pattern information in the fusion model, and the second rendering optimization is used to perform a smooth connection process on the overlapping area.

[0013] In the second aspect of the present application, a three-dimensional content generation device for a cultural and tourism digital scene is provided. The device includes an acquisition module, a feature extraction module, a component generation module, a hybrid optimization module, and a rendering optimization module, where,

[0014] The acquisition module is configured to obtain the attribute description information input by the user in response to the operation of the user to create target three-dimensional content, where the attribute description information includes the types of components, the number of components, the component colors, the component materials, and the component geometric dimensions required for the target three-dimensional content.

[0015] The feature extraction module is configured to input the attribute description information into an attribute extraction model for feature extraction to output the attribute description corresponding to the target three-dimensional content, where the attribute description includes geometric parameters, constraint relationships, color information, and pattern information.

[0016] The component generation module is configured to input the attribute description into a component generator to output target components for constructing the target three-dimensional content, where the target components include a first component and a second component.

[0017] The hybrid optimization module is configured to perform fusion optimization on the first component and the second component based on geometric parameters and constraint relationships to obtain a fusion model.

[0018] A rendering optimization module, configured to perform rendering optimization on a fusion model based on color information and pattern information to generate target three-dimensional content.

[0019] In a third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of the above.

[0020] In a fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to perform the method according to any one of the above.

[0021] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0022] 1. By obtaining the attribute description information input by the user; inputting the attribute description information into an attribute extraction model for feature extraction to output the attribute description corresponding to the target three-dimensional content; inputting the attribute description into a component generator to output the first component and the second component for constructing the target three-dimensional content; based on geometric parameters and constraint relationships, performing fusion optimization on the first component and the second component to obtain a fusion model; and based on color information and pattern information, performing rendering optimization on the fusion model to generate the target three-dimensional content, thereby realizing a full-automatic modeling process from natural language input to three-dimensional model output, significantly improving the generation efficiency of three-dimensional content, and meeting the diverse and personalized customization requirements of users for three-dimensional content.

[0023] 2. Constructing an attribute pre-extraction model, the attribute pre-extraction model includes a preprocessing module and a bidirectional long short-term memory network; inputting the attribute description information into the preprocessing module to obtain a description sequence; training the bidirectional long short-term memory network through the description sequence to construct an attribute extraction model, so as to accurately understand the semantic elements in the user input and extract the attribute information highly relevant to the generation of three-dimensional content, enhancing the model's parsing ability for natural language descriptions, and improving the matching degree between the finally generated target three-dimensional content and the user's needs.

[0024] 3. Construct a three-dimensional model set, add first noise to the standard three-dimensional model to obtain a noisy point cloud, then construct a diffusion model, and input the standard three-dimensional description, the noisy point cloud, and the diffusion time into the diffusion model to output second noise. Thus, calculate the training loss based on the first noise and the second noise, and update the parameters of the diffusion model according to the training loss to construct a component generator. Furthermore, generate components of various styles and types through the finally constructed component generator, and through different combinations and fusions, countless different cultural and tourism scenarios can be created, providing more diverse digital content for the cultural and tourism industry. Description of the Drawings

[0025] Figure 1 is a schematic flowchart of a method for generating three-dimensional content of a cultural and tourism digital scenario provided by an embodiment of the present application;

[0026] Figure 2 is a schematic block diagram of a device for generating three-dimensional content of a cultural and tourism digital scenario provided by an embodiment of the present application;

[0027] Figure 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0028] Description of the reference numerals: 21, acquisition module; 22, feature extraction module; 23, component generation module; 24, hybrid optimization module; 25, rendering optimization module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. Detailed Embodiments

[0029] In order to enable those skilled in the art to better understand the technical solutions in this specification, the following will clearly and completely describe the technical solutions in the embodiments of this specification in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0030] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. As used in the specification of the present application, the singular forms "a", "an", "the", "above-mentioned", "this", and "such" are also intended to include the plural forms, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more of the listed items.

[0031] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and should not be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0032] In order to enable those skilled in the art to better understand the technical solutions of the present application, the present application will be further described in detail below with reference to the accompanying drawings.

[0033] Please refer to Figure 1 , which shows a schematic flowchart of a method for generating three-dimensional content in a cultural and tourism digital scenario provided by an embodiment of the present application. The flowchart mainly includes the following steps: S101 to S105.

[0034] Step S101, in response to an operation by the user to create target three-dimensional content, obtain the attribute description information input by the user.

[0035] Specifically, in the solution of the embodiments of the present application, by providing personalized customization for the user, the user can generate unique target three-dimensional content according to their own imagination and needs by inputting different attribute description information. At this time, obtain the attribute description information input by the user, and the attribute description information includes but is not limited to text form and voice form. The attribute description information includes but is not limited to: the type of components required for the target three-dimensional content, the number of components, the color of the components, the material of the components, the geometric dimensions of the components, etc.

[0036] Step S102, input the attribute description information into an attribute extraction model for feature extraction to output the attribute description corresponding to the target three-dimensional content.

[0037] Specifically, by training the attribute extraction model and performing feature extraction on the attribute description information, the attribute description corresponding to the target three-dimensional content is output. The attribute description includes not only static geometric or visual features, but also composite information such as logical constraints, semantic identifiers, and spatial relationships, and is the core variable driving the entire three-dimensional content generation process. For the specific training steps of the attribute extraction model, please refer to Step S21 to Step S23.

[0038] Step S21, construct an attribute pre-extraction model, and the attribute pre-extraction model includes a preprocessing module and a bidirectional long short-term memory network.

[0039] Specifically, an attribute pre-extraction model is constructed. The attribute pre-extraction model includes a description sample set, a preprocessing module, and a bidirectional long short-term memory network (BiLSTM). The description sample set includes a number of user description information samples; these samples can come from different channels, such as the text descriptions of expected scenarios by users on cultural and tourism related platforms, and the voice communication records of users' cultural and tourism experiences when communicating with customer service. By having diverse sample sources, the training set can be made more representative, covering various possible user expression forms and needs. The preprocessing module is responsible for converting different forms of user description information (such as voice, text, etc.) into a description sequence suitable for BiLSTM processing. BiLSTM consists of a forward LSTM and a backward LSTM. LSTM can effectively handle the long-term dependence problem in sequence data and control the flow and memory of information through gating mechanisms (input gate, forget gate, and output gate). BiLSTM can process sequence data from both the forward and backward directions simultaneously, thus better capturing the context information in the sequence. For example, for the description "a teacup with a lid and a handle", BiLSTM can combine the information of "handle" behind when processing "lid" to more accurately understand the attributes of the entire teacup.

[0040] Step S22: Input the attribute description information into the preprocessing module to obtain a description sequence.

[0041] Specifically, the attribute description information includes voice description information and text description information. When the attribute description information is voice description information, the description sequence can be obtained in the following way, that is, by extracting the Mel-frequency cepstral coefficients in the voice description information through the preprocessing module and segmenting them according to a preset time window to form a voice sequence: Extract the Mel-frequency cepstral coefficients (MFCC) of the voice in the description sample and segment them according to a preset time window to form a voice sequence of a fixed length. For voice-form user description information, the Mel-frequency cepstral coefficients (MFCC) can be extracted. MFCC can simulate the perception characteristics of the human auditory system for sound frequencies. When extracting MFCC, first frame the voice signal, then perform Fourier transform on each frame to obtain the spectrum, then convert the spectrum to the Mel-frequency domain through a Mel filter bank, and finally perform discrete cosine transform (DCT) to obtain the MFCC coefficients. The MFCC coefficients can be segmented according to a time window of 10 - 30 milliseconds to form a voice sequence of a fixed length, which can convert the continuous voice signal into a discrete data form suitable for model processing.

[0042] When the attribute description information is voice description information, the description sequence can be obtained in the following way, that is, constructing a vocabulary, and based on the vocabulary, mapping the text description information into a digital sequence through a preprocessing module: constructing a vocabulary, numbering the words in the text of the description sample, and establishing a mapping relationship from the word to the index. For the user description information in text form, a vocabulary can be constructed. The vocabulary can summarize all the words that appear in the text and assign a unique number to each word, establishing a mapping relationship from the word to the index; in this way, when processing the text, each word can be represented by the corresponding index value, thus converting the text into a digital sequence for easy computer processing. For example, for the text "I want a blue teacup", after mapping by the vocabulary, it may become a digital sequence like [1, 2, 3, 4, 5], where each number corresponds to a word in the vocabulary. Finally, the voice sequence and the digital sequence are used as the description sequence.

[0043] Step S23, training the bidirectional long short-term memory network with the description sequence to construct an attribute extraction model.

[0044] Specifically, training the bidirectional long short-term memory network (BiLSTM) with the description sequence to obtain an attribute extraction model. During the training process, the model can continuously adjust its own parameters (such as weights and biases) to minimize the error between the prediction result and the true label. For example, for the user description "a teacup with a handle", the goal of the model is to accurately extract attributes such as "handle" and "teacup"; through multiple iterative trainings, the model gradually learns the mapping relationship between different feature description sequences and attributes. When training the model, some optimization algorithms can be used, such as Stochastic Gradient Descent (SGD), Adagrad, Adadelta, etc., to update the parameters of the model, so that the model can converge to the optimal solution faster. When the error of the attribute extraction model on the training set reaches a preset threshold or after a preset number of training rounds, it can be considered that the model training is completed, and an attribute extraction model that can accurately extract the attributes in the user description information is obtained.

[0045] Step S103, inputting the attribute description into the component generator to output target components for constructing the target three-dimensional content, where the target components include a first component and a second component.

[0046] Specifically, training the component generator in a preset manner, and by inputting the attribute description into the component generator, thereby outputting target components for constructing the target three-dimensional content, that is, the first component and the second component. In the embodiments of the present application, the first component and the second component represent all the components for constructing the target three-dimensional content, and the number of components for constructing the target three-dimensional content is not limited to two, and can also be three or more. For the steps of training the component generator in a preset manner, please refer to Step S31 to Step S35.

[0047] Step S31: Construct a three-dimensional model set, which includes multiple standard three-dimensional models and multiple standard three-dimensional descriptions. Among them, one standard three-dimensional model corresponds to one standard three-dimensional description.

[0048] Specifically, construct a three-dimensional model set, which includes a number of standard three-dimensional models and a number of standard three-dimensional descriptions. Each standard three-dimensional model corresponds to a unique standard three-dimensional description. The three-dimensional model set can provide basic data for component generation, enabling the component generator to learn the mapping relationship from description to model; each standard three-dimensional model corresponds one-to-one with its description, so as to establish the connection between semantics and geometric structure during the training process. In addition, in the embodiments of the present application, standard three-dimensional models can be collected from various sources (such as existing three-dimensional model libraries, open-source data sets, etc.), and corresponding standard three-dimensional descriptions can be written for them to ensure that the descriptions are as accurate and detailed as possible, so as to provide valuable information for the diffusion model in subsequent training; these standard three-dimensional models can cover different categories, such as daily necessities, building facilities, natural landscapes, local components, etc., providing a basis for generating various types of components.

[0049] Step S32: Add first noise to the standard three-dimensional model to obtain a noisy point cloud. The first noise is obtained by sampling from the standard normal distribution.

[0050] Specifically, add first noise to the standard three-dimensional model to obtain a noisy point cloud; the first noise is obtained by sampling from the standard normal distribution. During the training of the component generator, the standard three-dimensional model is converted into a point cloud. By adding the first noise sampled from the standard normal distribution, the original geometric structure is broken, enabling the standard three-dimensional model to learn how to recover the target structure from the noise, and at the same time providing the initial state for the subsequent diffusion process. In this embodiment, for each standard three-dimensional model, it is converted into a point cloud representation. An algorithm for converting a three-dimensional model to a point cloud can be used to sample the model surface into point cloud data. Suppose the noisy point cloud contains n points, denoted as P i =(X i , Y i , Z i ), where i = 1, 2, 3,..., n. The first noise is obtained by sampling from the standard normal distribution. For each point in the point cloud, a set of noise values needs to be sampled. Since the point cloud is three-dimensional, for each point P i , three values need to be sampled from the standard normal distribution to form a noise vector Here, each noise component are all independently sampled from the standard normal distribution N(0, 1). In actual programming, functions provided by various mathematical libraries can be used for sampling. For example, in Python, numpy.random.normal(0, 1) can be used to generate random numbers that follow the standard normal distribution N(0, 1). The sampled noise vector N i is added to the corresponding point P i to obtain the point P of the noisy point cloud 1i . The specific calculation formula is: where i = 1, 2, 3,..., n. By performing the above operations on each point P i , the noisy point cloud is finally obtained, and the point set of the noisy point cloud is {P 1i |i = 1, 2, 3,..., n}.

[0051] Step S33: Construct a diffusion model, which includes a three-dimensional description extraction structure, a point cloud description extraction structure, a time description extraction structure, a transformation network, and a predicted noise structure.

[0052] Specifically, the diffusion model is the core part of the generation of the entire component and can generate three-dimensional components with specific attributes and styles. The diffusion model includes a three-dimensional description extraction structure, a point cloud description extraction structure, a time description extraction structure, a transformation network, and a predicted noise structure. Among them, each three-dimensional description extraction structure corresponds to an attribute description; the three-dimensional description extraction structure includes a text-image comparison pre-trained network (CLIP) and a perceptron (MLP). The time description extraction structure and the point cloud description extraction structure both include a perceptron (MLP). Among them, different extraction structures of the diffusion model are used to extract feature information from different inputs; the three-dimensional description extraction structure uses the text-image comparison pre-trained network (CLIP) and the perceptron (MLP) to convert the attribute description into a more informative feature representation. CLIP can associate the semantic representations of the text and the three-dimensional model of the image, and the MLP further processes and transforms this information to better cooperate with the subsequent network structure; the MLP in the time description extraction structure and the point cloud description extraction structure extracts features from the time information and the noisy point cloud information respectively to meet different conditional requirements in the generation process; the transformation network integrates and transforms various description blocks, and the predicted noise structure is finally used to predict the noise for model training and optimization.

[0053] Step S34: Input the standard three-dimensional description, the noisy point cloud, and the diffusion time into the diffusion model to output the second noise.

[0054] Specifically, input the attribute description of the standard three-dimensional description into the corresponding three-dimensional description extraction structure to obtain a three-dimensional description block, which contains the representation of the attribute description of the standard three-dimensional description in the high-dimensional space: The Contrastive Language-Image Pretraining (CLIP) network can convert text information into semantically related vectors, and the Multi-Layer Perceptron (MLP) can further refine and adjust it to make it more suitable for calculation and fusion in the generation process. In the embodiment of the present application, taking the attribute description of "a teacup with a circular cup lid" as an example, input it into the three-dimensional description extraction structure. First, the CLIP network generates a feature vector, and then through the MLP for non-linear transformation, a three-dimensional description block is obtained. This block contains the representation of this attribute description in the high-dimensional space and stores the key information about this description, such as shape, structural features, etc. The high-dimensional space has multiple dimensions, each dimension corresponding to an attribute description of the standard three-dimensional description, and also including the material, surface texture, color, etc. of the object. The high-dimensional space can capture the complex relationships and potential features between attributes, enabling the three-dimensional description block to more accurately express the information of the standard three-dimensional description at a richer feature level.

[0055] Input the noisy point cloud into the point cloud description extraction structure to obtain a point cloud description block, which is used to reflect the characteristics and distribution of the noisy point cloud: The Multi-Layer Perceptron (MLP) in the point cloud description extraction structure can extract the features of the information of the noisy point cloud and convert it into a form that can be processed in subsequent networks; this helps the diffusion model understand the state of the current noisy point cloud and provides a basis for predicting and adding noise. In the embodiment of the present application, take the point coordinates and their features (such as color, normal, etc.) of the noisy point cloud as input, and through the MLP network, convert it into a low-dimensional or high-dimensional feature vector, that is, the point cloud description block, which can reflect the characteristics and distribution of the noisy point cloud.

[0056] Input the diffusion time into the time description extraction structure to obtain a time description block, which is used to store the information of the current diffusion time: Input the diffusion time into the time description extraction structure to obtain a time description block. During the diffusion process, the Multi-Layer Perceptron (MLP) in the time description structure can convert the diffusion time (representing the stage or step of diffusion) into a feature representation that can be used in the generation process to control the progress and quality of generation. Input the diffusion time t, and through the MLP, convert it into a time-related feature vector. The time description block stores the information about the current diffusion time and can provide time conditions for subsequent generation processes.

[0057] Input the 3D description block, point cloud description block, and time description block into the conversion network, and output a conversion description: In this embodiment, the conversion network can integrate and transform information from different sources (attribute description, point cloud information, and time information) to predict the noise structure and perform noise prediction based on this comprehensive information. Concatenate the 3D description block, point cloud description block, and time description block or input them into the conversion network through some complex fusion operations to obtain a conversion description; this conversion description combines the current attribute information, point cloud state, and diffusion time information to prepare for the final noise prediction.

[0058] Finally, input the conversion description into the predicted noise structure in the diffusion model to output the second noise. In the embodiment of this application, the predicted noise structure can predict the second noise that needs to be added according to the conversion description; this process is similar to the inverse process in the denoising diffusion process, that is, predicting the noise that needs to be added or removed at the current diffusion stage based on the current information, so as to gradually recover the target component from the noise. In addition, the predicted noise structure can be a multi-layer perceptron or other neural networks, and finally output the second noise, which will be used to calculate the loss in subsequent training.

[0059] Step S35, calculate the training loss according to the first noise and the second noise, and update the parameters of the diffusion model according to the training loss to construct a component generator.

[0060] Specifically, calculate the training loss by comparing the first noise (the originally added noise) and the second noise (the noise predicted by the model), so as to optimize the diffusion model. The loss function can adopt the mean square error (MSE). By minimizing the difference between the two, the model can better predict the noise that should be added or removed during the diffusion process, and thus achieve better component generation. Among them, for each noisy point cloud, calculate the mean square error L = MSE(N1, N2) between the first noise N1 and the second noise N2; use the Adam optimizer to update the parameters in each structure of the diffusion model according to this loss, including the text-image comparison pre-training network (CLIP) and the multi-layer perceptron (MLP) in the 3D description extraction structure, the MLP in the time description extraction structure, the MLP in the point cloud description extraction structure, the parameters of the conversion network, and the predicted noise structure. In each training cycle, repeat the above process, continuously adjust the model to make the predicted noise closer to the actually added noise, and finally improve the component generation ability.

[0061] It should be noted that in the embodiments of the present application, the training process of the component generator adopts a generation process based on a diffusion model; it can take conditional information (attribute descriptions, time, etc.) and noisy point clouds as inputs, and through a series of extraction structures and transformation networks, finally predict the noise that needs to be added or removed during the diffusion process. During the training process, noisy point clouds are obtained by adding noise to a standard 3D model as the starting state. The diffusion model is used to process the noisy point clouds, predict the noise and compare it with the original noise, and optimize each structure of the diffusion model by minimizing the difference between the two (training loss). In this way, the diffusion model can learn how to generate 3D components with specific attribute descriptions from noise, and finally realize the generation from the user's attribute description to the target component. When the user inputs an attribute description such as "a teacup with a circular cup lid", the diffusion model can gradually generate a teacup component that meets this description from the noise according to the trained network structure and parameters, providing a basis for subsequent fusion optimization and rendering. At the same time, the introduction of the diffusion time can control the progress and quality of the generation, making the generation process more controllable and stable; under the iteration of multiple training cycles, the diffusion model will continuously improve the quality and diversity of component generation to meet various different user needs and scenarios.

[0062] Step S104: Based on geometric parameters and constraint relationships, perform fusion optimization on the first component and the second component to obtain a fusion model.

[0063] Specifically, the attribute description includes geometric parameters and constraint relationships; the constraint relationships include: coincidence, parallelism, perpendicularity, coaxiality, internal tangency, external tangency, and inclination angle. Geometric parameters and constraint relationships provide basic information for component fusion; geometric parameters determine the initial shape and size of the component, while constraint relationships define the spatial position relationships between components. In the embodiments of the present application, geometric parameters and constraint relationships are extracted from the input attribute description. For example, for a description "a cup body (the first component) with a height of 10 cm, a cup lid handle (the second component) with a diameter of 5 cm, the bottom surface of the 'cup lid' is parallel and coaxial with one surface of the 'cup body'", here "a height of 10 cm" and "a diameter of 5 cm" are geometric parameters, and "parallel" and "coaxial" are constraint relationships. Coincidence in the constraint relationships means that two components completely coincide on a certain surface or point; parallelism means that specific planes or axes of two components are parallel to each other; perpendicularity means that specific planes or axes are perpendicular to each other; coaxiality means that the axes of two components are on the same straight line; internal tangency and external tangency respectively mean that one component is tangent to another component internally or externally; the inclination angle defines the angular relationship between two components. For the specific construction process of the fusion model based on geometric parameters and constraint relationships, please refer to Step S41 to Step S46.

[0064] Step S41: Based on the geometric parameters, perform a first geometric transformation on the first component and the second component. The first geometric transformation operations include a scaling operation and a mirroring operation.

[0065] Specifically, in this embodiment, the size of the component can be adjusted through a scaling operation according to the geometric parameters to meet the size requirements for fusion: If the geometric parameters of the first component indicate that its size is too large and it needs to be reduced to match the second component, then the coordinates of each point of the first component are scaled according to a certain ratio; assuming that the geometric parameter of the first component is that its length needs to be scaled to 0.5 times the original, then in three-dimensional space, for each vertex coordinate (x, y, z) of the first component, the new coordinate becomes (0.5x, 0.5y, 0.5z). Additionally, in some cases, according to the attribute description or design requirements, a mirroring operation can be performed on the component to make its orientation or shape in space adapt to that of another component. If it is necessary to perform a mirroring operation on the first component along the x-axis, for each vertex coordinate (x, y, z) of the component, the new coordinate becomes (-x, y, z). The above scaling operation and mirroring operation are both the first geometric transformation operations.

[0066] Step S42: Determine the first splicing area of the first component and the second splicing area of the second component.

[0067] Specifically, the first splicing area is the connection head of the first component, and the second splicing area is the connection head of the second component. In the embodiments of this application, the first splicing area of the first component and the second splicing area of the second component can be determined according to the shape and attribute description of the components; these areas can be the parts on the components used for connection, and the shape can be a plane, a curved surface, or a specific geometric shape.

[0068] Step S43: Splice the first component and the second component based on the constraint relationship.

[0069] Specifically, the first component and the second component are spliced by making the centroid of the first splicing region coincide with the centroid of the second splicing region. Further, for the above-mentioned cup body and cup lid, if the attribute description specifies that the bottom surface of the cup lid is connected to one surface of the cup body, then the bottom surface of the cup lid is the first splicing region, and the corresponding surface of the cup body is the second splicing region. Assuming that the two components are in an inclined state, the two components can be spliced according to the inclination angle so that the centroid of the first splicing region coincides with the centroid of the second splicing region; the centroid is the central position of a geometric shape. By making the centroids coincide, the symmetry and stability of the component splicing can be ensured to a certain extent. During the implementation process, the centroid coordinates of the first splicing region and the second splicing region can be calculated first; assuming that the centroid coordinates of the first splicing region are (x1, y1, z1) and the centroid coordinates of the second splicing region are (x2, y2, z2), the second component is moved along the vector (x1 - x2, y1 - y2, z1 - z2) through translation operations to make the two centroids coincide.

[0070] Step S44: Based on the overlapping region between the first splicing region and the second splicing region, evaluate the similarity between the first splicing region and the second splicing region to obtain a similarity evaluation result.

[0071] Specifically, the overlapping region includes the contact area and the overlapping volume, and the similarity is the ratio of the contact area to the overlapping volume. In the embodiments of the present application, determining the overlapping region between the first splicing region and the second splicing region includes the contact area and the overlapping volume. For planar splicing regions, calculate the contact area; for three-dimensional components, the overlapping volume also needs to be calculated; if the two splicing regions are planar figures, the contact area can be calculated by methods such as the triangle area formula and the polygon area formula; for the overlapping volume of three-dimensional components, if the component shape is regular, the corresponding volume formula can be used; if the shape is irregular, the method of first voxelizing and then counting the number of overlapping voxels can be used to estimate the overlapping volume. In addition, when calculating the similarity, the ratio of the contact area to the overlapping volume can be used as the similarity evaluation index. The higher this ratio is, the better the matching degree of the two splicing regions is. Assuming that the contact area is S and the overlapping volume is V, the similarity Sim = S / V.

[0072] Step S45: According to the similarity evaluation result, perform a second geometric transformation operation on the first component and the second component and determine a set of adjustment parameters.

[0073] Specifically, in this embodiment, the position of the component can be finely adjusted through translation operations to further optimize the coincidence degree of the splicing area, thereby improving the similarity. In three-dimensional space, the first component or the second component is translated along the x, y, and z axes, each time by a small distance Δx, Δy, Δz, and the similarity is recalculated, and the similarity values at different translation distances are recorded. During the rotation operation, the component can be rotated around the mutually perpendicular x, y, and z axes to change the spatial direction of the component, so that the splicing areas coincide better; for rotation around the x axis, the rotation matrix can be used, where θ is the rotation angle. The coordinates (x, y, z) of each vertex of the component are transformed as follows: Similarly, for rotation around the y axis and z axis, the corresponding rotation matrices are also used for operations. Each time it is rotated by 1°, the similarity is recalculated, and the similarity values at different rotation angles are recorded. When determining the adjustment parameter group, the similarity values under different combinations of translation distances and rotation angles are recorded to form an adjustment parameter group, and each parameter group corresponds to a similarity evaluation result. The translation operation and the rotation operation are both the second geometric transformation operations.

[0074] Step S46: Select the target adjustment parameters from the adjustment parameter group, and fuse the first component and the second component according to the target adjustment parameters to obtain a fused model.

[0075] Specifically, the optimal adjustment parameters are selected from the adjustment parameter group to fuse the first component and the second component to maximize the similarity and obtain a fused model; the optimal adjustment parameters include the optimal translation distance and the optimal rotation angle. In the embodiments of the present application, the parameter combination that maximizes the similarity is selected from the adjustment parameter group, that is, the optimal adjustment parameters, including the optimal translation distance and the optimal rotation angle; these parameters are used to perform the final geometric transformation on the first component and the second component to obtain a fused model. In implementation, the adjustment parameter group can be traversed to find the parameter combination with the largest similarity value; if the optimal translation distance is (ΔXopt, ΔYopt, ΔZopt), and the optimal rotation angles are θ around the x axis X,opt , θ around the y axis Y,opt and θ around the z axis Z,opt , the corresponding translation and rotation operations are performed on the component to complete the fusion of the component and obtain a fused model.

[0076] Step S105: Based on the color information and the pattern information, perform rendering optimization on the fused model to generate the target three-dimensional content.

[0077] Specifically, the first rendering optimization and the second rendering optimization are performed on the fused model to generate the target three-dimensional content. The first rendering optimization is used to adjust the color information and the pattern information in the fused model, and the second rendering optimization is used to perform smooth connection processing on the overlapping area. Please refer to steps S51 to S52 for the rendering optimization process.

[0078] Step S51, adjust the color or pattern on the surfaces of the first component and the second component in the fusion model according to the color and pattern information in the attribute description.

[0079] Specifically, adjusting the surfaces of the first component and the second component in the fusion model aims to make the generated 3D content more in line with the user's expectations or design requirements; color and pattern are important elements that endow 3D models with visual features. By changing these elements, the expressiveness and aesthetics of the 3D content can be enhanced. For color, the surface material properties of the component can be adjusted to make it have different hues, saturations, and lightness; for pattern, textures or pattern mappings can be added to the surface of the component to make the model more detailed and distinctive.

[0080] Among them, color and pattern are achieved through material properties; the material determines how light interacts with the object surface. Different colors correspond to different diffuse and ambient light reflection characteristics; physically based rendering (PBR) can be used. The color of the material affects the reflection and absorption of light, thus affecting the color of the object we see; for patterns, texture mapping technology can be used to map a 2D texture image onto the surface of a 3D component. According to the texture coordinates, the pixels of the texture are mapped to the corresponding positions on the surface of the component.

[0081] In color adjustment, first, parse the color information in the attribute description; if the attribute description mentions that "the cup body is blue and the cup handle is yellow", for the first component (cup body), rendering engines such as OpenGL and DirectX or 3D modeling software such as Blender and 3ds Max can be used to find the corresponding material property settings; in physically based rendering, the diffuse color of the first component can be set to blue; the appropriate blue can also be selected by adjusting the RGB values of the material or using a color picker; for the material of the cup body, its diffuse color can be set to (0, 0, 255) to represent blue. For the second component (cup handle), its diffuse color is also set to yellow, such as (255, 255, 0); at the same time, according to different rendering engines, other color properties such as specular color and ambient color can be adjusted to enhance the color hierarchy and realism; setting the specular color can make the component produce different reflective effects in the specular area, and setting the ambient color can make the component present different hues under ambient light.

[0082] In pattern adjustment, if the attribute description contains pattern information, such as "the cup body has a flower pattern", it is first necessary to obtain the corresponding texture image of the flower pattern; this image can be a two-dimensional picture file (such as PNG, JPEG, etc.); then, add the texture image to the surface of the first component (the cup body) by means of texture mapping. In 3D modeling software, map the texture image to the surface coordinates of the cup body; for complex surfaces, the UV unwrapping method can be used to unfold the 3D surface into a 2D plane for more accurate texture mapping; according to the texture coordinates, map the pixels of the image to the corresponding positions on the surface of the cup body. For an irregularly shaped cup body, ensure that the stretching and deformation of the texture meet the expectations and avoid pattern distortion. For different rendering engines, the attributes of the texture can be adjusted, such as the repetition mode of the texture (whether to tile, how to tile), the offset and rotation of the texture, to achieve different visual effects; OpenGL can be used to implement texture mapping and sampling through texture coordinates and glTexCoord, glTexImage2D texture sampling functions.

[0083] In step S52, perform a smoothing connection process on the overlapping area of the first component and the second component to obtain the target 3D content with seamless fusion.

[0084] Specifically, when fusing the first component and the second component, in order to avoid unevenness or discontinuity problems in the overlapping area, which affect the quality and realism of the final 3D content; through the smoothing connection process, these discontinuities can be eliminated to make the transition between components more natural and achieve the effect of seamless fusion. Geometrically, the vertices of the overlapping area can be resampled and adjusted to make the surface smoother; in rendering, algorithms such as bilinear interpolation and bicubic interpolation can be used to make the texture and lighting smoothly transition in the overlapping area; for the lighting effect, ensure that the lighting calculations of adjacent components are consistent in the overlapping area to avoid obvious shadows or lighting mutations.

[0085] During the implementation process, the geometric shape can be adjusted. First, the overlapping area of the first component and the second component can be determined through spatial geometric calculations or marking information; if the first component and the second component are fused by splicing, the splicing area is the overlapping area; for the vertices of the overlapping area, a smoothing algorithm can be used for processing; Laplacian smoothing can also be used, which can adjust the position of the current vertex according to the adjacent vertices around the vertex to make it move towards the average position of the adjacent vertices. For example, for a vertex V and its adjacent vertices V1, V2, V3,..., V n , the new vertex position V' can be obtained through Calculation is performed, where λ is a smoothing factor that controls the degree of smoothing. A subdivision algorithm can also be used to subdivide the faces in the overlapping region, adding more vertices to make the surface smoother; for a triangular face, it can be subdivided into multiple smaller triangles, and then the new vertices are repositioned to achieve a smoothing effect.

[0086] During rendering adjustment, global illumination algorithms (such as ray tracing, radiosity algorithm) can be used to ensure that when calculating illumination in the overlapping region, adjacent components share the same illumination information. When using the ray tracing algorithm, rays are emitted from points in the overlapping region to calculate the intersection points of the rays with the scene and the illumination contribution, avoiding the situation of discontinuous illumination between adjacent components; for textures, an interpolation algorithm can be used for transition in the overlapping region; for texture coordinates, bilinear interpolation or bicubic interpolation can be used to make the texture smoothly transition in the overlapping region. In texture mapping, for a point P in the overlapping region, its texture coordinates are (u1, v1) in the texture mapping of the first component and (u2, v2) in the texture mapping of the second component. Bilinear interpolation u = (1 - t)*u1 + t*u2, v = (1 - t)*v1 + t*v2 can be used to obtain smoothly transitioned texture coordinates, where t is an interpolation factor between 0 and 1. In shadow processing, shadow mapping or ambient occlusion can be used to ensure consistent shadows in the overlapping region; in shadow mapping, ensure accurate depth information in the overlapping region to avoid the situation where part has a shadow while another part does not, making the shadow smoothly transition in the overlapping region.

[0087] By adopting the above methods, this application obtains the attribute description information input by the user; inputs the attribute description information into the attribute extraction model for feature extraction to output the attribute description corresponding to the target 3D content; inputs the attribute description into the component generator to output the first component and the second component used to construct the target 3D content; based on geometric parameters and constraint relationships, fuses and optimizes the first component and the second component to obtain a fusion model; based on color information and pattern information, renders and optimizes the fusion model to generate the target 3D content, thereby realizing a fully automatic modeling process from natural language input to 3D model output, significantly improving the generation efficiency of 3D content, and meeting the diverse and personalized customization requirements of users for 3D content.

[0088] Please refer to Figure 2 , which shows a schematic diagram of the modules of a 3D content generation device for a cultural and tourism digitalization scenario provided by an embodiment of this application. The device includes an acquisition module 21, a feature extraction module 22, a component generation module 23, a hybrid optimization module 24, and a rendering optimization module 25.

[0089] Among them, the acquisition module 21 is configured to, in response to an operation by a user to create target three-dimensional content, acquire attribute description information input by the user, where the attribute description information includes the types of components required for the target three-dimensional content, the number of components, the color of the components, the material of the components, and the geometric dimensions of the components.

[0090] The feature extraction module 22 is configured to input the attribute description information into an attribute extraction model for feature extraction, so as to output an attribute description corresponding to the target three-dimensional content, where the attribute description includes geometric parameters, constraint relationships, color information, and pattern information.

[0091] The component generation module 23 inputs the attribute description into a component generator, so as to output target components for constructing the target three-dimensional content, where the target components include first components and second components.

[0092] The hybrid optimization module 24 performs fusion optimization on the first components and the second components based on the geometric parameters and the constraint relationships, so as to obtain a fusion model.

[0093] The rendering optimization module 25 performs rendering optimization on the fusion model based on the color information and the pattern information, so as to generate the target three-dimensional content.

[0094] In a possible implementation manner, before the feature extraction module 22 inputs the attribute description information into an attribute extraction model for feature extraction to output an attribute description corresponding to the target three-dimensional content, it is necessary to construct an attribute extraction model, which specifically includes: constructing an attribute pre-extraction model, where the attribute pre-extraction model includes a preprocessing module and a bidirectional long short-term memory network; inputting the attribute description information into the preprocessing module to obtain a description sequence; and training the bidirectional long short-term memory network through the description sequence to construct the attribute extraction model.

[0095] In a possible implementation manner, when the attribute description information input by the feature extraction module 22 includes voice description information and text description information, and the attribute description information is input into the preprocessing module to obtain a description sequence, it specifically includes: extracting mel-frequency cepstral coefficients in the voice description information through the preprocessing module and segmenting them according to a preset time window to form a voice sequence; constructing a vocabulary table, and mapping the text description information into a digital sequence through the preprocessing module based on the vocabulary table; and using the voice sequence and the digital sequence as the description sequence.

[0096] In a possible implementation manner, the component generation module 23 is used to construct a component generator before inputting the attribute description into the component generator to output a target component for constructing the target three-dimensional content. Specifically, it includes: constructing a three-dimensional model set, where the three-dimensional model set includes a plurality of standard three-dimensional models and a plurality of standard three-dimensional descriptions. Among them, one standard three-dimensional model corresponds to one standard three-dimensional description; adding first noise to the standard three-dimensional model to obtain a noisy point cloud, where the first noise is obtained by sampling from a standard normal distribution; constructing a diffusion model, where the diffusion model includes a three-dimensional description extraction structure, a point cloud description extraction structure, a time description extraction structure, a conversion network, and a predicted noise structure; inputting the standard three-dimensional description, the noisy point cloud, and the diffusion time into the diffusion model to output second noise; calculating a training loss based on the first noise and the second noise, and updating the parameters of the diffusion model according to the training loss to construct the component generator.

[0097] In a possible implementation manner, the component generation module 23 is used to input the standard three-dimensional description, the noisy point cloud, and the diffusion time into the diffusion model to output second noise. Specifically, it includes: inputting the attribute description of the standard three-dimensional description into the corresponding three-dimensional description extraction structure to obtain a three-dimensional description block, where the three-dimensional description block contains the representation of the attribute description of the standard three-dimensional description in a high-dimensional space; inputting the noisy point cloud into the point cloud description extraction structure to obtain a point cloud description block, where the point cloud description block is used to reflect the characteristics and distribution of the noisy point cloud; inputting the diffusion time into the time description extraction structure to obtain a time description block, where the time description block is used to store the information of the current diffusion time; inputting the three-dimensional description block, the point cloud description block, and the time description block into the conversion network and outputting a conversion description; inputting the conversion description into the predicted noise structure in the diffusion model to output second noise.

[0098] In a possible implementation, the hybrid optimization module 24 is used to perform fusion optimization on the first component and the second component based on geometric parameters and constraint relationships to obtain a fusion model, specifically including: performing a first geometric transformation on the first component and the second component based on geometric parameters, and the first geometric transformation operation includes a scaling operation and a mirror operation; determining a first splicing area of the first component and a second splicing area of the second component, where the first splicing area is the connection head of the first component and the second splicing area is the connection head of the second component; splicing the first component and the second component based on the constraint relationship, and the constraint relationship is to make the centroid of the first splicing area coincide with the centroid of the second splicing area; evaluating the similarity between the first splicing area and the second splicing area based on the overlapping area between the first splicing area and the second splicing area to obtain a similarity evaluation result, and the overlapping area includes the contact area and the overlapping volume between the first splicing area and the second splicing area; performing a second geometric transformation operation on the first component and the second component according to the similarity evaluation result, and determining a set of adjustment parameters, where the second geometric transformation operation includes a translation operation and a rotation operation; selecting a target adjustment parameter from the set of adjustment parameters, and fusing the first component and the second component according to the target adjustment parameter to obtain a fusion model.

[0099] In a possible implementation, the rendering optimization module 25 is used to perform rendering optimization on the fusion model based on color information and pattern information to generate target three-dimensional content, specifically including: performing a first rendering optimization and a second rendering optimization on the fusion model to generate target three-dimensional content, where the first rendering optimization is used to adjust the color information and pattern information in the fusion model, and the second rendering optimization is used to perform a smooth connection process on the overlapping area.

[0100] It should be noted that: when the device provided in the above embodiment realizes its functions, only the above-mentioned division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process can be seen in the method embodiment, which will not be repeated here.

[0101] This application also provides an electronic device. Refer to Figure 3 , Figure 3 which is a schematic structural diagram of an electronic device provided in an embodiment of this application. The electronic device may include: at least one processor 301, at least one communication bus 302, a user interface 303, at least one network interface 304, and a memory 305.

[0102] Among them, the communication bus 302 is used to realize the connection and communication between these components.

[0103] Among them, the user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may further include standard wired interfaces and wireless interfaces.

[0104] Among them, the network interface 304 may optionally include standard wired interfaces and wireless interfaces (such as WI-FI interfaces).

[0105] Among them, the processor 301 may include one or more processing cores. The processor 301 connects various parts within the entire server using various interfaces and circuits. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and by calling the data stored in the memory 305, it performs various functions of the server and processes data. Optionally, the processor 301 may be implemented in at least one of the hardware forms of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 301 may integrate one or several combinations of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301 and may be implemented separately through a single chip.

[0106] Among them, the memory 305 may include random access memory (RAM) and may also include read-only memory. Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store the data involved in the above-mentioned method embodiments. Optionally, the memory 305 may further be at least one storage device located far from the aforementioned processor 301. Refer to Figure 3In the memory 305, which is a computer storage medium, an operating system, a network communication module, a user interface module, and a three-dimensional content generation application program for the cultural and tourism digital scenario may be included.

[0107] In Figure 3 In the electronic device shown, the user interface 303 is mainly used to provide an interface for the user to obtain the data input by the user; and the processor 301 can be used to call the three-dimensional content generation application program stored in the memory 305. When executed by one or more processors 301, the electronic device is caused to execute one or more of the methods as described in the above embodiments. It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0108] This application also provides a computer-readable storage medium, which stores instructions. When executed by one or more processors, the electronic device is caused to execute one or more of the methods as described in the above embodiments.

[0109] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0110] In several implementation manners provided by this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some service interfaces. The indirect couplings or communication connections of the devices or units can be in an electrical or other form.

[0111] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0112] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0113] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present application. The aforementioned memory includes: various media such as USB flash drives, mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0114] The above are only exemplary embodiments disclosed in the present application and should not be used to limit the scope of the disclosure of the present application. That is, any equivalent changes and modifications made in accordance with the teachings disclosed in the present application still fall within the scope covered by the disclosure of the present application.

[0115] The present application aims to cover any variations, uses, or adaptive changes of the present application, and these variations, uses, or adaptive changes follow the general principles disclosed in the present application and include common general knowledge or conventional technical means in the technical field not recorded in the present application.

Claims

1. A method for generating three-dimensional content in a digital cultural and tourism scenario, characterized in that, The method includes: In response to an operation by the user to create target three-dimensional content, obtain the attribute description information input by the user, where the attribute description information includes the types of components required for the target three-dimensional content, the number of components, the color of the components, the material of the components, and the geometric dimensions of the components; Input the attribute description information into an attribute extraction model for feature extraction to output the attribute description corresponding to the target three-dimensional content, where the attribute description includes geometric parameters, constraint relationships, color information, and pattern information; Input the attribute description into a component generator to output target components for constructing the target three-dimensional content, where the target components include a first component and a second component; Based on the geometric parameters and the constraint relationships, perform fusion optimization on the first component and the second component to obtain a fusion model; Based on the color information and the pattern information, perform rendering optimization on the fusion model to generate the target three-dimensional content.

2. The method according to claim 1, characterized in that Before inputting the attribute description information into the attribute extraction model for feature extraction to output the attribute description corresponding to the target three-dimensional content, it is necessary to construct the attribute extraction model, specifically including: Construct an attribute pre-extraction model, where the attribute pre-extraction model includes a preprocessing module and a bidirectional long short-term memory network; Input the attribute description information into the preprocessing module to obtain a description sequence; Train the bidirectional long short-term memory network through the description sequence to construct the attribute extraction model.

3. The method according to claim 2, wherein The attribute description information includes speech description information and text description information. The inputting the attribute description information into the preprocessing module to obtain a description sequence specifically includes: Extract the Mel-frequency cepstral coefficients from the speech description information through the preprocessing module and segment them according to a preset time window to form a speech sequence; Construct a vocabulary, and based on the vocabulary, map the text description information into a digital sequence through the preprocessing module; Use the speech sequence and the digital sequence as the description sequence.

4. The method according to claim 1, wherein Before inputting the attribute description into the component generator to output target components for constructing the target three-dimensional content, it is also necessary to construct the component generator, specifically including: Construct a three-dimensional model set, where the three-dimensional model set includes a plurality of standard three-dimensional models and a plurality of standard three-dimensional descriptions, and one of the standard three-dimensional models corresponds to one of the standard three-dimensional descriptions; Add a first noise to the standard three-dimensional model to obtain a noisy point cloud, where the first noise is obtained by sampling from a standard normal distribution; Construct a diffusion model, where the diffusion model includes a three-dimensional description extraction structure, a point cloud description extraction structure, a time description extraction structure, a conversion network, and a predicted noise structure; Input the standard three-dimensional description, the noisy point cloud, and the diffusion time into the diffusion model to output a second noise; Calculate a training loss based on the first noise and the second noise, and update the parameters of the diffusion model according to the training loss to construct the component generator.

5. The method according to claim 4, characterized in that Inputting the standard three-dimensional description, the noise point cloud, and the diffusion time into the diffusion model to output a second noise specifically includes: Inputting the attribute description of the standard three-dimensional description into the corresponding three-dimensional description extraction structure to obtain a three-dimensional description block, where the three-dimensional description block contains the representation of the attribute description of the standard three-dimensional description in a high-dimensional space; Inputting the noise point cloud into the point cloud description extraction structure to obtain a point cloud description block, where the point cloud description block is used to reflect the characteristics and distribution of the noise point cloud; Inputting the diffusion time into the time description extraction structure to obtain a time description block, where the time description block is used to store the information of the current diffusion time; Inputting the three-dimensional description block, the point cloud description block, and the time description block into the conversion network and outputting a conversion description; Inputting the conversion description into the prediction noise structure in the diffusion model to output the second noise.

6. The method according to claim 1, wherein Based on the geometric parameters and the constraint relationship, performing fusion optimization on the first component and the second component to obtain a fusion model, specifically including: Performing a first geometric transformation on the first component and the second component based on geometric parameters, where the first geometric transformation operation includes a scaling operation and a mirror operation; Determining a first splicing area of the first component and a second splicing area of the second component, where the first splicing area is the connection head of the first component and the second splicing area is the connection head of the second component; Based on the constraint relationship, splicing the first component and the second component, where the constraint relationship is to make the centroid of the first splicing area coincide with the centroid of the second splicing area; Based on the overlapping area of the first splicing area and the second splicing area, evaluating the similarity between the first splicing area and the second splicing area to obtain a similarity evaluation result, where the overlapping area includes the contact area and the overlapping volume of the first splicing area and the second splicing area; According to the similarity evaluation result, performing a second geometric transformation operation on the first component and the second component and determining a set of adjustment parameters, where the second geometric transformation operation includes a translation operation and a rotation operation; Selecting a target adjustment parameter from the set of adjustment parameters and fusing the first component and the second component according to the target adjustment parameter to obtain the fusion model.

7. The method according to claim 6, wherein Based on the color information and the pattern information, performing rendering optimization on the fusion model to generate the target three-dimensional content, specifically including: Performing a first rendering optimization and a second rendering optimization on the fusion model to generate the target three-dimensional content, where the first rendering optimization is used to adjust the color information and the pattern information in the fusion model, and the second rendering optimization is used to perform a smooth connection process on the overlapping area.

8. A three-dimensional content generation device for a cultural and tourism digital scenario, characterized in that, The device includes an acquisition module, a feature extraction module, a component generation module, a hybrid optimization module, and a rendering optimization module, where The acquisition module is configured to acquire the attribute description information input by the user in response to an operation of the user to create target three-dimensional content, where the attribute description information includes the types of components, the number of components, the colors of components, the materials of components, and the geometric dimensions of components required for the target three-dimensional content; The feature extraction module is configured to input the attribute description information into an attribute extraction model for feature extraction to output an attribute description corresponding to the target three-dimensional content, where the attribute description includes geometric parameters, constraint relationships, color information, and pattern information; The component generation module is configured to input the attribute description into a component generator to output target components for constructing the target three-dimensional content, where the target components include first components and second components; The hybrid optimization module is configured to perform fusion optimization on the first components and the second components based on the geometric parameters and the constraint relationships to obtain a fusion model; The rendering optimization module is configured to perform rendering optimization on the fusion model based on the color information and the pattern information to generate the target three-dimensional content.

9. An electronic device, characterized in that, It includes a processor, a communication bus, a user interface, a network interface, and a memory. The memory is used to store instructions. The user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, execute the method according to any one of claims 1 to 7.

Images