Three-dimensional memory carrier generation method and device

Abstract

The invention provides a three-dimensional memory carrier generation method and device, and the method comprises the steps: generating a first style feature vector and a first semantic feature vector based on a first scene description text inputted by a target user, and extracting first keyword information based on a first to-be-memorized text inputted by the target user, inputting the first style feature vector and the first keyword information into a first combined neural network model to obtain first stylized keyword information output by the first combined neural network model, generating a first memory carrier model based on the first stylized keyword information, generating a first three-dimensional geometric scene model based on the first semantic feature vector, and generating a second three-dimensional geometric scene model based on the second semantic feature vector. And on the basis of the first semantic feature vector, the first memory carrier modeling and the first three-dimensional geometric scene modeling, the first memory carrier set modeling is generated, so that the matching degree between the memory carrier set modeling and the to-be-memorized text is improved to a certain extent, and the usability of the memory carrier geometric model is improved.

Description

Technical Field

[0001] The present application relates to the field of model processing technology, and in particular to a method and device for generating a three-dimensional memory carrier. Background Art

[0002] The Memory Palace method is a memory method that originated in ancient Rome. Its principle is to establish a connection between the memory content and the imaginary buildings, rooms and interiors, and to quickly and orderly recall the original content by memorizing the building, internal objects and positions. Thousands of years of practice have shown that the Memory Palace method is a very effective means of auxiliary memory. In modern times, this method has a very wide range of uses. In addition to daily memorization content, memory competition contestants use this method to quickly memorize large amounts of information. In addition, memory-impaired patients and the elderly can also use it for cognitive rehabilitation. However, this method has high requirements on the user's imagination, which makes it difficult for ordinary users to master it.

[0003] In the related art, two-dimensional graphics and three-dimensional models are mainly used to assist users in reducing the difficulty of imagining three-dimensional space. For example, users can build their own memory space, select scenes and memory carriers, and then make associations. In addition, memory training can be provided for users, and statistical and testing functions can be provided.

[0004] However, in this method, the memory carrier is often quite different from the content to be remembered, the accuracy and availability of the memory carrier are not high, and the memory carrier is not very helpful to the user in memory. Summary of the invention

[0005] In view of the above problems, embodiments of the present application provide a three-dimensional memory carrier generation method, device, electronic device and readable storage medium to overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect of an embodiment of the present application, a method for generating a three-dimensional memory carrier is provided, the method comprising:

[0007] Based on the first scene description text input by the target user, a first style feature vector and a first semantic feature vector are generated, and based on the first text to be memorized input by the target user, first keyword information is extracted;

[0008] Inputting the first style feature vector and the first keyword information into a first combined neural network model to obtain first stylized keyword information output by the first combined neural network model; wherein the first combined neural network model is obtained by combining a convolutional neural network and a recurrent neural network;

[0009] Generate a first memory carrier model based on the first stylized keyword information, and generate a first three-dimensional geometric scene model based on the first semantic feature vector;

[0010] Based on the first semantic feature vector, the first memory carrier modeling and the first three-dimensional geometric scene modeling, a first memory carrier set modeling is generated.

[0011] Optionally, the generating a first style feature vector and a first semantic feature vector based on the first scene description text input by the target user includes:

[0012] Inputting a first scene description text input by a target user into a second combined neural network model to obtain a first style feature vector output by the second combined neural network model; wherein the second combined neural network model is obtained by combining a traditional bidirectional encoder representation model and a style transfer network model;

[0013] The first scene description text is input into a visual converter model to obtain a first semantic feature vector output by the visual converter model.

[0014] Optionally, the extracting first keyword information based on the first text to be memorized input by the target user includes:

[0015] Inputting the first to-be-remembered text input by the target user into the extractive summarization model to obtain first summary memory information output by the extractive summarization model;

[0016] Based on the attention weights between the word segments contained in the first summary memory information, the first summary information is optimized to obtain first keyword information.

[0017] Optionally, generating a first memory carrier model based on the first stylized keyword information includes:

[0018] generating a first virtual model based on the first stylized keyword information;

[0019] When the similarity between the first real model corresponding to the first memory text and the first scene description text and the first virtual model is greater than or equal to a first threshold, the first virtual model is determined to be modeled as the first memory carrier.

[0020] Optionally, generating a first three-dimensional geometric scene modeling based on the first semantic feature vector includes:

[0021] Inputting the first semantic feature vector into a conditional generative adversarial network model to obtain a geometric scene feature vector output by the conditional generative adversarial network model;

[0022] Geometric constraints are applied to the geometric scene feature vector to obtain a first three-dimensional geometric scene model.

[0023] Optionally, performing geometric constraints on the geometric scene feature vector to obtain first three-dimensional geometric scene modeling includes:

[0024] Inputting the first semantic feature vector into a conditional generative adversarial network model to obtain a confidence level output by the conditional generative adversarial network model;

[0025] When the confidence level is greater than or equal to the second threshold, geometric constraints are performed on the geometric scene feature vector to obtain a first three-dimensional geometric scene modeling.

[0026] Optionally, generating a first memory carrier set modeling based on the first semantic feature vector, the first memory carrier modeling and the first three-dimensional geometric scene modeling includes:

[0027] Based on the first interface where the entrance of the first three-dimensional geometric scene modeling is located, and the second interfaces corresponding to each of the first memory carriers, determine the first entrance connection surface contained in the first three-dimensional geometric scene modeling; wherein the first entrance connection surface means that the curve segment formed by connecting any point in the second interface excluding the boundary and any point in the first interface has no intersection with any surface of the first three-dimensional geometric scene modeling; all points contained in the line between any two points in the interface are on the same side of the interface, and the boundary of the interface is a closed curve;

[0028] Determine the midpoint of the line between the two points with the largest distance in each of the first inlet connecting surfaces as the memory carrier modeling rendering point;

[0029] Based on the first semantic feature vector, the first memory carrier modeling is rendered at the memory carrier modeling rendering point in the first three-dimensional geometric scene modeling to obtain a first three-dimensional memory carrier set modeling.

[0030] Optionally, determining the midpoint of the line between two points with the largest distance in each of the first inlet connecting surfaces as the memory carrier modeling rendering point includes:

[0031] In a case where the number of the first inlet connecting surfaces is less than the number of the first memory carrier modeling, determining an orthogonal plane corresponding to the first inlet connecting surface; wherein the orthogonal plane includes a line between two points with the largest distance in the first inlet connecting surface;

[0032] Based on the orthogonal plane, the first inlet communication surface is divided to obtain a plurality of second inlet communication surfaces;

[0033] When the number of the second inlet connecting surfaces is greater than or equal to the number of the first memory carrier modeling, the midpoint of the line between the two points with the largest distance in each of the second inlet connecting surfaces is determined as the memory carrier modeling rendering point.

[0034] Optionally, based on the first semantic feature vector, rendering the first memory carrier modeling at the memory carrier modeling rendering point to obtain a first three-dimensional memory carrier set modeling includes:

[0035] Determine a word vector sequence corresponding to the first semantic feature vector;

[0036] Based on the sequence of each word vector in the word vector sequence, the first memory carrier modeling corresponding to each word vector in the word vector sequence is rendered on the memory carrier modeling rendering point to obtain a first three-dimensional memory carrier set modeling.

[0037] In a second aspect, an embodiment of the present application provides a three-dimensional memory carrier generation device, the device comprising:

[0038] A generating and extracting module is used to generate a first style feature vector and a first semantic feature vector based on a first scene description text input by a target user, and to extract first keyword information based on a first text to be memorized input by the target user;

[0039] An input module, used for inputting the first style feature vector and the first keyword information into a first combined neural network model to obtain first stylized keyword information output by the first combined neural network model; wherein the first combined neural network model is obtained by combining a convolutional neural network and a recurrent neural network;

[0040] A first generating module, configured to generate a first memory carrier modeling based on the first stylized keyword information, and generate a first three-dimensional geometric scene modeling based on the first semantic feature vector;

[0041] The second generating module is used to generate a first memory carrier set modeling based on the first semantic feature vector, the first memory carrier modeling and the first three-dimensional geometric scene modeling.

[0042] Optionally, the generating and extracting module comprises:

[0043] A first input submodule, configured to input a first scene description text input by a target user into a second combined neural network model, and obtain a first style feature vector output by the second combined neural network model; wherein the second combined neural network model is obtained by combining a traditional bidirectional encoder representation model and a style transfer network model;

[0044] The second input submodule is used to input the first scene description text into the visual converter model to obtain a first semantic feature vector output by the visual converter model.

[0045] Optionally, the generating and extracting module comprises:

[0046] A third input submodule, configured to input the first text to be memorized input by the target user into the extractive summarization model, and obtain first summary memory information output by the extractive summarization model;

[0047] The fourth input submodule is used to optimize the first summary information based on the attention weights between the word segments contained in the first summary memory information to obtain the first keyword information.

[0048] Optionally, the first generating module includes:

[0049] A first generating submodule, configured to generate a first virtual model based on the first stylized keyword information;

[0050] The first determination submodule is used to determine the first virtual model as the first memory carrier modeling when the similarity between the first real model corresponding to the first memory text and the first scene description text and the first virtual model is greater than or equal to a first threshold.

[0051] Optionally, the first generating module includes:

[0052] A fifth input submodule, configured to input the first semantic feature vector into a conditional generative adversarial network model to obtain a geometric scene feature vector output by the conditional generative adversarial network model;

[0053] The geometric constraint submodule is used to geometrically constrain the geometric scene feature vector to obtain the first three-dimensional geometric scene modeling.

[0054] Optionally, the geometric constraint submodule includes:

[0055] A first input unit, used to input the first semantic feature vector into a conditional generative adversarial network model to obtain a confidence level output by the conditional generative adversarial network model;

[0056] The geometric constraint unit is used to perform geometric constraints on the geometric scene feature vector when the confidence level is greater than or equal to a second threshold value to obtain a first three-dimensional geometric scene modeling.

[0057] Optionally, the second generating module includes:

[0058] A second determination submodule is used to determine a first entrance connection surface included in the first three-dimensional geometric scene modeling based on a first interface where the entrance of the first three-dimensional geometric scene modeling is located, and second interfaces corresponding to each of the first memory carriers; wherein the first entrance connection surface means that a curve segment formed by connecting any point in the second interface excluding the boundary and any point in the first interface has no intersection with any surface of the first three-dimensional geometric scene modeling; all points included in the line between any two points in the interface are on the same side of the interface, and the boundary of the interface is a closed curve;

[0059] A third determination submodule is used to determine the midpoint of the line between the two points with the largest distance in each of the first inlet connection surfaces as a memory carrier modeling rendering point;

[0060] A rendering submodule is used to render the first memory carrier modeling at the memory carrier modeling rendering point in the first three-dimensional geometric scene modeling based on the first semantic feature vector to obtain a first three-dimensional memory carrier set modeling.

[0061] Optionally, the third determining submodule includes:

[0062] A first determining unit, configured to determine an orthogonal plane corresponding to the first inlet connecting surface when the number of the first inlet connecting surfaces is less than the number of the first memory carrier modeling; wherein the orthogonal plane includes a line between two points with the largest distance in the first inlet connecting surface;

[0063] a segmentation unit, configured to segment the first inlet communication surface based on the orthogonal plane to obtain a plurality of second inlet communication surfaces;

[0064] The second determining unit is used to determine the midpoint of the line between two points with the largest distance in each of the second inlet connecting surfaces as the memory carrier modeling rendering point when the number of the second inlet connecting surfaces is greater than or equal to the number of the first memory carrier modeling.

[0065] Optionally, the rendering submodule includes:

[0066] A third determining unit, used to determine a word vector sequence corresponding to the first semantic feature vector;

[0067] A rendering unit is used to render the first memory carrier modeling corresponding to each word vector in the word vector sequence at the memory carrier modeling rendering point based on the sequence of each word vector in the word vector sequence to obtain a first three-dimensional memory carrier set modeling.

[0068] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that the processor executes the computer program to implement a three-dimensional memory carrier generation method as described in any one of the above.

[0069] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, a three-dimensional memory carrier generation method as described in any one of the above is implemented.

[0070] The specific beneficial effects are:

[0071] The embodiment of the present application generates a first style feature vector and a first semantic feature vector based on a first scene description text input by a target user, extracts first keyword information based on a first text to be remembered input by the target user, inputs the first style feature vector and the first keyword information into a first combined neural network model, and obtains first stylized keyword information output by the first combined neural network model; wherein the first combined neural network model is obtained by combining a convolutional neural network, a recurrent neural network, and a generative adversarial network, generates first memory carrier modeling based on the first stylized keyword information, generates first three-dimensional geometric scene modeling based on the first semantic feature vector, generates first memory carrier set modeling based on the first semantic feature vector, the first memory carrier modeling, and the first three-dimensional geometric scene modeling, and can generate a corresponding first memory carrier geometric model according to the first scene description text input by the target user and the first text to be remembered, so as to assist the user in memorizing the first text to be remembered, improve the matching degree between the memory carrier set modeling and the text to be remembered to a certain extent, and improve the usability of the memory carrier geometric model. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0073] Figure 1 It is a flow chart of a method for generating a three-dimensional memory carrier provided in an embodiment of the present application;

[0074] Figure 2 It is a flowchart of another method for generating a three-dimensional memory carrier provided in an embodiment of the present application;

[0075] Figure 3 It is a flowchart of a method for generating a memory carrier provided in an embodiment of the present application;

[0076] Figure 4 It is a flowchart of a memory carrier set modeling generation method provided in an embodiment of the present application;

[0077] Figure 5 It is a flowchart of a specific implementation method of a three-dimensional memory carrier generation method provided in an embodiment of the present application;

[0078] Figure 6 It is a logic block diagram of a three-dimensional memory carrier generation device provided in an embodiment of the present application;

[0079] Figure 7 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0080] The exemplary embodiments of the present application will be described in more detail below in conjunction with the accompanying drawings in the embodiments of the present application. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to enable the scope of the present application to be fully communicated to those skilled in the art.

[0081] Reference Figure 1 , Figure 1 A schematic flow chart of a method for generating a three-dimensional memory carrier provided in an embodiment of the present application, the method comprising:

[0082] Step 101: Generate a first style feature vector and a first semantic feature vector based on a first scene description text input by a target user, and extract first keyword information based on a first text to be memorized input by the target user.

[0083] In an embodiment of the present application, the target user can input a first scene description text and a first text to be memorized into the control center, wherein the first scene description text can be used to limit the scene background of the memory carrier, and the first text to be memorized is the content text that the target user wants to memorize. The control center can generate a corresponding first style feature vector and a first semantic feature vector based on the first scene description text; and can also extract the first keyword information of the first text to be memorized. Among them, the first style feature vector can be obtained by inputting the first scene description text into a style feature extraction neural network model, such as a recurrent neural network (RNN), a long short-term memory network (LSTM), and a traditional bidirectional encoder representation model (BERT), or by inputting the first scene description text into a style feature extraction module in the above network; the first keyword information can be obtained by inputting the first text to be memorized into a text semantic segmentation model based on Transformer, or by inputting the first text to be memorized into a specific keyword extraction model, wherein the keyword library built into the keyword extraction model can be manually set and updated.

[0084] For example, the scene description text may be "empty palace", and its corresponding style feature is "empty" and the semantic feature is "palace"; if the text to be remembered is "after lighting dry firewood, a fire will occur, releasing light and heat at the same time", then the first keyword information may be "firewood, fire, light, heat".

[0085] Step 102: input the first style feature vector and the first keyword information into a first combined neural network model to obtain first stylized keyword information output by the first combined neural network model; wherein the first combined neural network model is obtained by combining a convolutional neural network and a recurrent neural network.

[0086] In an embodiment of the present application, the output ends of a convolutional neural network (CNN) and a recurrent neural network can be connected through a fully connected layer to form a fusion network, and the output end of the fusion network can be connected to the input end of a generative adversarial network (GAN) to form a first combined neural network model. After the first style feature vector and the first keyword information are input into the first combined neural network model, the convolutional neural network in the first combined neural network model can extract the visual information in the style feature vector, and the sequence information in the key content list can be processed through the recurrent neural network. After that, a fully connected layer can be introduced to connect the output ends of the convolutional neural network and the recurrent neural network, so that the first stylized keyword information can be output at the output end.

[0087] Continuing with the above example, if the style feature is "empty" and the first keyword information is "firewood, fire, light, heat", the first stylized keyword information finally obtained can be "firewood-empty, fire-empty, light-empty, heat-empty", where the symbol "-" is a connector.

[0088] Step 103: Generate a first memory carrier model based on the first stylized keyword information, and generate a first three-dimensional geometric scene model based on the first semantic feature vector.

[0089] In an embodiment of the present application, a first memory carrier modeling can be generated by various virtual reality (VR) software or augmented reality (AR) software according to the first stylized keyword information; similarly, a first three-dimensional geometric scene modeling can be generated by VR software or AR software according to the first semantic feature vector. Specifically, VR software and AR software can extract memory carrier rendering features from the first stylized keyword information and match them with the database in the software. When the corresponding data is matched, the data is rendered to form a first memory carrier, and can be displayed in the form of a model on the display interface. Similarly, VR software and AR software can extract three-dimensional geometric scene rendering features from the first semantic feature vector and match them with the database in the software. When the corresponding data is matched, the data is rendered to form a first three-dimensional geometric scene, and can be displayed in the form of a model on the display interface.

[0090] Step 104: Generate a first memory carrier set modeling based on the first semantic feature vector, the first memory carrier modeling and the first three-dimensional geometric scene modeling.

[0091] In an embodiment of the present application, a first memory carrier geometric model can be generated according to the first semantic feature vector, the first memory carrier modeling and the first three-dimensional geometric scene modeling. Specifically, the word sequence corresponding to the first semantic feature vector can be determined, and the correspondence between the first memory carrier modeling and the first semantic feature vector can be further determined according to the word sequence. After that, the first memory carrier modeling can be rendered in the first three-dimensional geometric scene modeling according to the correspondence, so that the first memory carrier geometric model can be obtained. Among them, the first memory carrier modeling and the first three-dimensional geometric scene modeling can be fused using a texture mapping algorithm to form the first memory carrier set modeling that finally carries the memory carrier.

[0092] In an embodiment of the present application, a first style feature vector and a first semantic feature vector are generated based on a first scene description text input by a target user, first keyword information is extracted based on a first text to be remembered input by the target user, and the first style feature vector and the first keyword information are input into a first combined neural network model to obtain first stylized keyword information output by the first combined neural network model; wherein the first combined neural network model is obtained by combining a convolutional neural network, a recurrent neural network and a generative adversarial network, and a first memory carrier modeling is generated based on the first stylized keyword information, a first three-dimensional geometric scene modeling is generated based on the first semantic feature vector, and a first memory carrier set modeling is generated based on the first semantic feature vector, the first memory carrier modeling and the first three-dimensional geometric scene modeling, and a corresponding first memory carrier geometric model can be generated according to the first scene description text input by the target user and the first text to be remembered, so as to assist the user in memorizing the first text to be remembered, and to a certain extent improve the matching degree between the memory carrier set modeling and the text to be remembered, and improve the availability of the memory carrier geometric model.

[0093] Reference Figure 2 , Figure 2 A schematic flow chart of another method for generating a three-dimensional memory carrier provided in an embodiment of the present application, the method may include:

[0094] Step 201 : generating a first style feature vector and a first semantic feature vector based on a first scene description text input by a target user, and determining to extract first keyword information based on a first text to be memorized input by the target user.

[0095] In the embodiments of the present application, the implementation content of this step can refer to the embodiment content of step 101, which will not be repeated here.

[0096] Optionally, step 201 may include the following sub-steps:

[0097] Sub-step 2011, inputting the first scene description text input by the target user into the second combined neural network model, and obtaining the first style feature vector output by the second combined neural network model; wherein the second combined neural network model is obtained by combining a traditional bidirectional encoder representation model and a style transfer network model.

[0098] In an embodiment of the present application, after receiving the first scene description text input by the target user, the first scene description text can be input into the second combined neural network model, so as to obtain the first style feature vector output by the second combined neural network model. Among them, the second combined neural network model can be obtained by combining a traditional bidirectional encoder representation model (BERT) and a style transfer network model. The style transfer network model can be composed of multiple fully connected layers, each of which is followed by a ReLU activation function. At the same time, in order to enhance the model's understanding of the text style, a style encoder can be introduced between the BERT model and the style transfer network. The style encoder can be adjusted and trained based on a variational autoencoder (VAE). All style features corresponding to the first scene description text can be included in the first style feature vector.

[0099] Sub-step 2012: input the first scene description text into a visual converter model to obtain a first semantic feature vector output by the visual converter model.

[0100] In an embodiment of the present application, the first scene description text may be input into a vision transformer model (ViT), so as to obtain a first semantic feature vector output by the vision transformer model. The first semantic feature vector may indicate a specific scene corresponding to the first scene description text, such as "building", "palace", "quadrangle", "grassland", etc.

[0101] In an embodiment of the present application, a first scene description text input by a target user is input into a second combined neural network model to obtain a first style feature vector output by the second combined neural network model; wherein the second combined neural network model is obtained by combining a traditional bidirectional encoder representation model and a style transfer network model, and the first scene description text is input into a visual converter model to obtain a first semantic feature vector output by the visual converter model. The first style feature vector and the first semantic feature vector corresponding to the first scene description text can be obtained respectively through the second combined neural network model and the visual converter model, thereby improving the accuracy and availability of the first style feature vector and the first semantic feature vector to a certain extent.

[0102] Sub-step 2013: inputting the first text to be memorized input by the target user into the extractive summarization model to obtain first summary memory information output by the extractive summarization model.

[0103] In an embodiment of the present application, the extractive summary model can be composed of a plurality of traditional recurrent neural network layers, and the output ends of the plurality of recurrent neural networks can be connected by a fully connected layer, and the activation function of the fully connected layer can be a ReLU function. After the first text to be memorized input by the target user is input into the extractive summary model, the first summary memory information output by the extractive summary model can be obtained.

[0104] Continuing with the above example, if the first text to be memorized is "After lighting dry firewood, a fire will occur, releasing light and heat at the same time", the first summary memory information may be "Lighting firewood will cause a fire, releasing light and heat".

[0105] Sub-step 2014: optimizing the first summary information based on the attention weights between the word segments contained in the first summary memory information to obtain first keyword information.

[0106] In an embodiment of the present application, an attention layer can be embedded at the output end of the extractive summary model. The attention layer is composed of multiple fully connected layers and can be used to calculate attention weights. These attention weights can then be applied to the initial summary to optimize the selection of summary information. The first summary memory information can contain multiple word segments, and these word segments can be obtained after the first summary memory information is segmented. At the same time, since the attention weights in the extractive summary model are introduced when generating the first summary memory information, the attention weights obtained by the extractive summary model can be obtained, and the first summary memory information can be optimized by the attention weights, so that the first keyword information can be obtained.

[0107] In an embodiment of the present application, the first text to be remembered input by the target user is input into the extractive summary model to obtain the first summary memory information output by the extractive summary model, and the first summary information is optimized based on the attention weights between each word segment contained in the first summary memory information to obtain the first keyword information. The first keyword information corresponding to the first text to be remembered can be obtained, thereby improving the matching degree between the first keyword information and the text to be remembered and improving the reliability of the first keyword information.

[0108] Step 202: Input the first style feature vector and the first keyword information into a first combined neural network model to obtain first stylized keyword information output by the first combined neural network model; wherein the first combined neural network model is obtained by combining a convolutional neural network and a recurrent neural network.

[0109] In the embodiments of the present application, the implementation content of this step can refer to the implementation content of step 102, which will not be repeated here.

[0110] Step 203: Generate a first memory carrier modeling based on the first stylized keyword information, and generate a first three-dimensional geometric scene modeling based on the first semantic feature vector.

[0111] In the embodiments of the present application, the implementation content of this step can refer to the implementation content of step 103, which will not be repeated here.

[0112] Optionally, step 203 may include the following sub-steps:

[0113] Sub-step 2031: generating a first virtual model based on the first stylized keyword information.

[0114] In an embodiment of the present application, the first stylized keyword information can be input into a generator network or a language model based on the generator network, so as to obtain a virtual model output by the generator network or the language model. The generator network or the language model based on the generator network can generate a corresponding first virtual model according to the first stylized keyword information.

[0115] Sub-step 2032, when the similarity between the first real model corresponding to the first memory text and the first scene description text and the first virtual model is greater than or equal to a first threshold, the first virtual model is determined to be modeled as the first memory carrier.

[0116] In an embodiment of the present application, the similarity between the first virtual model and the first real model can be determined by a discriminator. The first real model can correspond to the first memory text and the first scene description text, that is, the model expressed by the first memory text and the first scene description text; the discriminator can be composed of multiple convolutional layers, batch normalization layers, and Dropout layers, which are used to distinguish the generated virtual model and the real model by similarity. When the similarity is greater than or equal to the first threshold, the first virtual model can be determined as the first memory carrier modeling.

[0117] In an embodiment of the present application, a first virtual model is generated based on the first stylized keyword information, and the first virtual model is determined as the first memory carrier modeling when the similarity between the first real model corresponding to the first memory text and the first scene description text and the first virtual model is greater than or equal to a first threshold. The first virtual model can be generated according to the first stylized keyword information, and the first virtual model can be determined as the first memory carrier modeling when the similarity between the first virtual model and the first real model is greater than or equal to the first threshold, which improves the accuracy and reliability of the first memory carrier modeling to a certain extent.

[0118] Sub-step 2033: input the first semantic feature vector into a conditional generative adversarial network model to obtain a geometric scene feature vector output by the conditional generative adversarial network model.

[0119] In an embodiment of the present application, a conditional generative adversarial network model (cGAN) can be specifically optimized to adapt to the text-driven three-dimensional scene generation task. At the same time, in order to avoid generating incoherent geometric scenes, the conditional generative adversarial network model can be adjusted so that it can output a feature vector. On this basis, the first semantic feature vector can be input into the conditional generative adversarial network model, so that the geometric scene feature vector output by the conditional generative adversarial network model can be obtained.

[0120] Sub-step 2034, geometrically constraining the geometric scene feature vector to obtain a first three-dimensional geometric scene model.

[0121] In an embodiment of the present application, the geometric scene features contained in the geometric scene feature vector can together constitute a geometric scene. After obtaining the geometric scene feature vector, the geometric scene feature vector can be geometrically constrained, so as to obtain a coherent first three-dimensional geometric scene model. The geometric constraint can be a loss function F loss =Loss(L MSE ,R MSE ), where L MSE is the mean square error of the edge length, R MSE It is the mean square error of the angle deviation, so as to achieve the geometric connectivity and consistency in the predicted scene. The first three-dimensional geometric scene modeling is a triangular mesh composed of vertices and faces, which represents the shape of the three-dimensional space scene and is a three-dimensional space model.

[0122] Optionally, sub-step 2034 may include the following sub-steps:

[0123] Sub-step A1: input the first semantic feature vector into a conditional generative adversarial network model to obtain the confidence level of the output of the conditional generative adversarial network model.

[0124] In an embodiment of the present application, a discriminator network can be installed in the conditional generative adversarial network model. After the first semantic feature vector is input into the conditional generative adversarial network model, the discriminator network can output a confidence level, and the value of the confidence level can indicate the probability that the input data is the data corresponding to the true model.

[0125] Sub-step A2, when the confidence level is greater than or equal to a second threshold, geometrically constraining the geometric scene feature vector to obtain a first three-dimensional geometric scene model.

[0126] In an embodiment of the present application, if the confidence of the conditional generative adversarial network model output is greater than or equal to the second threshold, it can be considered that the first semantic feature vector is close to the real model, and the geometric scene feature vector can be geometrically constrained at this time, so that the first three-dimensional geometric scene modeling can be obtained. The specific implementation content of the geometric constraint can refer to the embodiment content of step 2034, which will not be repeated here.

[0127] In an embodiment of the present application, by inputting the first semantic feature vector into a conditional generative adversarial network model, the confidence of the output of the conditional generative adversarial network model is obtained, and when the confidence is greater than or equal to a second threshold, the geometric scene feature vector is geometrically constrained to obtain a first three-dimensional geometric scene modeling. When the confidence is greater than or equal to the second threshold, the geometric scene feature vector is geometrically constrained to obtain a first three-dimensional geometric scene modeling, which improves the similarity between the first three-dimensional geometric scene modeling and the real scene to a certain extent.

[0128] In an embodiment of the present application, a first semantic feature vector is input into a conditional generative adversarial network model to obtain a geometric scene feature vector output by the conditional generative adversarial network model, and geometric constraints are performed on the geometric scene feature vector to obtain a first three-dimensional geometric scene modeling, thereby avoiding modeling inconsistencies in the first three-dimensional geometric scene modeling and improving the reliability of the first three-dimensional geometric scene modeling.

[0129] Step 204, based on the first interface where the entrance of the first three-dimensional geometric scene modeling is located, and the second interfaces corresponding to each of the first memory carriers, determine the first entrance connecting surface contained in the first three-dimensional geometric scene modeling; wherein, the first entrance connecting surface represents the second interface that has no intersection with any surface of the first three-dimensional geometric scene modeling, and the points contained in the line between any two points in the interface are all on the same side of the interface, and the boundary of the interface is a closed curve.

[0130] In an embodiment of the present application, an interface refers to a plane or a curved surface with a boundary, that is, a surface with a fixed shape and size, and the boundaries of such planes or curved surfaces are all closed curves. In addition, all points contained in the line between any two points in the interface are on the same side of the interface. Among them, the first interface can be the surface where the entrance of the first three-dimensional geometric scene modeling is located, and the second interface can correspond to each first memory carrier respectively. Among them, if the interface intersects with the memory carrier, it can be considered that the interface corresponds to the memory carrier. After determining the first interface and the second interface, the geometric relationship between the first interface and the second interface can be determined, and then the first entrance connecting surface contained in the first three-dimensional geometric scene modeling can be determined according to the geometric relationship. Among them, the first entrance connecting surface can represent a second interface in which any point in the surface not including the boundary and any point in the first interface are connected to form a curve segment and any surface of the first three-dimensional geometric scene modeling without intersection.

[0131] For example, if there are three faces a, b, and c, any two points in faces a and c (the two points come from different faces) can be determined to be connected as long as there is a curve segment that does not pass through face b, and face a can be called a connected face of face c. When the line connecting any two points in faces a and c must pass through face b, then a and c can be considered disconnected. For example, if a is enclosed in a closed space by some faces, then a and c are disconnected, and a is not a connected face of c.

[0132] In a possible embodiment, the inter-surface relationship between each second interface can also be determined. The inter-surface relationship can include two surfaces sharing a common edge and two surfaces not sharing a common edge. When determining the first inlet connecting surface, since the second interfaces sharing two common edges can obtain the same first inlet connecting surface, one of the second interfaces sharing two common edges can be selected to participate in the determination process of the first inlet connecting surface.

[0133] Step 205: determine the midpoint of the line between the two points with the largest distance in each of the first inlet connecting surfaces as the memory carrier modeling rendering point.

[0134] In an embodiment of the present application, the midpoint of the line between two points with the largest distance in the first inlet connection surface can be determined as a memory carrier modeling rendering point, and the point position can be used to render the memory carrier. The distance between the two points can refer to the shortest connection path between the two points along the first inlet connection surface.

[0135] Optionally, step 205 may include the following sub-steps:

[0136] Sub-step 2051, when the number of the first inlet connecting surfaces is less than the number modeled by the first memory carrier, determine the orthogonal plane corresponding to the first inlet connecting surface; wherein the orthogonal plane includes a line between two points with the largest distance in the first inlet connecting surface.

[0137] In an embodiment of the present application, if the number of first inlet connecting surfaces is less than the number of first memory carrier modeling, the orthogonal planes corresponding to the first inlet connecting surfaces can be determined. The orthogonal planes corresponding to each first inlet connecting surface can be determined, or a portion of the first inlet connecting surfaces can be selected and then the orthogonal planes corresponding to the portion of the first inlet connecting surfaces can be determined. Among them, the orthogonal plane can include the line between the two points with the largest distance in the first inlet connecting surface. If the first inlet connecting surface is a plane, the corresponding orthogonal plane is perpendicular to the first inlet connecting surface.

[0138] Sub-step 2052: based on the orthogonal plane, split the first inlet connecting surface to obtain a plurality of second inlet connecting surfaces.

[0139] In the embodiment of the present application, the first inlet communication surface can be divided according to the orthogonal plane to obtain multiple second inlet communication surfaces. That is, the first inlet communication surface can be cut along the orthogonal plane direction to obtain multiple second inlet communication surfaces.

[0140] Sub-step 2053, when the number of the second inlet connecting surfaces is greater than or equal to the number of the first memory carrier modeling, the midpoint of the line connecting the two points with the largest distance in each of the second inlet connecting surfaces is determined as the memory carrier modeling rendering point.

[0141] In an embodiment of the present application, if the number of second inlet connecting surfaces is greater than or equal to the number of first memory carrier modeling, the midpoint of the line between the two points with the largest distance in each second inlet connecting surface can be determined as the memory carrier modeling rendering point. The distance between the two points can refer to the shortest connection path between the two points along the first inlet connecting surface. Obviously, the memory carrier modeling rendering points are all in the first three-dimensional geometric scene modeling.

[0142] In an embodiment of the present application, when the number of first inlet connecting surfaces is less than the number of first memory carrier modelings, an orthogonal plane corresponding to the first inlet connecting surface is determined; wherein the orthogonal plane includes a line between two points with the largest distance in the first inlet connecting surface; based on the orthogonal plane, the first inlet connecting surface is divided to obtain a plurality of second inlet connecting surfaces; when the number of second inlet connecting surfaces is greater than or equal to the number of first memory carrier modelings, the midpoint of the line between two points with the largest distance in each second inlet connecting surface is determined as a memory carrier modeling rendering point; when the number of first inlet connecting surfaces is less than the number of first memory carrier modelings, the first inlet connecting surface can be cut to obtain a plurality of second inlet connecting surfaces; and when the number of second inlet connecting surfaces is greater than or equal to the number of first memory carrier modelings, the memory carrier modeling rendering point can be determined, thereby ensuring that all first memory carrier modelings can have corresponding memory carrier modeling rendering points, thereby improving the reliability of the overall memory carrier modeling generation method.

[0143] Step 206: based on the first semantic feature vector, render the first memory carrier modeling at the memory carrier modeling rendering point in the first three-dimensional geometric scene modeling to obtain a first three-dimensional memory carrier set modeling.

[0144] In an embodiment of the present application, the first memory carrier modeling can be rendered at the memory carrier modeling rendering point in the first three-dimensional geometric scene modeling according to the first semantic feature vector, so as to obtain the first three-dimensional memory carrier set modeling. Specifically, the to-be-memorized feature contained in the first semantic feature vector can be obtained, and the first memory carrier modeling corresponding to the to-be-memorized feature can be rendered at the corresponding memory carrier modeling rendering point, so as to obtain the first three-dimensional memory carrier set modeling.

[0145] Optionally, step 206 may include the following sub-steps:

[0146] Sub-step 2061, determining the word vector sequence corresponding to the first semantic feature vector.

[0147] In an embodiment of the present application, the first semantic feature vector may include multiple word vectors. Since the first semantic feature vector is obtained through semantic recognition, these word vectors may have a certain arrangement order, thereby forming a word vector sequence. On this basis, the word vector sequence corresponding to the first semantic feature vector may be determined.

[0148] Sub-step 2062, based on the order of each word vector in the word vector sequence, render the first memory carrier modeling corresponding to each word vector in the word vector sequence at the memory carrier modeling rendering point to obtain a first three-dimensional memory carrier set modeling.

[0149] In an embodiment of the present application, the first memory carrier modeling corresponding to each word vector in the word vector sequence can be rendered on the memory carrier modeling rendering point according to the order of each word vector in the word vector sequence, so that the first three-dimensional memory carrier set modeling can be obtained.

[0150] In an embodiment of the present application, by determining a word vector sequence corresponding to a first semantic feature vector, based on the order of each word vector in the word vector sequence, the first memory carrier modeling corresponding to each word vector in the word vector sequence is rendered on a memory carrier modeling rendering point to obtain a first three-dimensional memory carrier set modeling. The first memory carrier modeling can be rendered on a memory carrier modeling rendering point according to the order of each word vector contained in the word vector sequence corresponding to the first semantic feature vector, thereby obtaining a first three-dimensional memory carrier set modeling, thereby improving the matching degree between the first three-dimensional memory carrier set modeling and the first text to be remembered input by the target user, and further improving the usability of the first three-dimensional memory carrier set modeling.

[0151] In an embodiment of the present application, the first entrance connecting surface contained in the first three-dimensional geometric scene modeling is determined by the first interface at which the entrance of the first three-dimensional geometric scene modeling is located, and the second interface corresponding to each of the first memory carriers; wherein the first entrance connecting surface indicates that the curve segment formed by connecting any point in the second interface excluding the boundary and any point in the first interface has no intersection with any surface of the first three-dimensional geometric scene modeling; all points contained in the line between any two points in the interface are on the same side of the interface, and the boundary of the interface is a closed curve, and the line connecting the two points with the largest distance in each of the first entrance connecting surfaces is The midpoint of is determined as the memory carrier modeling rendering point, and based on the first semantic feature vector, the first memory carrier modeling is rendered at the memory carrier modeling rendering point in the first three-dimensional geometric scene modeling to obtain a first three-dimensional memory carrier set modeling. The first entrance connecting surface can be obtained according to the geometric relationship between the first interface and the second interface, and the memory carrier modeling rendering point can be determined according to the first entrance connecting surface, so that the first memory carrier modeling can be rendered at the memory carrier modeling rendering point to obtain the first three-dimensional memory carrier set modeling, which improves the matching degree of the first three-dimensional memory carrier set modeling with the first text to be remembered input by the target user, and improves the usability of the first three-dimensional memory carrier geometric model.

[0152] Reference Figure 3 , Figure 31 is a flow chart of a memory carrier generation method provided in an embodiment of the present application. In the figure, the memory carrier generation method can be divided into three steps, namely: text preprocessing, stylized content fusion and memory carrier image generation. Among them, in the text preprocessing step, a BERT basic model and a RNN model can be included. The variational autoencoder VAE and the style transfer network can be embedded in the BERT basic model to output the style feature vector when the scene description text is input. A Transformer-based attention layer can be added after the RNN model to output the key memory information when the text to be remembered is input; in the stylized content fusion step, the key memory information and the style feature vector can be input into the CNN network and the RNN network respectively, wherein the CNN network is used to extract the key memory information and the visual information in the style feature vector, and the RNN network is used to extract the sequence information in the key memory information. After that, the visual information and the sequence information can be input into the GAN network through a fully connected layer, so that the stylized key information output by the GAN network can be obtained; in the memory carrier image generation step, the stylized key information can be input into the cGAN network, and the geometric constraint can be performed through the geometric constraint function, so that the stylized image output by the cGAN network, that is, the memory carrier, can be obtained.

[0153] Reference Figure 4 , Figure 4 It is a flow chart of a memory carrier set modeling generation method provided by an embodiment of the present application. In the figure, the memory carrier set modeling generation method may include three steps, namely: text feature extraction, three-dimensional scene generation and image fusion. Among them, in the step of text feature extraction, the scene description text may be firstly segmented to obtain each word, and then each word may be position-encoded to obtain a word sequence of the scene description text, and then the word sequence and the memory carrier may be input into the extractive summary model together to obtain the stylized key information output by the extractive summary model; in the step of three-dimensional scene generation, a three-dimensional scene may be generated according to the stylized key information according to the VR software or AR software; in the scene of image fusion, the stylized key information and the three-dimensional scene may be fused by executing the texture mapping algorithm through the VGG model, so as to obtain the memory carrier set modeling. The memory carrier set modeling includes the three-dimensional scene and each memory carrier.

[0154] Reference Figure 5 , Figure 5This is a flowchart of a specific implementation of a three-dimensional memory carrier generation method provided by an embodiment of the present application. In the figure, the scene description text and the text to be memorized can be combined to generate a memory carrier modeling, and the scene description text can generate a three-dimensional scene modeling. After obtaining the three-dimensional scene modeling and the memory carrier modeling, rendering point planning can be performed to obtain the memory carrier rendering point in the three-dimensional scene modeling. After that, the memory carrier modeling is rendered on the memory carrier rendering point to obtain the memory carrier set modeling.

[0155] Reference Figure 6 , Figure 6 A logic block diagram of a three-dimensional memory carrier generating device provided in an embodiment of the present application, the three-dimensional memory carrier generating device 600 may include:

[0156] A generating and extracting module 601 is used to generate a first style feature vector and a first semantic feature vector based on a first scene description text input by a target user, and to extract first keyword information based on a first text to be memorized input by the target user;

[0157] An input module 602 is used to input the first style feature vector and the first keyword information into a first combined neural network model to obtain first stylized keyword information output by the first combined neural network model; wherein the first combined neural network model is obtained by combining a convolutional neural network and a recurrent neural network;

[0158] A first generating module 603 is used to generate a first memory carrier modeling based on the first stylized keyword information, and to generate a first three-dimensional geometric scene modeling based on the first semantic feature vector;

[0159] The second generating module 604 is used to generate a first memory carrier set modeling based on the first semantic feature vector, the first memory carrier modeling and the first three-dimensional geometric scene modeling.

[0160] Optionally, the generating and extracting module 601 includes:

[0161] A first input submodule, configured to input a first scene description text input by a target user into a second combined neural network model, and obtain a first style feature vector output by the second combined neural network model; wherein the second combined neural network model is obtained by combining a traditional bidirectional encoder representation model and a style transfer network model;

[0162] The second input submodule is used to input the first scene description text into the visual converter model to obtain a first semantic feature vector output by the visual converter model.

[0163] Optionally, the generating and extracting module 601 includes:

[0164] A third input submodule, configured to input the first text to be memorized input by the target user into the extractive summarization model, and obtain first summary memory information output by the extractive summarization model;

[0165] The fourth input submodule is used to optimize the first summary information based on the attention weights between the word segments contained in the first summary memory information to obtain the first keyword information.

[0166] Optionally, the first generating module 603 includes:

[0167] A first generating submodule, configured to generate a first virtual model based on the first stylized keyword information;

[0168] The first determination submodule is used to determine the first virtual model as the first memory carrier modeling when the similarity between the first real model corresponding to the first memory text and the first scene description text and the first virtual model is greater than or equal to a first threshold.

[0169] Optionally, the first generating module 603 includes:

[0170] A fifth input submodule, configured to input the first semantic feature vector into a conditional generative adversarial network model to obtain a geometric scene feature vector output by the conditional generative adversarial network model;

[0171] The geometric constraint submodule is used to geometrically constrain the geometric scene feature vector to obtain the first three-dimensional geometric scene modeling.

[0172] Optionally, the geometric constraint submodule includes:

[0173] A first input unit, used to input the first semantic feature vector into a conditional generative adversarial network model to obtain a confidence level output by the conditional generative adversarial network model;

[0174] The geometric constraint unit is used to perform geometric constraints on the geometric scene feature vector when the confidence level is greater than or equal to a second threshold value to obtain a first three-dimensional geometric scene modeling.

[0175] Optionally, the second generating module 604 includes:

[0176] A second determination submodule is used to determine a first entrance connection surface included in the first three-dimensional geometric scene modeling based on a first interface where the entrance of the first three-dimensional geometric scene modeling is located, and second interfaces corresponding to each of the first memory carriers; wherein the first entrance connection surface means that a curve segment formed by connecting any point in the second interface excluding the boundary and any point in the first interface has no intersection with any surface of the first three-dimensional geometric scene modeling; all points included in the line between any two points in the interface are on the same side of the interface, and the boundary of the interface is a closed curve;

[0177] A third determination submodule is used to determine the midpoint of the line between the two points with the largest distance in each of the first inlet connection surfaces as a memory carrier modeling rendering point;

[0178] A rendering submodule is used to render the first memory carrier modeling at the memory carrier modeling rendering point in the first three-dimensional geometric scene modeling based on the first semantic feature vector to obtain a first three-dimensional memory carrier set modeling.

[0179] Optionally, the third determining submodule includes:

[0180] A first determining unit, configured to determine an orthogonal plane corresponding to the first inlet connecting surface when the number of the first inlet connecting surfaces is less than the number of the first memory carrier modeling; wherein the orthogonal plane includes a line between two points with the largest distance in the first inlet connecting surface;

[0181] a segmentation unit, configured to segment the first inlet communication surface based on the orthogonal plane to obtain a plurality of second inlet communication surfaces;

[0182] The second determining unit is used to determine the midpoint of the line between two points with the largest distance in each of the second inlet connecting surfaces as the memory carrier modeling rendering point when the number of the second inlet connecting surfaces is greater than or equal to the number of the first memory carrier modeling.

[0183] Optionally, the rendering submodule includes:

[0184] A third determining unit, used to determine a word vector sequence corresponding to the first semantic feature vector;

[0185] A rendering unit is used to render the first memory carrier modeling corresponding to each word vector in the word vector sequence at the memory carrier modeling rendering point based on the sequence of each word vector in the word vector sequence to obtain a first three-dimensional memory carrier set modeling.

[0186] The three-dimensional memory carrier generating device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a GPU BOX, a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0187] The three-dimensional memory carrier generating device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, a Linux operating system, a Windows operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0188] The three-dimensional memory carrier generation device provided in the embodiment of the present application can achieve Figures 1 to 5 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0189] The present application embodiment provides an electronic device, see Figure 7 The electronic device 70 includes: a processor 701, a memory 702, and a computer program 7021 stored in the memory 702 and executable on the processor 701. When the processor 701 executes the program, the three-dimensional memory carrier generation method of the aforementioned embodiment is implemented.

[0190] The embodiment of the present application also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps in the three-dimensional memory carrier generation method disclosed in the embodiment of the present application are implemented.

[0191] The embodiment of the present application also provides a computer program product. When the computer program product is run on an electronic device, the processor implements the steps in the method for generating a three-dimensional memory carrier as disclosed in the embodiment of the present application.

[0192] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0193] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0194] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0195] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0196] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0197] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0198] The above is a detailed introduction to a resource decoupling system, execution method and device for deep learning applications provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for generating a three-dimensional memory carrier, characterized in that: The method comprises: Based on the first scene description text input by the target user, a first style feature vector and a first semantic feature vector are generated, and based on the first text to be memorized input by the target user, first keyword information is extracted; Inputting the first style feature vector and the first keyword information into a first combined neural network model to obtain first stylized keyword information output by the first combined neural network model; wherein the first combined neural network model is obtained by combining a convolutional neural network and a recurrent neural network; Generate a first memory carrier model based on the first stylized keyword information, and generate a first three-dimensional geometric scene model based on the first semantic feature vector; Based on the first semantic feature vector, the first memory carrier modeling and the first three-dimensional geometric scene modeling, a first memory carrier set modeling is generated.

2. The method according to claim 1, characterized in that The step of generating a first style feature vector and a first semantic feature vector based on a first scene description text input by a target user includes: Inputting a first scene description text input by a target user into a second combined neural network model to obtain a first style feature vector output by the second combined neural network model; wherein the second combined neural network model is obtained by combining a traditional bidirectional encoder representation model and a style transfer network model; The first scene description text is input into a visual converter model to obtain a first semantic feature vector output by the visual converter model.

3. The method according to claim 1, characterized in that The extracting first keyword information based on the first text to be memorized input by the target user includes: Inputting the first to-be-remembered text input by the target user into the extractive summarization model to obtain first summary memory information output by the extractive summarization model; Based on the attention weights between the word segments contained in the first summary memory information, the first summary information is optimized to obtain first keyword information.

4. The method according to claim 1, characterized in that: The step of generating a first memory carrier model based on the first stylized keyword information includes: generating a first virtual model based on the first stylized keyword information; When the similarity between the first real model corresponding to the first memory text and the first scene description text and the first virtual model is greater than or equal to a first threshold, the first virtual model is determined to be modeled as the first memory carrier.

5. The method according to claim 1, characterized in that The step of generating a first three-dimensional geometric scene model based on the first semantic feature vector includes: Inputting the first semantic feature vector into a conditional generative adversarial network model to obtain a geometric scene feature vector output by the conditional generative adversarial network model; Geometric constraints are applied to the geometric scene feature vector to obtain a first three-dimensional geometric scene model.

6. The method according to claim 5, characterized in that The step of geometrically constraining the geometric scene feature vector to obtain first three-dimensional geometric scene modeling includes: Inputting the first semantic feature vector into a conditional generative adversarial network model to obtain a confidence level output by the conditional generative adversarial network model; When the confidence level is greater than or equal to the second threshold, geometric constraints are performed on the geometric scene feature vector to obtain a first three-dimensional geometric scene modeling.

7. The method according to claim 1, characterized in that The generating a first memory carrier set modeling based on the first semantic feature vector, the first memory carrier modeling and the first three-dimensional geometric scene modeling includes: Based on the first interface where the entrance of the first three-dimensional geometric scene modeling is located, and the second interfaces corresponding to each of the first memory carriers, determine the first entrance connection surface contained in the first three-dimensional geometric scene modeling; wherein the first entrance connection surface means that the curve segment formed by connecting any point in the second interface excluding the boundary and any point in the first interface has no intersection with any surface of the first three-dimensional geometric scene modeling; all points contained in the line between any two points in the interface are on the same side of the interface, and the boundary of the interface is a closed curve; Determine the midpoint of the line between the two points with the largest distance in each of the first inlet connecting surfaces as the memory carrier modeling rendering point; Based on the first semantic feature vector, the first memory carrier modeling is rendered at the memory carrier modeling rendering point in the first three-dimensional geometric scene modeling to obtain a first three-dimensional memory carrier set modeling.

8. The method according to claim 7, characterized in that The step of determining the midpoint of the line between two points with the largest distance in each of the first inlet connecting surfaces as the memory carrier modeling rendering point includes: In a case where the number of the first inlet communication surfaces is less than the number of the first memory carrier modeling, determining an orthogonal plane corresponding to the first inlet communication surface; wherein the orthogonal plane includes a line between two points with the largest distance in the first inlet communication surface; Based on the orthogonal plane, the first inlet communication surface is divided to obtain a plurality of second inlet communication surfaces; When the number of the second inlet connecting surfaces is greater than or equal to the number of the first memory carrier modeling, the midpoint of the line between the two points with the largest distance in each of the second inlet connecting surfaces is determined as the memory carrier modeling rendering point.

9. The method according to claim 7, characterized in that: The step of rendering the first memory carrier modeling at the memory carrier modeling rendering point based on the first semantic feature vector to obtain a first three-dimensional memory carrier set modeling includes: Determine a word vector sequence corresponding to the first semantic feature vector; Based on the sequence of each word vector in the word vector sequence, the first memory carrier modeling corresponding to each word vector in the word vector sequence is rendered on the memory carrier modeling rendering point to obtain a first three-dimensional memory carrier set modeling.

10. A three-dimensional memory carrier generating device, characterized in that: The device comprises: A generating and extracting module is used to generate a first style feature vector and a first semantic feature vector based on a first scene description text input by a target user, and to extract first keyword information based on a first text to be memorized input by the target user; An input module, used for inputting the first style feature vector and the first keyword information into a first combined neural network model to obtain the first stylized keyword information output by the first combined neural network model; wherein the first combined neural network model is obtained by combining a convolutional neural network and a recurrent neural network; A first generating module, configured to generate a first memory carrier modeling based on the first stylized keyword information, and generate a first three-dimensional geometric scene modeling based on the first semantic feature vector; The second generating module is used to generate a first memory carrier set modeling based on the first semantic feature vector, the first memory carrier modeling and the first three-dimensional geometric scene modeling.

Images